2023-10-06T21:43:57.607 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T21:43:57.608 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} --verbose --owner scheduled_rishabh@teuthology --name rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695 -- /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/orig.config.yaml 2023-10-06T21:43:57.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T21:43:57.761 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695 branch: rishabh-cephfs-mon description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7414695' kernel: client: branch: testing flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-cephfs-mon ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_client_delegate_inos_pct: 50 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 14d8d032507701c76ca224e769ee65a96b660a8c subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 14d8d032507701c76ca224e769ee65a96b660a8c kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-cephfs-mon sha1: 14d8d032507701c76ca224e769ee65a96b660a8c owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7175 sha1: 14d8d032507701c76ca224e769ee65a96b660a8c sleep_before_teardown: 0 subset: 59/512 suite: fs suite_branch: rishabh-cephfs-mon suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 14d8d032507701c76ca224e769ee65a96b660a8c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-06_11:56:52 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T21:43:57.762 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa; will attempt to use it 2023-10-06T21:43:57.763 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks 2023-10-06T21:43:57.763 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T21:43:57.765 INFO:teuthology.task.internal:Checking packages... 2023-10-06T21:43:57.787 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '14d8d032507701c76ca224e769ee65a96b660a8c' 2023-10-06T21:43:57.787 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T21:43:57.787 INFO:teuthology.packaging:ref: None 2023-10-06T21:43:57.787 INFO:teuthology.packaging:tag: None 2023-10-06T21:43:57.787 INFO:teuthology.packaging:branch: rishabh-cephfs-mon 2023-10-06T21:43:57.787 INFO:teuthology.packaging:sha1: 14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T21:43:57.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-cephfs-mon 2023-10-06T21:43:57.934 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6616.g14d8d032 2023-10-06T21:43:57.934 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T21:43:57.945 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T21:43:57.945 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T21:43:57.996 INFO:teuthology.task.internal:Saving configuration 2023-10-06T21:43:58.017 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T21:43:58.030 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T21:43:58.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 21:37:23.215983', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:43:58.074 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 21:37:23.212972', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:43:58.099 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 21:37:23.214517', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:43:58.099 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T21:43:58.113 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-06T21:43:58.113 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-06T21:43:58.114 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-06T21:43:58.114 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T21:43:58.251 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1716686af0>, signals=[15]) 2023-10-06T21:43:58.251 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T21:43:58.263 INFO:teuthology.task.internal:Opening connections... 2023-10-06T21:43:58.263 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T21:43:58.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:43:58.333 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-10-06T21:43:58.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:43:58.401 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2023-10-06T21:43:58.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:43:58.468 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T21:43:58.477 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2023-10-06T21:43:58.500 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2023-10-06T21:43:58.500 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2023-10-06T21:43:58.561 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T21:43:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T21:43:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:ID="rhel" 2023-10-06T21:43:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="fedora" 2023-10-06T21:43:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="8.6" 2023-10-06T21:43:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el8" 2023-10-06T21:43:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T21:43:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T21:43:58.564 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2023-10-06T21:43:58.589 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-10-06T21:43:58.610 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-10-06T21:43:58.610 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-10-06T21:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T21:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T21:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-10-06T21:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-10-06T21:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-10-06T21:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-10-06T21:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T21:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-10-06T21:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T21:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T21:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T21:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T21:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T21:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T21:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T21:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T21:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T21:43:58.673 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-10-06T21:43:58.697 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-10-06T21:43:58.718 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-10-06T21:43:58.718 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-10-06T21:43:58.777 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T21:43:58.777 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T21:43:58.777 INFO:teuthology.orchestra.run.smithi190.stdout:ID="rhel" 2023-10-06T21:43:58.777 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="fedora" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8.6" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T21:43:58.778 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T21:43:58.779 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T21:43:58.779 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T21:43:58.779 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T21:43:58.779 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-10-06T21:43:58.797 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T21:43:58.812 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T21:43:58.848 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T21:43:58.849 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2023-10-06T21:43:58.851 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-10-06T21:43:58.853 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2023-10-06T21:43:58.871 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T21:43:58.886 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T21:43:58.886 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2023-10-06T21:43:58.911 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-10-06T21:43:58.913 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2023-10-06T21:43:58.945 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T21:43:59.019 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T21:43:59.036 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-10-06T21:43:59.036 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-06T21:43:59.037 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-06T21:43:59.037 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T21:43:59.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:43:59.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T21:43:59.302 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T21:43:59.302 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T21:43:59.302 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-10-06T21:43:59.322 INFO:teuthology.orchestra.run.smithi057.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-06T21:43:59.322 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T21:43:59.322 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T21:43:59.323 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum upgrade -y linux-firmware 2023-10-06T21:44:00.522 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-06T21:44:00.523 INFO:teuthology.orchestra.run.smithi057.stdout:Unable to read consumer identity 2023-10-06T21:44:02.087 INFO:teuthology.orchestra.run.smithi057.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-10-06T21:44:02.763 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-10-06T21:44:06.530 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-10-06T21:44:07.306 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-06T21:44:07.307 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2023-10-06T21:44:07.307 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-06T21:44:07.417 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T21:44:07.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T21:44:07.566 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T21:44:07.566 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T21:44:07.568 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi057.front.sepia.ceph.com to complete... 2023-10-06T21:44:20.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06 21:44:20 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T21:44:20.422 INFO:teuthology.orchestra.run.smithi057.stderr:FINISHED --2023-10-06 21:44:20-- 2023-10-06T21:44:20.422 INFO:teuthology.orchestra.run.smithi057.stderr:Total wall clock time: 13s 2023-10-06T21:44:20.422 INFO:teuthology.orchestra.run.smithi057.stderr:Downloaded: 1 files, 363M in 13s (28.9 MB/s) 2023-10-06T21:44:20.423 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T21:44:20.423 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T21:44:22.085 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying... ######################################## 2023-10-06T21:44:22.142 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing... ######################################## 2023-10-06T21:44:22.228 INFO:teuthology.orchestra.run.smithi057.stdout:Updating / installing... 2023-10-06T21:45:02.871 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T21:45:27.685 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T21:45:27.685 DEBUG:teuthology.orchestra.run.smithi057:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T21:45:27.740 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.740 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.740 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.740 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T21:45:27.741 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T21:45:27.742 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T21:45:27.743 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T21:45:27.744 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T21:45:27.745 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T21:45:27.746 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T21:45:27.747 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T21:45:27.748 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T21:45:27.749 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T21:45:27.750 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T21:45:27.751 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T21:45:27.752 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T21:45:27.753 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T21:45:27.754 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T21:45:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T21:45:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T21:45:27.757 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T21:45:27.758 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T21:45:27.759 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T21:45:27.760 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T21:45:27.761 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T21:45:27.762 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T21:45:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T21:45:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T21:45:27.765 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T21:45:27.766 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T21:45:27.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T21:45:27.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T21:45:27.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T21:45:27.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T21:45:27.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T21:45:27.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T21:45:27.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T21:45:27.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T21:45:27.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T21:45:27.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T21:45:27.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T21:45:27.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T21:45:27.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T21:45:27.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T21:45:27.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T21:45:27.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T21:45:27.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T21:45:27.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T21:45:27.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T21:45:27.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T21:45:27.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T21:45:27.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T21:45:27.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T21:45:27.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T21:45:27.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T21:45:27.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T21:45:27.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T21:45:27.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T21:45:27.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T21:45:27.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T21:45:27.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T21:45:27.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T21:45:27.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T21:45:27.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T21:45:27.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T21:45:27.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T21:45:27.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T21:45:27.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T21:45:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T21:45:27.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T21:45:27.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T21:45:27.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T21:45:27.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T21:45:27.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T21:45:27.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T21:45:27.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T21:45:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T21:45:27.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T21:45:27.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T21:45:27.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T21:45:27.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T21:45:27.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T21:45:27.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T21:45:27.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T21:45:27.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T21:45:27.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T21:45:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T21:45:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T21:45:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T21:45:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T21:45:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T21:45:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T21:45:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T21:45:27.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T21:45:27.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T21:45:27.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T21:45:27.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T21:45:27.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T21:45:27.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T21:45:27.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T21:45:27.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T21:45:27.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T21:45:27.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T21:45:27.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T21:45:27.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T21:45:27.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T21:45:27.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T21:45:27.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T21:45:27.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T21:45:27.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T21:45:27.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T21:45:27.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T21:45:27.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T21:45:27.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T21:45:27.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T21:45:27.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T21:45:27.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T21:45:27.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T21:45:27.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T21:45:27.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T21:45:27.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T21:45:27.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T21:45:27.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T21:45:27.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T21:45:27.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T21:45:27.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T21:45:27.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T21:45:27.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T21:45:27.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T21:45:27.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T21:45:27.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T21:45:27.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T21:45:27.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T21:45:27.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T21:45:27.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T21:45:27.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T21:45:27.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T21:45:27.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T21:45:27.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T21:45:27.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T21:45:27.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T21:45:27.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T21:45:27.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T21:45:27.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T21:45:27.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T21:45:27.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T21:45:27.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T21:45:27.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T21:45:27.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T21:45:27.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T21:45:27.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T21:45:27.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T21:45:27.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T21:45:27.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T21:45:27.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T21:45:27.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T21:45:27.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T21:45:27.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T21:45:27.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T21:45:27.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T21:45:27.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T21:45:27.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T21:45:27.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T21:45:27.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T21:45:27.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T21:45:27.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T21:45:27.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T21:45:27.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T21:45:27.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T21:45:27.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T21:45:27.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T21:45:27.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T21:45:27.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T21:45:27.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T21:45:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T21:45:27.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T21:45:27.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T21:45:27.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T21:45:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T21:45:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T21:45:27.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T21:45:27.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T21:45:27.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T21:45:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T21:45:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T21:45:27.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T21:45:27.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T21:45:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T21:45:27.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T21:45:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T21:45:27.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T21:45:27.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T21:45:27.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T21:45:27.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T21:45:27.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T21:45:27.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T21:45:27.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T21:45:27.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T21:45:27.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T21:45:27.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T21:45:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T21:45:27.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T21:45:27.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T21:45:27.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T21:45:27.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T21:45:27.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T21:45:27.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T21:45:27.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T21:45:27.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T21:45:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T21:45:27.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T21:45:27.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T21:45:27.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T21:45:27.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T21:45:27.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T21:45:27.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T21:45:27.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T21:45:27.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T21:45:27.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T21:45:27.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T21:45:27.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T21:45:27.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T21:45:27.911 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T21:45:27.912 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T21:45:27.913 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T21:45:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T21:45:27.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T21:45:27.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T21:45:27.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T21:45:27.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T21:45:27.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T21:45:27.915 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T21:45:27.915 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.916 DEBUG:teuthology.orchestra.run.smithi057:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T21:45:27.943 INFO:teuthology.orchestra.run.smithi057.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T21:45:27.943 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T21:45:27.944 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.945 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2023-10-06T21:45:27.945 INFO:teuthology.orchestra.run.smithi057.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T21:45:27.945 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T21:45:27.945 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:27.945 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2023-10-06T21:45:27.946 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -qi grub2-tools 2023-10-06T21:45:28.026 INFO:teuthology.orchestra.run.smithi057.stdout:Name : grub2-tools 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Epoch : 1 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Version : 2.02 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Release : 123.el8 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: x86_64 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Group : System Environment/Base 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Size : 9265450 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:License : GPLv3+ 2023-10-06T21:45:28.027 INFO:teuthology.orchestra.run.smithi057.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-06T21:45:28.028 INFO:teuthology.orchestra.run.smithi057.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-06T21:45:28.028 INFO:teuthology.orchestra.run.smithi057.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-06T21:45:28.028 INFO:teuthology.orchestra.run.smithi057.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-06T21:45:28.028 INFO:teuthology.orchestra.run.smithi057.stdout:Relocations : (not relocatable) 2023-10-06T21:45:28.028 INFO:teuthology.orchestra.run.smithi057.stdout:Packager : Red Hat, Inc. 2023-10-06T21:45:28.028 INFO:teuthology.orchestra.run.smithi057.stdout:Vendor : Red Hat, Inc. 2023-10-06T21:45:28.029 INFO:teuthology.orchestra.run.smithi057.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T21:45:28.029 INFO:teuthology.orchestra.run.smithi057.stdout:Summary : Support tools for GRUB. 2023-10-06T21:45:28.029 INFO:teuthology.orchestra.run.smithi057.stdout:Description : 2023-10-06T21:45:28.029 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T21:45:28.029 INFO:teuthology.orchestra.run.smithi057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T21:45:28.029 INFO:teuthology.orchestra.run.smithi057.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T21:45:28.030 INFO:teuthology.orchestra.run.smithi057.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T21:45:28.030 INFO:teuthology.orchestra.run.smithi057.stdout:hardware devices. 2023-10-06T21:45:28.030 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T21:45:28.030 INFO:teuthology.orchestra.run.smithi057.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T21:45:28.031 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T21:45:28.031 INFO:teuthology.task.kernel:Updating grub on smithi057 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:28.032 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T21:45:32.507 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2023-10-06T21:45:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:done 2023-10-06T21:45:34.745 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2023-10-06T21:45:34.761 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.SLmVnUS4D8 2023-10-06T21:45:34.762 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SLmVnUS4D8 2023-10-06T21:45:34.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.SLmVnUS4D8 2023-10-06T21:45:34.975 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.SLmVnUS4D8 is 6KB 2023-10-06T21:45:35.024 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.SLmVnUS4D8 2023-10-06T21:45:35.041 DEBUG:teuthology.orchestra.run.smithi057:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T21:45:35.106 INFO:teuthology.orchestra.run.smithi057.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-06T21:45:35.106 INFO:teuthology.orchestra.run.smithi057.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-06T21:45:35.106 INFO:teuthology.orchestra.run.smithi057.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T21:45:35.108 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-10-06T21:45:35.273 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2023-10-06T21:46:05.276 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T21:46:05.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-06T21:46:05.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:46:23.738 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2023-10-06T21:46:32.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-06T21:46:32.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:46:35.802 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2023-10-06T21:46:47.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-06T21:46:47.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:47:47.805 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T21:48:02.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-06T21:48:02.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T21:48:03.154 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-10-06T21:48:03.581 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-06T21:48:03.581 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T21:48:03.581 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-10-06T21:48:03.624 INFO:teuthology.orchestra.run.smithi057.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T21:48:03.624 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T21:48:03.624 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T21:48:03.624 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T21:48:04.625 DEBUG:teuthology.parallel:result is None 2023-10-06T21:48:04.625 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T21:48:04.632 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T21:48:04.632 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T21:48:04.637 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T21:48:04.640 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T21:48:04.681 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T21:48:04.688 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T21:48:04.723 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T21:48:04.724 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T21:48:04.728 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T21:48:04.732 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T21:48:04.782 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T21:48:04.790 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T21:48:04.790 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T21:48:04.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:48:04.819 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T21:48:04.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:48:04.838 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T21:48:04.843 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T21:48:04.862 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T21:48:04.900 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T21:48:04.917 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T21:48:04.944 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T21:48:04.957 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T21:48:04.959 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T21:48:04.991 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T21:48:04.994 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T21:48:05.001 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T21:48:05.002 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T21:48:05.006 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T21:48:05.008 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T21:48:05.071 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T21:48:05.081 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T21:48:05.082 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T21:48:05.086 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T21:48:05.088 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T21:48:05.124 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T21:48:05.234 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T21:48:05.320 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T21:48:05.390 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T21:48:05.474 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T21:48:05.474 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T21:48:05.542 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T21:48:05.589 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T21:48:05.644 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T21:48:05.726 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T21:48:05.775 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T21:48:05.775 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T21:48:05.845 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T21:48:05.894 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T21:48:05.947 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T21:48:06.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T21:48:06.078 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T21:48:06.078 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T21:48:06.148 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2023-10-06T21:48:06.153 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-10-06T21:48:06.155 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2023-10-06T21:48:06.258 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:48:06.268 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:48:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:48:06.707 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T21:48:06.718 INFO:teuthology.task.internal:Starting timer... 2023-10-06T21:48:06.718 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T21:48:06.772 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T21:48:06.821 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T21:48:06.821 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2023-10-06T21:48:06.946 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2023-10-06T21:48:06.949 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-10-06T21:48:07.013 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-10-06T21:48:07.015 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2023-10-06T21:48:07.080 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2023-10-06T21:48:07.082 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T21:48:07.082 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2023-10-06T21:48:07.139 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2023-10-06T21:48:07.139 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-10-06T21:48:07.162 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-10-06T21:48:07.163 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2023-10-06T21:48:07.184 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2023-10-06T21:48:07.185 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2023-10-06T21:48:07.185 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:48:07.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:48:07.262 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:48:07.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:48:07.289 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:48:07.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:48:07.316 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T21:48:07.316 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2023-10-06T21:48:07.389 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-10-06T21:48:07.416 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2023-10-06T21:48:07.443 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T21:48:07.461 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T21:48:07.461 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T21:48:07.479 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T21:48:07.480 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi156.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T22:01:09.172 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2023-10-06T22:01:09.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-06T22:01:09.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T22:01:09.282 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-10-06T22:01:09.406 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-06T22:01:09.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-10-06T22:01:09.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T22:01:09.475 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-10-06T22:01:09.549 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-10-06T22:01:09.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-10-06T22:01:09.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T22:01:09.619 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-10-06T22:01:09.648 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-10-06T22:01:09.649 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T22:01:09.660 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T22:01:09.661 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T22:01:09.661 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T22:01:09.666 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T22:01:09.666 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T22:01:09.668 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T22:01:09.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T22:01:09.701 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T22:01:09.716 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T22:01:09.723 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T22:01:09.738 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T22:01:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T22:01:09.775 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T22:01:09.796 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-10-06T22:01:09.808 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-10-06T22:01:09.810 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2023-10-06T22:01:09.822 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2023-10-06T22:01:09.823 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T22:01:09.837 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T22:01:09.838 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T22:01:09.852 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T22:01:09.865 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2023-10-06T22:01:09.895 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2023-10-06T22:01:09.913 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-10-06T22:01:09.917 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T22:01:09.917 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-10-06T22:01:09.917 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.918 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.918 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.918 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.927 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-10-06T22:01:09.931 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T22:01:09.931 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-10-06T22:01:09.931 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.931 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.932 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.932 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:09.932 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T22:01:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T22:01:10.056 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-10-06T22:01:10.064 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T22:01:10.064 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-10-06T22:01:10.064 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:10.064 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:10.064 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:10.064 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:01:10.065 INFO:teuthology.run_tasks:Running task pexec... 2023-10-06T22:01:10.076 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-06T22:01:10.076 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:01:10.077 DEBUG:teuthology.orchestra.run.smithi057:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T22:01:10.077 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi156.front.sepia.ceph.com 2023-10-06T22:01:10.077 DEBUG:teuthology.orchestra.run.smithi156:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T22:01:10.078 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi190.front.sepia.ceph.com 2023-10-06T22:01:10.078 DEBUG:teuthology.orchestra.run.smithi190:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T22:01:10.448 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:10.454 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:10.595 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:11.274 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:05:27 ago on Fri 06 Oct 2023 09:55:43 PM UTC. 2023-10-06T22:01:11.289 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:05:29 ago on Fri 06 Oct 2023 09:55:41 PM UTC. 2023-10-06T22:01:11.588 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:52 ago on Fri 06 Oct 2023 09:59:19 PM UTC. 2023-10-06T22:01:13.027 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-10-06T22:01:13.028 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-10-06T22:01:13.028 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2023-10-06T22:01:13.028 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-10-06T22:01:13.028 INFO:teuthology.orchestra.run.smithi156.stdout:Resetting modules: 2023-10-06T22:01:13.028 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools 2023-10-06T22:01:13.029 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:01:13.029 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-10-06T22:01:13.029 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-10-06T22:01:13.029 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:01:13.159 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-10-06T22:01:13.160 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2023-10-06T22:01:13.160 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2023-10-06T22:01:13.160 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2023-10-06T22:01:13.160 INFO:teuthology.orchestra.run.smithi190.stdout:Resetting modules: 2023-10-06T22:01:13.160 INFO:teuthology.orchestra.run.smithi190.stdout: container-tools 2023-10-06T22:01:13.160 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:01:13.160 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-10-06T22:01:13.161 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2023-10-06T22:01:13.161 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:01:13.675 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-06T22:01:13.676 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-06T22:01:13.676 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2023-10-06T22:01:13.677 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-06T22:01:13.677 INFO:teuthology.orchestra.run.smithi057.stdout:Resetting modules: 2023-10-06T22:01:13.677 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools 2023-10-06T22:01:13.677 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:01:13.677 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-06T22:01:13.677 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-06T22:01:13.677 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:01:14.038 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:05:30 ago on Fri 06 Oct 2023 09:55:43 PM UTC. 2023-10-06T22:01:14.122 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:05:32 ago on Fri 06 Oct 2023 09:55:41 PM UTC. 2023-10-06T22:01:14.965 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:55 ago on Fri 06 Oct 2023 09:59:19 PM UTC. 2023-10-06T22:01:16.753 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-10-06T22:01:16.908 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-10-06T22:01:17.278 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:17.372 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:17.877 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-06T22:01:18.108 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:05:34 ago on Fri 06 Oct 2023 09:55:43 PM UTC. 2023-10-06T22:01:18.210 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:05:36 ago on Fri 06 Oct 2023 09:55:41 PM UTC. 2023-10-06T22:01:18.506 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:19.482 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:59 ago on Fri 06 Oct 2023 09:59:19 PM UTC. 2023-10-06T22:01:19.815 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-10-06T22:01:19.909 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout:Installing group/module packages: 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-10-06T22:01:19.910 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout:Installing module profiles: 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools/common 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools rhel8 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout:Install 10 Packages 2023-10-06T22:01:19.911 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:01:19.912 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 18 M 2023-10-06T22:01:19.912 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 65 M 2023-10-06T22:01:19.913 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-10-06T22:01:19.929 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-10-06T22:01:19.987 INFO:teuthology.orchestra.run.smithi156.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-10-06T22:01:20.012 INFO:teuthology.orchestra.run.smithi156.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 248 kB/s | 25 kB 00:00 2023-10-06T22:01:20.020 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================================ 2023-10-06T22:01:20.020 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2023-10-06T22:01:20.020 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================================ 2023-10-06T22:01:20.020 INFO:teuthology.orchestra.run.smithi190.stdout:Installing group/module packages: 2023-10-06T22:01:20.020 INFO:teuthology.orchestra.run.smithi190.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T22:01:20.021 INFO:teuthology.orchestra.run.smithi190.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout:Installing module profiles: 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout: container-tools/common 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling module streams: 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout: container-tools rhel8 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================================ 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout:Install 10 Packages 2023-10-06T22:01:20.022 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:01:20.023 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 18 M 2023-10-06T22:01:20.023 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 65 M 2023-10-06T22:01:20.023 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2023-10-06T22:01:20.037 INFO:teuthology.orchestra.run.smithi156.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 326 kB/s | 41 kB 00:00 2023-10-06T22:01:20.062 INFO:teuthology.orchestra.run.smithi156.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 656 kB/s | 49 kB 00:00 2023-10-06T22:01:20.094 INFO:teuthology.orchestra.run.smithi190.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-10-06T22:01:20.096 INFO:teuthology.orchestra.run.smithi156.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-06T22:01:20.119 INFO:teuthology.orchestra.run.smithi190.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 257 kB/s | 25 kB 00:00 2023-10-06T22:01:20.121 INFO:teuthology.orchestra.run.smithi156.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-06T22:01:20.145 INFO:teuthology.orchestra.run.smithi190.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 335 kB/s | 41 kB 00:00 2023-10-06T22:01:20.170 INFO:teuthology.orchestra.run.smithi190.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-10-06T22:01:20.171 INFO:teuthology.orchestra.run.smithi156.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-10-06T22:01:20.203 INFO:teuthology.orchestra.run.smithi190.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-06T22:01:20.229 INFO:teuthology.orchestra.run.smithi190.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-06T22:01:20.254 INFO:teuthology.orchestra.run.smithi190.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-06T22:01:20.272 INFO:teuthology.orchestra.run.smithi156.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-10-06T22:01:20.338 INFO:teuthology.orchestra.run.smithi190.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-10-06T22:01:20.389 INFO:teuthology.orchestra.run.smithi156.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 26 MB/s | 7.0 MB 00:00 2023-10-06T22:01:20.472 INFO:teuthology.orchestra.run.smithi156.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-10-06T22:01:20.473 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-10-06T22:01:20.473 INFO:teuthology.orchestra.run.smithi156.stdout:Total 32 MB/s | 18 MB 00:00 2023-10-06T22:01:20.488 INFO:teuthology.orchestra.run.smithi190.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-10-06T22:01:20.580 INFO:teuthology.orchestra.run.smithi190.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-10-06T22:01:20.580 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2023-10-06T22:01:20.580 INFO:teuthology.orchestra.run.smithi190.stdout:Total 33 MB/s | 18 MB 00:00 2023-10-06T22:01:20.673 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-10-06T22:01:20.692 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-10-06T22:01:20.692 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-10-06T22:01:20.781 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2023-10-06T22:01:20.800 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2023-10-06T22:01:20.800 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2023-10-06T22:01:20.873 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-10-06T22:01:20.876 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-10-06T22:01:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2023-10-06T22:01:20.986 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2023-10-06T22:01:21.374 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-10-06T22:01:21.525 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2023-10-06T22:01:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T22:01:21.603 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-06T22:01:21.708 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================================================ 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================================================ 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout:Installing group/module packages: 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T22:01:21.709 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout:Installing module profiles: 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools/common 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling module streams: 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools rhel8 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================================================ 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout:Install 10 Packages 2023-10-06T22:01:21.710 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:01:21.711 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 18 M 2023-10-06T22:01:21.711 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 65 M 2023-10-06T22:01:21.711 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-10-06T22:01:21.732 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T22:01:21.753 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T22:01:21.787 INFO:teuthology.orchestra.run.smithi057.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-10-06T22:01:21.812 INFO:teuthology.orchestra.run.smithi057.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 249 kB/s | 25 kB 00:00 2023-10-06T22:01:21.837 INFO:teuthology.orchestra.run.smithi057.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 325 kB/s | 41 kB 00:00 2023-10-06T22:01:21.862 INFO:teuthology.orchestra.run.smithi057.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-10-06T22:01:21.896 INFO:teuthology.orchestra.run.smithi057.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-06T22:01:21.922 INFO:teuthology.orchestra.run.smithi057.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-06T22:01:21.922 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T22:01:21.952 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T22:01:21.955 INFO:teuthology.orchestra.run.smithi057.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-10-06T22:01:22.072 INFO:teuthology.orchestra.run.smithi057.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-10-06T22:01:22.117 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T22:01:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T22:01:22.239 INFO:teuthology.orchestra.run.smithi057.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 22 MB/s | 7.0 MB 00:00 2023-10-06T22:01:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T22:01:22.447 INFO:teuthology.orchestra.run.smithi057.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 13 MB/s | 8.0 MB 00:00 2023-10-06T22:01:22.448 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-06T22:01:22.448 INFO:teuthology.orchestra.run.smithi057.stdout:Total 25 MB/s | 18 MB 00:00 2023-10-06T22:01:22.465 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T22:01:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T22:01:22.791 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-10-06T22:01:22.815 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-10-06T22:01:22.816 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-10-06T22:01:23.259 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T22:01:23.309 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-10-06T22:01:23.313 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-10-06T22:01:23.502 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T22:01:23.998 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-10-06T22:01:24.135 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T22:01:24.166 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T22:01:24.357 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T22:01:24.381 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T22:01:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T22:01:24.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T22:01:24.586 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T22:01:24.617 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T22:01:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T22:01:24.816 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T22:01:24.837 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T22:01:24.988 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T22:01:25.112 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T22:01:25.112 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T22:01:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T22:01:25.199 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T22:01:25.245 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T22:01:25.245 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T22:01:25.245 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T22:01:25.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T22:01:25.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T22:01:25.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T22:01:25.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T22:01:25.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T22:01:25.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T22:01:25.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T22:01:26.046 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T22:01:26.306 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T22:01:26.306 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:05:42 ago on Fri 06 Oct 2023 09:55:43 PM UTC. 2023-10-06T22:01:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T22:01:26.448 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:05:44 ago on Fri 06 Oct 2023 09:55:41 PM UTC. 2023-10-06T22:01:27.025 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T22:01:27.302 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T22:01:27.471 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T22:01:27.661 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T22:01:28.059 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T22:01:28.060 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T22:01:28.061 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T22:01:29.475 INFO:teuthology.orchestra.run.smithi190.stdout:Installed products updated. 2023-10-06T22:01:29.505 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T22:01:29.506 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:02:09 ago on Fri 06 Oct 2023 09:59:19 PM UTC. 2023-10-06T22:01:29.607 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T22:01:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T22:01:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:01:29.694 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T22:01:29.813 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T22:01:29.814 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.814 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:29.814 INFO:teuthology.orchestra.run.smithi156.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T22:01:29.814 INFO:teuthology.orchestra.run.smithi156.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T22:01:29.814 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:01:29.814 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-10-06T22:01:29.911 DEBUG:teuthology.parallel:result is None 2023-10-06T22:01:29.983 DEBUG:teuthology.parallel:result is None 2023-10-06T22:01:34.609 INFO:teuthology.orchestra.run.smithi057.stdout:Installed products updated. 2023-10-06T22:01:34.831 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:01:34.831 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-10-06T22:01:34.831 INFO:teuthology.orchestra.run.smithi057.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:34.831 INFO:teuthology.orchestra.run.smithi057.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T22:01:34.831 INFO:teuthology.orchestra.run.smithi057.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:01:34.832 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-06T22:01:35.038 DEBUG:teuthology.parallel:result is None 2023-10-06T22:01:35.038 INFO:teuthology.run_tasks:Running task install... 2023-10-06T22:01:35.050 DEBUG:teuthology.task.install:project ceph 2023-10-06T22:01:35.050 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c'}} 2023-10-06T22:01:35.050 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c'} 2023-10-06T22:01:35.050 INFO:teuthology.task.install:Using flavor: default 2023-10-06T22:01:35.057 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T22:01:35.057 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T22:01:35.057 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T22:01:35.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:01:35.060 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T22:01:35.060 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:01:35.061 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T22:01:35.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:01:35.200 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T22:01:35.200 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T22:01:35.206 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T22:01:35.206 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T22:01:35.279 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T22:01:35.279 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T22:01:35.313 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T22:01:35.313 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:01:35.313 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T22:01:35.340 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T22:01:35.340 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T22:01:35.381 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T22:01:35.383 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:01:35.383 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T22:01:35.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T22:01:35.411 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T22:01:35.411 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T22:01:35.469 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T22:01:35.469 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:01:35.469 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T22:01:35.480 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T22:01:35.491 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T22:01:35.541 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T22:01:35.541 DEBUG:teuthology.orchestra.run.smithi057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T22:01:35.556 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-10-06T22:01:35.557 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T22:01:35.558 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-10-06T22:01:35.623 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2023-10-06T22:01:35.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2023-10-06T22:01:35.652 DEBUG:teuthology.orchestra.run.smithi057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T22:01:35.750 DEBUG:teuthology.orchestra.run.smithi057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T22:01:35.818 INFO:teuthology.orchestra.run.smithi057.stdout:check_obsoletes = 1 2023-10-06T22:01:35.821 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum clean all 2023-10-06T22:01:35.978 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:36.046 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:36.303 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:37.109 INFO:teuthology.orchestra.run.smithi190.stdout:44 files removed 2023-10-06T22:01:37.110 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-10-06T22:01:37.157 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T22:01:37.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T22:01:37.470 INFO:teuthology.orchestra.run.smithi057.stdout:44 files removed 2023-10-06T22:01:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:37.516 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:37.528 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T22:01:38.005 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-06T22:01:38.978 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 153 kB/s | 81 kB 00:00 2023-10-06T22:01:38.978 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 155 kB/s | 81 kB 00:00 2023-10-06T22:01:39.448 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 22 kB/s | 9.4 kB 00:00 2023-10-06T22:01:39.464 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 21 kB/s | 9.4 kB 00:00 2023-10-06T22:01:39.649 INFO:teuthology.orchestra.run.smithi057.stdout:ceph packages for x86_64 250 kB/s | 81 kB 00:00 2023-10-06T22:01:39.884 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 3.6 kB/s | 1.5 kB 00:00 2023-10-06T22:01:39.952 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 3.2 kB/s | 1.5 kB 00:00 2023-10-06T22:01:40.085 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-10-06T22:01:40.102 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-10-06T22:01:40.133 INFO:teuthology.orchestra.run.smithi057.stdout:ceph noarch packages 21 kB/s | 9.4 kB 00:00 2023-10-06T22:01:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:ceph source packages 4.0 kB/s | 1.5 kB 00:00 2023-10-06T22:01:40.695 INFO:teuthology.orchestra.run.smithi057.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-06T22:01:40.795 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-10-06T22:01:41.310 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 27 MB/s | 16 MB 00:00 2023-10-06T22:01:42.841 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 16 MB 00:02 2023-10-06T22:01:44.386 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 208 kB/s | 24 kB 00:00 2023-10-06T22:01:45.604 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 53 MB 00:01 2023-10-06T22:01:45.652 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 431 kB/s | 24 kB 00:00 2023-10-06T22:01:46.391 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 299 kB/s | 24 kB 00:00 2023-10-06T22:01:46.937 INFO:teuthology.orchestra.run.smithi057.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 53 MB 00:01 2023-10-06T22:01:47.368 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 52 MB/s | 47 MB 00:00 2023-10-06T22:01:53.697 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-10-06T22:01:55.945 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 MB/s | 53 MB 00:01 2023-10-06T22:01:56.080 INFO:teuthology.orchestra.run.smithi057.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 47 MB 00:01 2023-10-06T22:02:04.780 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T22:02:04.781 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T22:02:04.796 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T22:02:04.797 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T22:02:04.798 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T22:02:04.799 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T22:02:04.801 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T22:02:04.805 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T22:02:04.806 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T22:02:04.935 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-10-06T22:02:04.947 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T22:02:04.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T22:02:04.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T22:02:04.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T22:02:04.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T22:02:04.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T22:02:04.949 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T22:02:04.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T22:02:04.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T22:02:04.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T22:02:04.950 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T22:02:04.950 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T22:02:04.950 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T22:02:04.951 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T22:02:04.951 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T22:02:04.951 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T22:02:04.951 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T22:02:04.951 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T22:02:04.951 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T22:02:04.952 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T22:02:04.952 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T22:02:04.952 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T22:02:04.952 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T22:02:04.952 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T22:02:04.952 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T22:02:04.952 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T22:02:04.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T22:02:04.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T22:02:04.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T22:02:04.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T22:02:04.953 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T22:02:04.953 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T22:02:04.954 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T22:02:04.954 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T22:02:04.954 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-10-06T22:02:04.954 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T22:02:04.954 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T22:02:04.954 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T22:02:04.955 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-10-06T22:02:04.955 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T22:02:04.955 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T22:02:04.955 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T22:02:04.955 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T22:02:04.956 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T22:02:04.956 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T22:02:04.956 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T22:02:04.956 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T22:02:04.956 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T22:02:04.956 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T22:02:04.957 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T22:02:04.957 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T22:02:04.957 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T22:02:04.957 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T22:02:04.957 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T22:02:04.957 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T22:02:04.957 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T22:02:04.958 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T22:02:04.958 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T22:02:04.958 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T22:02:04.958 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T22:02:04.958 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T22:02:04.958 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T22:02:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T22:02:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T22:02:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T22:02:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T22:02:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T22:02:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T22:02:04.959 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T22:02:04.960 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T22:02:04.960 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T22:02:04.960 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T22:02:04.960 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T22:02:04.960 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T22:02:04.960 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T22:02:04.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T22:02:04.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T22:02:04.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T22:02:04.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T22:02:04.961 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T22:02:04.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T22:02:04.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T22:02:04.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T22:02:04.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T22:02:04.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T22:02:04.962 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T22:02:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T22:02:04.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T22:02:04.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T22:02:04.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T22:02:04.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T22:02:04.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T22:02:04.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T22:02:04.964 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T22:02:04.965 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T22:02:04.965 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T22:02:04.965 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T22:02:04.965 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T22:02:04.965 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T22:02:04.965 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T22:02:04.965 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T22:02:04.966 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T22:02:04.967 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T22:02:04.967 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T22:02:04.967 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T22:02:04.967 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T22:02:04.967 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T22:02:04.967 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T22:02:04.968 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-10-06T22:02:04.969 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:02:04.970 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 208 M 2023-10-06T22:02:04.970 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-10-06T22:02:05.273 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 32 kB/s | 6.4 kB 00:00 2023-10-06T22:02:05.607 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 2.7 MB/s | 918 kB 00:00 2023-10-06T22:02:05.691 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 2.0 MB/s | 167 kB 00:00 2023-10-06T22:02:05.808 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 7.4 MB/s | 5.4 MB 00:00 2023-10-06T22:02:06.041 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 7.0 MB/s | 1.6 MB 00:00 2023-10-06T22:02:06.183 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 4.7 MB/s | 2.3 MB 00:00 2023-10-06T22:02:06.542 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 9.2 MB/s | 4.6 MB 00:00 2023-10-06T22:02:06.959 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T22:02:06.960 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T22:02:06.974 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T22:02:06.975 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T22:02:06.977 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T22:02:06.978 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T22:02:06.980 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T22:02:06.986 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T22:02:06.986 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T22:02:07.034 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-common-18.0.0-6616.g14d8d032.el8. 12 MB/s | 23 MB 00:01 2023-10-06T22:02:07.101 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 367 kB/s | 24 kB 00:00 2023-10-06T22:02:07.120 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-10-06T22:02:07.132 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T22:02:07.133 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T22:02:07.134 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T22:02:07.135 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T22:02:07.136 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T22:02:07.137 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2023-10-06T22:02:07.137 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T22:02:07.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T22:02:07.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T22:02:07.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T22:02:07.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T22:02:07.138 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T22:02:07.139 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T22:02:07.140 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T22:02:07.141 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T22:02:07.142 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T22:02:07.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T22:02:07.144 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling module streams: 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql 10 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout:Install 128 Packages 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 3 Packages 2023-10-06T22:02:07.145 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:02:07.146 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 208 M 2023-10-06T22:02:07.146 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2023-10-06T22:02:07.703 INFO:teuthology.orchestra.run.smithi190.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-10-06T22:02:08.176 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 8.5 MB/s | 14 MB 00:01 2023-10-06T22:02:08.259 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 3.1 MB/s | 260 kB 00:00 2023-10-06T22:02:08.264 INFO:teuthology.orchestra.run.smithi190.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 1.6 MB/s | 918 kB 00:00 2023-10-06T22:02:08.345 INFO:teuthology.orchestra.run.smithi190.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 2.0 MB/s | 167 kB 00:00 2023-10-06T22:02:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x8 7.9 MB/s | 18 MB 00:02 2023-10-06T22:02:08.468 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs-devel-18.0.0-6616.g14d8d032 149 kB/s | 31 kB 00:00 2023-10-06T22:02:08.541 INFO:teuthology.orchestra.run.smithi057.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T22:02:08.541 INFO:teuthology.orchestra.run.smithi057.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T22:02:08.557 INFO:teuthology.orchestra.run.smithi057.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T22:02:08.558 INFO:teuthology.orchestra.run.smithi057.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T22:02:08.559 INFO:teuthology.orchestra.run.smithi057.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T22:02:08.560 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-6616.g14d8d032.e 2.0 MB/s | 183 kB 00:00 2023-10-06T22:02:08.560 INFO:teuthology.orchestra.run.smithi057.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T22:02:08.563 INFO:teuthology.orchestra.run.smithi057.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T22:02:08.566 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T22:02:08.567 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T22:02:08.594 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 5.5 MB/s | 840 kB 00:00 2023-10-06T22:02:08.621 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): librados-devel-18.0.0-6616.g14d8d032. 2.3 MB/s | 139 kB 00:00 2023-10-06T22:02:08.694 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): libradosstriper1-18.0.0-6616.g14d8d03 5.1 MB/s | 518 kB 00:00 2023-10-06T22:02:08.718 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-06T22:02:08.731 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================================================== 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================================================== 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T22:02:08.732 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T22:02:08.733 INFO:teuthology.orchestra.run.smithi057.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T22:02:08.734 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-10-06T22:02:08.735 INFO:teuthology.orchestra.run.smithi057.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T22:02:08.736 INFO:teuthology.orchestra.run.smithi057.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T22:02:08.737 INFO:teuthology.orchestra.run.smithi057.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T22:02:08.738 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T22:02:08.739 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T22:02:08.740 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T22:02:08.741 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T22:02:08.742 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling module streams: 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql 10 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-06T22:02:08.743 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================================================== 2023-10-06T22:02:08.744 INFO:teuthology.orchestra.run.smithi057.stdout:Install 128 Packages 2023-10-06T22:02:08.744 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 3 Packages 2023-10-06T22:02:08.744 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:02:08.744 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 208 M 2023-10-06T22:02:08.744 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-10-06T22:02:08.795 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 446 kB/s | 45 kB 00:00 2023-10-06T22:02:08.846 INFO:teuthology.orchestra.run.smithi190.stdout:(4/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 4.6 MB/s | 2.3 MB 00:00 2023-10-06T22:02:08.928 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 941 kB/s | 126 kB 00:00 2023-10-06T22:02:09.037 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-6616.g14d8d032. 1.8 MB/s | 196 kB 00:00 2023-10-06T22:02:09.131 INFO:teuthology.orchestra.run.smithi057.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-10-06T22:02:09.163 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): python3-rados-18.0.0-6616.g14d8d032.e 2.9 MB/s | 371 kB 00:00 2023-10-06T22:02:09.213 INFO:teuthology.orchestra.run.smithi190.stdout:(5/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 4.4 MB/s | 1.6 MB 00:00 2023-10-06T22:02:09.271 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rbd-18.0.0-6616.g14d8d032.el8 3.3 MB/s | 364 kB 00:00 2023-10-06T22:02:09.413 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 8.7 MB/s | 6.9 MB 00:00 2023-10-06T22:02:09.447 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): python3-rgw-18.0.0-6616.g14d8d032.el8 607 kB/s | 107 kB 00:00 2023-10-06T22:02:09.481 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 1.4 MB/s | 94 kB 00:00 2023-10-06T22:02:09.581 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 1.7 MB/s | 179 kB 00:00 2023-10-06T22:02:09.665 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6616.g 277 kB/s | 23 kB 00:00 2023-10-06T22:02:09.757 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 1.4 MB/s | 132 kB 00:00 2023-10-06T22:02:10.039 INFO:teuthology.orchestra.run.smithi057.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 1.0 MB/s | 918 kB 00:00 2023-10-06T22:02:10.090 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 5.3 MB/s | 1.8 MB 00:00 2023-10-06T22:02:10.198 INFO:teuthology.orchestra.run.smithi057.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 1.0 MB/s | 167 kB 00:00 2023-10-06T22:02:10.221 INFO:teuthology.orchestra.run.smithi190.stdout:(6/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 4.6 MB/s | 4.6 MB 00:01 2023-10-06T22:02:10.299 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 4.2 MB/s | 3.6 MB 00:00 2023-10-06T22:02:10.424 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6616.g14 1.9 MB/s | 242 kB 00:00 2023-10-06T22:02:10.533 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 437 kB/s | 47 kB 00:00 2023-10-06T22:02:10.641 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6616.g1 134 kB/s | 15 kB 00:00 2023-10-06T22:02:10.767 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-volume-18.0.0-6616.g14d8d032.el8 2.0 MB/s | 261 kB 00:00 2023-10-06T22:02:10.867 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 1.9 MB/s | 192 kB 00:00 2023-10-06T22:02:10.959 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 197 kB/s | 18 kB 00:00 2023-10-06T22:02:11.351 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 782 kB/s | 306 kB 00:00 2023-10-06T22:02:11.435 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 707 kB/s | 59 kB 00:00 2023-10-06T22:02:11.481 INFO:teuthology.orchestra.run.smithi057.stdout:(4/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 1.8 MB/s | 2.3 MB 00:01 2023-10-06T22:02:11.526 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 820 kB/s | 75 kB 00:00 2023-10-06T22:02:11.610 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 4.9 MB/s | 7.4 MB 00:01 2023-10-06T22:02:11.661 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-10-06T22:02:11.744 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.3 MB/s | 185 kB 00:00 2023-10-06T22:02:11.856 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 296 kB/s | 33 kB 00:00 2023-10-06T22:02:11.903 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 149 kB/s | 44 kB 00:00 2023-10-06T22:02:11.929 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 226 kB/s | 16 kB 00:00 2023-10-06T22:02:12.020 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2023-10-06T22:02:12.088 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-10-06T22:02:12.113 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 825 kB/s | 173 kB 00:00 2023-10-06T22:02:12.155 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-10-06T22:02:12.189 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 232 kB/s | 18 kB 00:00 2023-10-06T22:02:12.198 INFO:teuthology.orchestra.run.smithi057.stdout:(5/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 2.3 MB/s | 1.6 MB 00:00 2023-10-06T22:02:12.264 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 651 kB/s | 49 kB 00:00 2023-10-06T22:02:12.298 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2023-10-06T22:02:12.340 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 780 kB/s | 59 kB 00:00 2023-10-06T22:02:12.365 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 902 kB/s | 60 kB 00:00 2023-10-06T22:02:12.432 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2023-10-06T22:02:12.482 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-10-06T22:02:12.507 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 448 kB/s | 34 kB 00:00 2023-10-06T22:02:12.555 INFO:teuthology.orchestra.run.smithi190.stdout:(7/131): ceph-common-18.0.0-6616.g14d8d032.el8. 4.3 MB/s | 23 MB 00:05 2023-10-06T22:02:12.599 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-10-06T22:02:12.616 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 591 kB/s | 64 kB 00:00 2023-10-06T22:02:12.630 INFO:teuthology.orchestra.run.smithi190.stdout:(8/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 1.0 MB/s | 5.4 MB 00:05 2023-10-06T22:02:12.675 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-10-06T22:02:12.700 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2023-10-06T22:02:12.713 INFO:teuthology.orchestra.run.smithi190.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 293 kB/s | 24 kB 00:00 2023-10-06T22:02:12.751 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-10-06T22:02:12.776 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-10-06T22:02:12.843 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-10-06T22:02:12.868 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-10-06T22:02:12.927 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 732 kB/s | 61 kB 00:00 2023-10-06T22:02:12.952 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-06T22:02:13.002 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-10-06T22:02:13.028 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-06T22:02:13.086 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-10-06T22:02:13.112 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 561 kB/s | 47 kB 00:00 2023-10-06T22:02:13.162 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 961 kB/s | 43 kB 00:00 2023-10-06T22:02:13.187 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2023-10-06T22:02:13.213 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-06T22:02:13.238 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2023-10-06T22:02:13.264 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-06T22:02:13.289 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 849 kB/s | 43 kB 00:00 2023-10-06T22:02:13.314 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-10-06T22:02:13.323 INFO:teuthology.orchestra.run.smithi057.stdout:(6/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-10-06T22:02:13.342 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-06T22:02:13.373 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-10-06T22:02:13.390 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-10-06T22:02:13.415 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-10-06T22:02:13.449 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-10-06T22:02:13.466 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 357 kB/s | 18 kB 00:00 2023-10-06T22:02:13.500 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-10-06T22:02:13.525 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-06T22:02:13.559 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.1 MB/s | 72 kB 00:00 2023-10-06T22:02:13.609 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 21 MB/s | 2.3 MB 00:00 2023-10-06T22:02:13.634 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-10-06T22:02:13.676 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 893 kB/s | 60 kB 00:00 2023-10-06T22:02:13.701 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.4 MB/s | 98 kB 00:00 2023-10-06T22:02:13.727 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-06T22:02:13.752 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-10-06T22:02:13.794 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-10-06T22:02:13.819 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-10-06T22:02:13.845 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-06T22:02:13.895 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-10-06T22:02:13.937 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-10-06T22:02:13.987 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-10-06T22:02:13.990 INFO:teuthology.orchestra.run.smithi057.stdout:(7/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 2.6 MB/s | 4.6 MB 00:01 2023-10-06T22:02:14.022 INFO:teuthology.orchestra.run.smithi190.stdout:(10/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x8 4.7 MB/s | 18 MB 00:03 2023-10-06T22:02:14.038 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-10-06T22:02:14.080 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 935 kB/s | 39 kB 00:00 2023-10-06T22:02:14.097 INFO:teuthology.orchestra.run.smithi190.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 3.4 MB/s | 260 kB 00:00 2023-10-06T22:02:14.172 INFO:teuthology.orchestra.run.smithi190.stdout:(12/131): libcephfs-devel-18.0.0-6616.g14d8d032 414 kB/s | 31 kB 00:00 2023-10-06T22:02:14.241 INFO:teuthology.orchestra.run.smithi057.stdout:(8/131): ceph-common-18.0.0-6616.g14d8d032.el8. 4.2 MB/s | 23 MB 00:05 2023-10-06T22:02:14.255 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 31 MB/s | 14 MB 00:00 2023-10-06T22:02:14.282 INFO:teuthology.orchestra.run.smithi057.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 590 kB/s | 24 kB 00:00 2023-10-06T22:02:14.306 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.0 MB/s | 688 kB 00:00 2023-10-06T22:02:14.339 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-10-06T22:02:14.356 INFO:teuthology.orchestra.run.smithi190.stdout:(13/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 7.7 MB/s | 14 MB 00:01 2023-10-06T22:02:14.364 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-06T22:02:14.390 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-10-06T22:02:14.398 INFO:teuthology.orchestra.run.smithi190.stdout:(14/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 3.6 MB/s | 840 kB 00:00 2023-10-06T22:02:14.423 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-10-06T22:02:14.432 INFO:teuthology.orchestra.run.smithi190.stdout:(15/131): libcephsqlite-18.0.0-6616.g14d8d032.e 2.4 MB/s | 183 kB 00:00 2023-10-06T22:02:14.474 INFO:teuthology.orchestra.run.smithi190.stdout:(16/131): librados-devel-18.0.0-6616.g14d8d032. 1.8 MB/s | 139 kB 00:00 2023-10-06T22:02:14.541 INFO:teuthology.orchestra.run.smithi190.stdout:(17/131): libradosstriper1-18.0.0-6616.g14d8d03 4.7 MB/s | 518 kB 00:00 2023-10-06T22:02:14.599 INFO:teuthology.orchestra.run.smithi190.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 769 kB/s | 45 kB 00:00 2023-10-06T22:02:14.666 INFO:teuthology.orchestra.run.smithi190.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 1.8 MB/s | 126 kB 00:00 2023-10-06T22:02:14.742 INFO:teuthology.orchestra.run.smithi190.stdout:(20/131): python3-cephfs-18.0.0-6616.g14d8d032. 2.6 MB/s | 196 kB 00:00 2023-10-06T22:02:14.825 INFO:teuthology.orchestra.run.smithi190.stdout:(21/131): python3-rados-18.0.0-6616.g14d8d032.e 4.4 MB/s | 371 kB 00:00 2023-10-06T22:02:14.907 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): ceph-test-18.0.0-6616.g14d8d032.el8. 6.3 MB/s | 49 MB 00:07 2023-10-06T22:02:14.908 INFO:teuthology.orchestra.run.smithi190.stdout:(22/131): python3-rbd-18.0.0-6616.g14d8d032.el8 4.3 MB/s | 364 kB 00:00 2023-10-06T22:02:14.932 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 52 kB/s | 27 kB 00:00 2023-10-06T22:02:14.976 INFO:teuthology.orchestra.run.smithi190.stdout:(23/131): python3-rgw-18.0.0-6616.g14d8d032.el8 1.6 MB/s | 107 kB 00:00 2023-10-06T22:02:14.991 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 3.8 MB/s | 320 kB 00:00 2023-10-06T22:02:15.016 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 565 kB/s | 44 kB 00:00 2023-10-06T22:02:15.042 INFO:teuthology.orchestra.run.smithi190.stdout:(24/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 1.4 MB/s | 94 kB 00:00 2023-10-06T22:02:15.091 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): python3-babel-2.5.1-7.el8.noarch.rpm 6.8 MB/s | 4.8 MB 00:00 2023-10-06T22:02:15.133 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 3.8 MB/s | 457 kB 00:00 2023-10-06T22:02:15.158 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 3.3 MB/s | 225 kB 00:00 2023-10-06T22:02:15.192 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-10-06T22:02:15.217 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.1 MB/s | 126 kB 00:00 2023-10-06T22:02:15.234 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 479 kB/s | 20 kB 00:00 2023-10-06T22:02:15.268 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-06T22:02:15.327 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2023-10-06T22:02:15.360 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-10-06T22:02:15.387 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-10-06T22:02:15.436 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-10-06T22:02:15.486 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.1 MB/s | 354 kB 00:00 2023-10-06T22:02:15.526 INFO:teuthology.orchestra.run.smithi190.stdout:(25/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 7.4 MB/s | 3.6 MB 00:00 2023-10-06T22:02:15.570 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 16 MB/s | 4.9 MB 00:00 2023-10-06T22:02:15.637 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-10-06T22:02:15.640 INFO:teuthology.orchestra.run.smithi190.stdout:(26/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 1.5 MB/s | 179 kB 00:00 2023-10-06T22:02:15.679 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-10-06T22:02:15.727 INFO:teuthology.orchestra.run.smithi190.stdout:(27/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 5.5 MB/s | 6.9 MB 00:01 2023-10-06T22:02:15.737 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-10-06T22:02:15.752 INFO:teuthology.orchestra.run.smithi190.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6616.g 206 kB/s | 23 kB 00:00 2023-10-06T22:02:15.786 INFO:teuthology.orchestra.run.smithi190.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 2.2 MB/s | 132 kB 00:00 2023-10-06T22:02:15.821 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-10-06T22:02:15.879 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-10-06T22:02:16.044 INFO:teuthology.orchestra.run.smithi190.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 6.0 MB/s | 1.8 MB 00:00 2023-10-06T22:02:16.145 INFO:teuthology.orchestra.run.smithi190.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6616.g14 2.4 MB/s | 242 kB 00:00 2023-10-06T22:02:16.270 INFO:teuthology.orchestra.run.smithi190.stdout:(32/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 379 kB/s | 47 kB 00:00 2023-10-06T22:02:16.398 INFO:teuthology.orchestra.run.smithi190.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6616.g1 114 kB/s | 15 kB 00:00 2023-10-06T22:02:16.504 INFO:teuthology.orchestra.run.smithi190.stdout:(34/131): ceph-volume-18.0.0-6616.g14d8d032.el8 2.4 MB/s | 261 kB 00:00 2023-10-06T22:02:16.588 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 4.4 MB/s | 3.7 MB 00:00 2023-10-06T22:02:16.595 INFO:teuthology.orchestra.run.smithi190.stdout:(35/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 2.0 MB/s | 192 kB 00:00 2023-10-06T22:02:16.671 INFO:teuthology.orchestra.run.smithi190.stdout:(36/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 242 kB/s | 18 kB 00:00 2023-10-06T22:02:17.247 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): librados2-18.0.0-6616.g14d8d032.el8. 2.4 MB/s | 3.7 MB 00:01 2023-10-06T22:02:17.249 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-10-06T22:02:17.250 INFO:teuthology.orchestra.run.smithi156.stdout:Total 17 MB/s | 208 MB 00:12 2023-10-06T22:02:17.263 INFO:teuthology.orchestra.run.smithi190.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 5.0 MB/s | 7.4 MB 00:01 2023-10-06T22:02:17.305 INFO:teuthology.orchestra.run.smithi190.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 483 kB/s | 306 kB 00:00 2023-10-06T22:02:17.422 INFO:teuthology.orchestra.run.smithi190.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 644 kB/s | 75 kB 00:00 2023-10-06T22:02:17.631 INFO:teuthology.orchestra.run.smithi190.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 160 kB/s | 59 kB 00:00 2023-10-06T22:02:17.857 INFO:teuthology.orchestra.run.smithi057.stdout:(10/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 3.6 MB/s | 14 MB 00:03 2023-10-06T22:02:17.966 INFO:teuthology.orchestra.run.smithi057.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 2.4 MB/s | 260 kB 00:00 2023-10-06T22:02:18.024 INFO:teuthology.orchestra.run.smithi057.stdout:(12/131): libcephfs-devel-18.0.0-6616.g14d8d032 531 kB/s | 31 kB 00:00 2023-10-06T22:02:18.102 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-10-06T22:02:18.216 INFO:teuthology.orchestra.run.smithi057.stdout:(13/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 4.3 MB/s | 840 kB 00:00 2023-10-06T22:02:18.222 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-10-06T22:02:18.222 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-10-06T22:02:18.291 INFO:teuthology.orchestra.run.smithi057.stdout:(14/131): libcephsqlite-18.0.0-6616.g14d8d032.e 2.4 MB/s | 183 kB 00:00 2023-10-06T22:02:18.358 INFO:teuthology.orchestra.run.smithi057.stdout:(15/131): librados-devel-18.0.0-6616.g14d8d032. 2.0 MB/s | 139 kB 00:00 2023-10-06T22:02:18.500 INFO:teuthology.orchestra.run.smithi057.stdout:(16/131): libradosstriper1-18.0.0-6616.g14d8d03 3.6 MB/s | 518 kB 00:00 2023-10-06T22:02:18.647 INFO:teuthology.orchestra.run.smithi190.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 43 kB/s | 44 kB 00:01 2023-10-06T22:02:18.715 INFO:teuthology.orchestra.run.smithi190.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 368 kB/s | 476 kB 00:01 2023-10-06T22:02:18.815 INFO:teuthology.orchestra.run.smithi190.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-10-06T22:02:18.907 INFO:teuthology.orchestra.run.smithi190.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 715 kB/s | 185 kB 00:00 2023-10-06T22:02:18.924 INFO:teuthology.orchestra.run.smithi190.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-10-06T22:02:19.074 INFO:teuthology.orchestra.run.smithi190.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2023-10-06T22:02:19.099 INFO:teuthology.orchestra.run.smithi190.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 898 kB/s | 173 kB 00:00 2023-10-06T22:02:19.191 INFO:teuthology.orchestra.run.smithi190.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 695 kB/s | 81 kB 00:00 2023-10-06T22:02:19.216 INFO:teuthology.orchestra.run.smithi190.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-10-06T22:02:19.308 INFO:teuthology.orchestra.run.smithi190.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2023-10-06T22:02:19.357 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-10-06T22:02:19.362 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-10-06T22:02:19.384 INFO:teuthology.orchestra.run.smithi057.stdout:(17/131): ceph-test-18.0.0-6616.g14d8d032.el8.x 9.6 MB/s | 49 MB 00:05 2023-10-06T22:02:19.434 INFO:teuthology.orchestra.run.smithi057.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 905 kB/s | 45 kB 00:00 2023-10-06T22:02:19.493 INFO:teuthology.orchestra.run.smithi057.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 2.1 MB/s | 126 kB 00:00 2023-10-06T22:02:19.551 INFO:teuthology.orchestra.run.smithi057.stdout:(20/131): python3-cephfs-18.0.0-6616.g14d8d032. 3.3 MB/s | 196 kB 00:00 2023-10-06T22:02:19.619 INFO:teuthology.orchestra.run.smithi057.stdout:(21/131): python3-rados-18.0.0-6616.g14d8d032.e 5.4 MB/s | 371 kB 00:00 2023-10-06T22:02:19.686 INFO:teuthology.orchestra.run.smithi057.stdout:(22/131): python3-rbd-18.0.0-6616.g14d8d032.el8 5.4 MB/s | 364 kB 00:00 2023-10-06T22:02:19.736 INFO:teuthology.orchestra.run.smithi057.stdout:(23/131): python3-rgw-18.0.0-6616.g14d8d032.el8 2.1 MB/s | 107 kB 00:00 2023-10-06T22:02:19.786 INFO:teuthology.orchestra.run.smithi057.stdout:(24/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 1.8 MB/s | 94 kB 00:00 2023-10-06T22:02:19.842 INFO:teuthology.orchestra.run.smithi190.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 91 kB/s | 49 kB 00:00 2023-10-06T22:02:19.975 INFO:teuthology.orchestra.run.smithi190.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 441 kB/s | 59 kB 00:00 2023-10-06T22:02:20.103 INFO:teuthology.orchestra.run.smithi057.stdout:(25/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x8 2.6 MB/s | 18 MB 00:06 2023-10-06T22:02:20.128 INFO:teuthology.orchestra.run.smithi190.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 1.3 MB/s | 1.2 MB 00:00 2023-10-06T22:02:20.159 INFO:teuthology.orchestra.run.smithi190.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 328 kB/s | 60 kB 00:00 2023-10-06T22:02:20.204 INFO:teuthology.orchestra.run.smithi057.stdout:(26/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 8.6 MB/s | 3.6 MB 00:00 2023-10-06T22:02:20.254 INFO:teuthology.orchestra.run.smithi057.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6616.g 465 kB/s | 23 kB 00:00 2023-10-06T22:02:20.276 INFO:teuthology.orchestra.run.smithi190.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-10-06T22:02:20.279 INFO:teuthology.orchestra.run.smithi057.stdout:(28/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 1.0 MB/s | 179 kB 00:00 2023-10-06T22:02:20.302 INFO:teuthology.orchestra.run.smithi190.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-10-06T22:02:20.304 INFO:teuthology.orchestra.run.smithi057.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 2.6 MB/s | 132 kB 00:00 2023-10-06T22:02:20.402 INFO:teuthology.orchestra.run.smithi190.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 269 kB/s | 34 kB 00:00 2023-10-06T22:02:20.427 INFO:teuthology.orchestra.run.smithi190.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-10-06T22:02:20.471 INFO:teuthology.orchestra.run.smithi057.stdout:(30/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 3.5 MB/s | 6.9 MB 00:01 2023-10-06T22:02:20.528 INFO:teuthology.orchestra.run.smithi190.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 513 kB/s | 64 kB 00:00 2023-10-06T22:02:20.553 INFO:teuthology.orchestra.run.smithi190.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 150 kB/s | 19 kB 00:00 2023-10-06T22:02:20.596 INFO:teuthology.orchestra.run.smithi057.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6616.g14 1.9 MB/s | 242 kB 00:00 2023-10-06T22:02:20.645 INFO:teuthology.orchestra.run.smithi190.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 205 kB/s | 24 kB 00:00 2023-10-06T22:02:20.670 INFO:teuthology.orchestra.run.smithi190.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-10-06T22:02:20.691 INFO:teuthology.orchestra.run.smithi057.stdout:(32/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 518 kB/s | 47 kB 00:00 2023-10-06T22:02:20.762 INFO:teuthology.orchestra.run.smithi190.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-10-06T22:02:20.764 INFO:teuthology.orchestra.run.smithi057.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6616.g1 194 kB/s | 15 kB 00:00 2023-10-06T22:02:20.812 INFO:teuthology.orchestra.run.smithi190.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-10-06T22:02:20.914 INFO:teuthology.orchestra.run.smithi057.stdout:(34/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 2.8 MB/s | 1.8 MB 00:00 2023-10-06T22:02:20.941 INFO:teuthology.orchestra.run.smithi057.stdout:(35/131): ceph-volume-18.0.0-6616.g14d8d032.el8 1.5 MB/s | 261 kB 00:00 2023-10-06T22:02:21.048 INFO:teuthology.orchestra.run.smithi057.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 10 MB/s | 7.4 MB 00:00 2023-10-06T22:02:21.073 INFO:teuthology.orchestra.run.smithi057.stdout:(37/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 136 kB/s | 18 kB 00:00 2023-10-06T22:02:21.099 INFO:teuthology.orchestra.run.smithi057.stdout:(38/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 1.0 MB/s | 192 kB 00:00 2023-10-06T22:02:21.491 INFO:teuthology.orchestra.run.smithi057.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 192 kB/s | 75 kB 00:00 2023-10-06T22:02:21.516 INFO:teuthology.orchestra.run.smithi057.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 133 kB/s | 59 kB 00:00 2023-10-06T22:02:21.541 INFO:teuthology.orchestra.run.smithi057.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 622 kB/s | 306 kB 00:00 2023-10-06T22:02:21.708 INFO:teuthology.orchestra.run.smithi057.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 227 kB/s | 44 kB 00:00 2023-10-06T22:02:21.825 INFO:teuthology.orchestra.run.smithi057.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 283 kB/s | 33 kB 00:00 2023-10-06T22:02:21.850 INFO:teuthology.orchestra.run.smithi057.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 600 kB/s | 185 kB 00:00 2023-10-06T22:02:21.875 INFO:teuthology.orchestra.run.smithi057.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-10-06T22:02:21.994 INFO:teuthology.orchestra.run.smithi057.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-10-06T22:02:22.009 INFO:teuthology.orchestra.run.smithi057.stdout:(47/131): python3-certifi-2018.10.15-7.el8.noar 88 kB/s | 16 kB 00:00 2023-10-06T22:02:22.013 INFO:teuthology.orchestra.run.smithi190.stdout:(65/131): ceph-test-18.0.0-6616.g14d8d032.el8.x 5.3 MB/s | 49 MB 00:09 2023-10-06T22:02:22.038 INFO:teuthology.orchestra.run.smithi190.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 50 kB/s | 61 kB 00:01 2023-10-06T22:02:22.063 INFO:teuthology.orchestra.run.smithi190.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 193 kB/s | 251 kB 00:01 2023-10-06T22:02:22.155 INFO:teuthology.orchestra.run.smithi190.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-10-06T22:02:22.180 INFO:teuthology.orchestra.run.smithi190.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-06T22:02:22.351 INFO:teuthology.orchestra.run.smithi057.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 227 kB/s | 81 kB 00:00 2023-10-06T22:02:22.410 INFO:teuthology.orchestra.run.smithi057.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 720 kB/s | 384 kB 00:00 2023-10-06T22:02:22.469 INFO:teuthology.orchestra.run.smithi057.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2023-10-06T22:02:22.519 INFO:teuthology.orchestra.run.smithi057.stdout:(51/131): python3-jaraco-6.2-6.el8.noarch.rpm 22 kB/s | 11 kB 00:00 2023-10-06T22:02:22.539 INFO:teuthology.orchestra.run.smithi190.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 38 MB/s | 14 MB 00:00 2023-10-06T22:02:22.573 INFO:teuthology.orchestra.run.smithi190.stdout:(71/131): python3-webtest-2.0.33-1.el8.noarch.r 152 kB/s | 85 kB 00:00 2023-10-06T22:02:22.598 INFO:teuthology.orchestra.run.smithi190.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-10-06T22:02:22.619 INFO:teuthology.orchestra.run.smithi057.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 324 kB/s | 49 kB 00:00 2023-10-06T22:02:22.640 INFO:teuthology.orchestra.run.smithi190.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 3.0 MB/s | 1.7 MB 00:00 2023-10-06T22:02:22.657 INFO:teuthology.orchestra.run.smithi190.stdout:(74/131): python3-pytz-2017.2-9.el8.noarch.rpm 647 kB/s | 54 kB 00:00 2023-10-06T22:02:22.690 INFO:teuthology.orchestra.run.smithi190.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.9 MB/s | 259 kB 00:00 2023-10-06T22:02:22.711 INFO:teuthology.orchestra.run.smithi057.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 660 kB/s | 60 kB 00:00 2023-10-06T22:02:22.740 INFO:teuthology.orchestra.run.smithi190.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 497 kB/s | 39 kB 00:00 2023-10-06T22:02:22.745 INFO:teuthology.orchestra.run.smithi057.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 260 kB/s | 59 kB 00:00 2023-10-06T22:02:22.766 INFO:teuthology.orchestra.run.smithi190.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2023-10-06T22:02:22.778 INFO:teuthology.orchestra.run.smithi057.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-10-06T22:02:22.799 INFO:teuthology.orchestra.run.smithi190.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2023-10-06T22:02:22.833 INFO:teuthology.orchestra.run.smithi190.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 5.7 MB/s | 538 kB 00:00 2023-10-06T22:02:22.858 INFO:teuthology.orchestra.run.smithi190.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-10-06T22:02:22.892 INFO:teuthology.orchestra.run.smithi190.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2023-10-06T22:02:22.926 INFO:teuthology.orchestra.run.smithi190.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-10-06T22:02:22.942 INFO:teuthology.orchestra.run.smithi190.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 528 kB/s | 27 kB 00:00 2023-10-06T22:02:22.962 INFO:teuthology.orchestra.run.smithi057.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 183 kB/s | 34 kB 00:00 2023-10-06T22:02:22.976 INFO:teuthology.orchestra.run.smithi190.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-10-06T22:02:22.987 INFO:teuthology.orchestra.run.smithi057.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 64 kB/s | 16 kB 00:00 2023-10-06T22:02:23.001 INFO:teuthology.orchestra.run.smithi190.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 750 kB/s | 44 kB 00:00 2023-10-06T22:02:23.052 INFO:teuthology.orchestra.run.smithi190.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-10-06T22:02:23.088 INFO:teuthology.orchestra.run.smithi057.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 752 kB/s | 283 kB 00:00 2023-10-06T22:02:23.113 INFO:teuthology.orchestra.run.smithi057.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 513 kB/s | 64 kB 00:00 2023-10-06T22:02:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-10-06T22:02:23.160 INFO:teuthology.orchestra.run.smithi190.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 20 MB/s | 3.7 MB 00:00 2023-10-06T22:02:23.186 INFO:teuthology.orchestra.run.smithi190.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.6 MB/s | 225 kB 00:00 2023-10-06T22:02:23.211 INFO:teuthology.orchestra.run.smithi190.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-06T22:02:23.263 INFO:teuthology.orchestra.run.smithi057.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 159 kB/s | 24 kB 00:00 2023-10-06T22:02:23.278 INFO:teuthology.orchestra.run.smithi190.stdout:(90/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-10-06T22:02:23.289 INFO:teuthology.orchestra.run.smithi057.stdout:(62/131): python3-simplegeneric-0.8.1-17.el8.no 93 kB/s | 19 kB 00:00 2023-10-06T22:02:23.303 INFO:teuthology.orchestra.run.smithi190.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 217 kB/s | 20 kB 00:00 2023-10-06T22:02:23.314 INFO:teuthology.orchestra.run.smithi057.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 167 kB/s | 29 kB 00:00 2023-10-06T22:02:23.328 INFO:teuthology.orchestra.run.smithi190.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 888 kB/s | 126 kB 00:00 2023-10-06T22:02:23.354 INFO:teuthology.orchestra.run.smithi190.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-10-06T22:02:23.395 INFO:teuthology.orchestra.run.smithi190.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-10-06T22:02:23.421 INFO:teuthology.orchestra.run.smithi190.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-10-06T22:02:23.448 INFO:teuthology.orchestra.run.smithi057.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 150 kB/s | 27 kB 00:00 2023-10-06T22:02:23.488 INFO:teuthology.orchestra.run.smithi190.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-10-06T22:02:23.523 INFO:teuthology.orchestra.run.smithi057.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-10-06T22:02:23.530 INFO:teuthology.orchestra.run.smithi190.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-06T22:02:23.565 INFO:teuthology.orchestra.run.smithi057.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.0 MB/s | 255 kB 00:00 2023-10-06T22:02:23.613 INFO:teuthology.orchestra.run.smithi190.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-10-06T22:02:23.615 INFO:teuthology.orchestra.run.smithi057.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 367 kB/s | 61 kB 00:00 2023-10-06T22:02:23.688 INFO:teuthology.orchestra.run.smithi190.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-10-06T22:02:23.730 INFO:teuthology.orchestra.run.smithi190.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-10-06T22:02:23.764 INFO:teuthology.orchestra.run.smithi190.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-10-06T22:02:23.830 INFO:teuthology.orchestra.run.smithi190.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-10-06T22:02:23.865 INFO:teuthology.orchestra.run.smithi190.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.9 MB/s | 200 kB 00:00 2023-10-06T22:02:23.882 INFO:teuthology.orchestra.run.smithi190.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 833 kB/s | 43 kB 00:00 2023-10-06T22:02:23.904 INFO:teuthology.orchestra.run.smithi057.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 224 kB/s | 85 kB 00:00 2023-10-06T22:02:23.915 INFO:teuthology.orchestra.run.smithi190.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-06T22:02:23.932 INFO:teuthology.orchestra.run.smithi190.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 958 kB/s | 48 kB 00:00 2023-10-06T22:02:23.933 INFO:teuthology.orchestra.run.smithi057.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.5 MB/s | 70 kB 00:00 2023-10-06T22:02:23.966 INFO:teuthology.orchestra.run.smithi190.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-06T22:02:23.982 INFO:teuthology.orchestra.run.smithi190.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-10-06T22:02:23.991 INFO:teuthology.orchestra.run.smithi057.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 806 kB/s | 47 kB 00:00 2023-10-06T22:02:24.008 INFO:teuthology.orchestra.run.smithi190.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.6 MB/s | 109 kB 00:00 2023-10-06T22:02:24.033 INFO:teuthology.orchestra.run.smithi057.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-10-06T22:02:24.033 INFO:teuthology.orchestra.run.smithi190.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-06T22:02:24.067 INFO:teuthology.orchestra.run.smithi190.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-10-06T22:02:24.084 INFO:teuthology.orchestra.run.smithi190.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 373 kB/s | 19 kB 00:00 2023-10-06T22:02:24.151 INFO:teuthology.orchestra.run.smithi190.stdout:(113/131): postgresql-server-10.21-2.module+el8 8.2 MB/s | 5.1 MB 00:00 2023-10-06T22:02:24.176 INFO:teuthology.orchestra.run.smithi190.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.4 MB/s | 155 kB 00:00 2023-10-06T22:02:24.226 INFO:teuthology.orchestra.run.smithi057.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 222 kB/s | 43 kB 00:00 2023-10-06T22:02:24.234 INFO:teuthology.orchestra.run.smithi190.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 214 kB/s | 18 kB 00:00 2023-10-06T22:02:24.268 INFO:teuthology.orchestra.run.smithi190.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.0 MB/s | 559 kB 00:00 2023-10-06T22:02:24.293 INFO:teuthology.orchestra.run.smithi190.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-10-06T22:02:24.310 INFO:teuthology.orchestra.run.smithi190.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-10-06T22:02:24.317 INFO:teuthology.orchestra.run.smithi057.stdout:(73/131): python3-zc-lockfile-2.0-2.el8.noarch. 30 kB/s | 23 kB 00:00 2023-10-06T22:02:24.336 INFO:teuthology.orchestra.run.smithi190.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-10-06T22:02:24.358 INFO:teuthology.orchestra.run.smithi057.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2023-10-06T22:02:24.361 INFO:teuthology.orchestra.run.smithi190.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-06T22:02:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-10-06T22:02:24.392 INFO:teuthology.orchestra.run.smithi057.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-10-06T22:02:24.411 INFO:teuthology.orchestra.run.smithi190.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-06T22:02:24.426 INFO:teuthology.orchestra.run.smithi057.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 442 kB/s | 48 kB 00:00 2023-10-06T22:02:24.436 INFO:teuthology.orchestra.run.smithi190.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-06T22:02:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 856 kB/s | 43 kB 00:00 2023-10-06T22:02:24.462 INFO:teuthology.orchestra.run.smithi190.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-10-06T22:02:24.476 INFO:teuthology.orchestra.run.smithi057.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 800 kB/s | 87 kB 00:00 2023-10-06T22:02:24.497 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-10-06T22:02:24.501 INFO:teuthology.orchestra.run.smithi057.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-10-06T22:02:24.512 INFO:teuthology.orchestra.run.smithi190.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-10-06T22:02:24.535 INFO:teuthology.orchestra.run.smithi057.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-10-06T22:02:24.537 INFO:teuthology.orchestra.run.smithi190.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-10-06T22:02:24.569 INFO:teuthology.orchestra.run.smithi057.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 280 kB/s | 19 kB 00:00 2023-10-06T22:02:24.587 INFO:teuthology.orchestra.run.smithi190.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.8 MB/s | 138 kB 00:00 2023-10-06T22:02:24.602 INFO:teuthology.orchestra.run.smithi057.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.3 MB/s | 155 kB 00:00 2023-10-06T22:02:24.645 INFO:teuthology.orchestra.run.smithi057.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-10-06T22:02:24.654 INFO:teuthology.orchestra.run.smithi190.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-10-06T22:02:24.670 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T22:02:24.670 INFO:teuthology.orchestra.run.smithi057.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 267 kB/s | 18 kB 00:00 2023-10-06T22:02:24.704 INFO:teuthology.orchestra.run.smithi057.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.1 MB/s | 559 kB 00:00 2023-10-06T22:02:24.705 INFO:teuthology.orchestra.run.smithi190.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-06T22:02:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-10-06T22:02:24.779 INFO:teuthology.orchestra.run.smithi057.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-10-06T22:02:24.796 INFO:teuthology.orchestra.run.smithi057.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-10-06T22:02:24.829 INFO:teuthology.orchestra.run.smithi057.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-06T22:02:24.845 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T22:02:24.871 INFO:teuthology.orchestra.run.smithi057.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 799 kB/s | 60 kB 00:00 2023-10-06T22:02:24.897 INFO:teuthology.orchestra.run.smithi057.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.4 MB/s | 98 kB 00:00 2023-10-06T22:02:24.989 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T22:02:25.013 INFO:teuthology.orchestra.run.smithi190.stdout:(130/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 8.7 MB/s | 3.7 MB 00:00 2023-10-06T22:02:25.043 INFO:teuthology.orchestra.run.smithi057.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 337 kB/s | 58 kB 00:00 2023-10-06T22:02:25.081 INFO:teuthology.orchestra.run.smithi057.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 210 kB/s | 39 kB 00:00 2023-10-06T22:02:25.098 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T22:02:25.131 INFO:teuthology.orchestra.run.smithi057.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 7.2 MB/s | 644 kB 00:00 2023-10-06T22:02:25.157 INFO:teuthology.orchestra.run.smithi057.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 171 kB 00:00 2023-10-06T22:02:25.182 INFO:teuthology.orchestra.run.smithi057.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-06T22:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-10-06T22:02:25.282 INFO:teuthology.orchestra.run.smithi057.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 4.0 MB/s | 2.3 MB 00:00 2023-10-06T22:02:25.307 INFO:teuthology.orchestra.run.smithi057.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 651 kB/s | 54 kB 00:00 2023-10-06T22:02:25.330 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T22:02:25.341 INFO:teuthology.orchestra.run.smithi057.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-10-06T22:02:25.375 INFO:teuthology.orchestra.run.smithi057.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-06T22:02:25.400 INFO:teuthology.orchestra.run.smithi057.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 665 kB/s | 39 kB 00:00 2023-10-06T22:02:25.405 INFO:teuthology.orchestra.run.smithi190.stdout:(131/131): librados2-18.0.0-6616.g14d8d032.el8. 4.3 MB/s | 3.7 MB 00:00 2023-10-06T22:02:25.408 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2023-10-06T22:02:25.408 INFO:teuthology.orchestra.run.smithi190.stdout:Total 11 MB/s | 208 MB 00:18 2023-10-06T22:02:25.452 INFO:teuthology.orchestra.run.smithi057.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-10-06T22:02:25.501 INFO:teuthology.orchestra.run.smithi057.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.3 MB/s | 538 kB 00:00 2023-10-06T22:02:25.526 INFO:teuthology.orchestra.run.smithi057.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-10-06T22:02:25.559 INFO:teuthology.orchestra.run.smithi057.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-06T22:02:25.610 INFO:teuthology.orchestra.run.smithi057.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-10-06T22:02:25.654 INFO:teuthology.orchestra.run.smithi057.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 641 kB/s | 27 kB 00:00 2023-10-06T22:02:25.702 INFO:teuthology.orchestra.run.smithi057.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-10-06T22:02:25.744 INFO:teuthology.orchestra.run.smithi057.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-10-06T22:02:25.758 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T22:02:25.886 INFO:teuthology.orchestra.run.smithi057.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-10-06T22:02:25.945 INFO:teuthology.orchestra.run.smithi057.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-10-06T22:02:25.988 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T22:02:26.028 INFO:teuthology.orchestra.run.smithi057.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2023-10-06T22:02:26.054 INFO:teuthology.orchestra.run.smithi057.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-10-06T22:02:26.079 INFO:teuthology.orchestra.run.smithi057.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-06T22:02:26.104 INFO:teuthology.orchestra.run.smithi057.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-06T22:02:26.130 INFO:teuthology.orchestra.run.smithi057.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-10-06T22:02:26.155 INFO:teuthology.orchestra.run.smithi057.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-06T22:02:26.197 INFO:teuthology.orchestra.run.smithi057.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-06T22:02:26.239 INFO:teuthology.orchestra.run.smithi057.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-10-06T22:02:26.247 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2023-10-06T22:02:26.366 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2023-10-06T22:02:26.367 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2023-10-06T22:02:26.394 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T22:02:26.406 INFO:teuthology.orchestra.run.smithi057.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 11 MB/s | 14 MB 00:01 2023-10-06T22:02:26.448 INFO:teuthology.orchestra.run.smithi057.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-06T22:02:26.498 INFO:teuthology.orchestra.run.smithi057.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-10-06T22:02:26.563 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:02:26.581 INFO:teuthology.orchestra.run.smithi057.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 11 MB/s | 4.9 MB 00:00 2023-10-06T22:02:26.674 INFO:teuthology.orchestra.run.smithi057.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2023-10-06T22:02:26.681 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:02:26.707 INFO:teuthology.orchestra.run.smithi057.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-10-06T22:02:26.799 INFO:teuthology.orchestra.run.smithi057.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-10-06T22:02:26.830 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:02:26.883 INFO:teuthology.orchestra.run.smithi057.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-10-06T22:02:26.928 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:02:26.941 INFO:teuthology.orchestra.run.smithi057.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-10-06T22:02:27.077 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T22:02:27.242 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-06T22:02:27.500 INFO:teuthology.orchestra.run.smithi057.stdout:(130/131): librados2-18.0.0-6616.g14d8d032.el8. 4.7 MB/s | 3.7 MB 00:00 2023-10-06T22:02:27.501 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2023-10-06T22:02:27.502 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T22:02:27.502 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2023-10-06T22:02:27.680 INFO:teuthology.orchestra.run.smithi057.stdout:(131/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 4.2 MB/s | 3.7 MB 00:00 2023-10-06T22:02:27.682 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-06T22:02:27.682 INFO:teuthology.orchestra.run.smithi057.stdout:Total 11 MB/s | 208 MB 00:18 2023-10-06T22:02:27.715 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T22:02:28.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T22:02:28.378 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T22:02:28.522 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T22:02:28.613 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T22:02:28.746 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T22:02:28.894 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T22:02:29.100 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-06T22:02:29.277 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-06T22:02:29.365 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-10-06T22:02:29.470 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-06T22:02:29.511 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-10-06T22:02:29.511 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-10-06T22:02:29.604 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T22:02:29.730 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T22:02:29.879 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T22:02:30.473 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T22:02:30.646 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T22:02:31.140 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-10-06T22:02:31.145 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-10-06T22:02:31.240 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T22:02:31.412 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T22:02:32.092 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T22:02:32.319 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-06T22:02:32.510 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-06T22:02:32.739 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T22:02:32.902 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T22:02:33.013 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2023-10-06T22:02:33.036 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T22:02:33.250 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T22:02:33.462 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T22:02:33.658 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T22:02:33.720 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T22:02:33.855 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-06T22:02:34.072 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T22:02:34.165 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T22:02:34.422 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T22:02:34.836 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T22:02:35.088 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:02:35.495 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T22:02:35.495 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:02:35.671 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:02:35.803 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:02:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T22:02:36.086 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T22:02:36.304 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T22:02:36.367 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-10-06T22:02:36.495 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-06T22:02:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T22:02:36.568 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T22:02:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T22:02:36.764 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T22:02:36.810 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-06T22:02:36.935 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T22:02:37.063 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-06T22:02:37.071 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-06T22:02:37.161 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T22:02:37.285 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T22:02:37.294 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-06T22:02:37.428 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T22:02:37.482 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T22:02:37.571 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T22:02:37.661 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T22:02:37.680 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T22:02:37.813 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T22:02:37.859 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T22:02:37.893 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T22:02:37.948 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-06T22:02:38.009 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T22:02:38.052 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T22:02:38.143 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-06T22:02:38.206 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T22:02:38.235 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T22:02:38.279 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-06T22:02:38.362 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T22:02:38.444 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T22:02:38.510 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T22:02:38.742 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 5/134 2023-10-06T22:02:38.751 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-06T22:02:38.918 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:02:38.957 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-06T22:02:39.004 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T22:02:39.077 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:02:39.103 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T22:02:39.180 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T22:02:39.221 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T22:02:39.243 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:02:39.346 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:02:39.390 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-06T22:02:39.557 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-06T22:02:39.726 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-06T22:02:39.805 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T22:02:39.825 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T22:02:39.896 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-06T22:02:40.000 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-06T22:02:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T22:02:40.080 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-06T22:02:40.230 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-06T22:02:40.431 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T22:02:40.440 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-06T22:02:40.634 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T22:02:40.638 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-06T22:02:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T22:02:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-06T22:02:41.010 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-06T22:02:41.060 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T22:02:41.214 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-06T22:02:41.286 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T22:02:41.313 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-06T22:02:41.468 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T22:02:41.483 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-06T22:02:41.523 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-06T22:02:41.599 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T22:02:41.685 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-06T22:02:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-06T22:02:41.868 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T22:02:41.919 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-06T22:02:41.965 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-06T22:02:42.057 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T22:02:42.148 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T22:02:42.277 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-06T22:02:42.291 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-06T22:02:42.318 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T22:02:42.448 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T22:02:42.478 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-06T22:02:42.485 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-06T22:02:42.670 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-06T22:02:42.677 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-06T22:02:42.801 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T22:02:42.890 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-06T22:02:42.968 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-06T22:02:43.064 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-06T22:02:43.118 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T22:02:43.239 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-06T22:02:43.376 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T22:02:43.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-06T22:02:43.685 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-06T22:02:43.790 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-06T22:02:43.854 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-06T22:02:43.965 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-06T22:02:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-06T22:02:44.026 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T22:02:44.284 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-06T22:02:44.295 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-06T22:02:44.460 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T22:02:44.495 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T22:02:44.630 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T22:02:44.751 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-06T22:02:44.870 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T22:02:45.034 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T22:02:45.085 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T22:02:45.206 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T22:02:45.337 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T22:02:45.527 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T22:02:45.721 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T22:02:45.895 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T22:02:45.933 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-06T22:02:46.088 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T22:02:46.183 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-06T22:02:46.233 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T22:02:46.262 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T22:02:46.391 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-06T22:02:46.441 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T22:02:46.579 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-06T22:02:46.626 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T22:02:46.673 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T22:02:46.783 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T22:02:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T22:02:46.841 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T22:02:46.926 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T22:02:47.022 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T22:02:47.075 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T22:02:47.208 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T22:02:47.211 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T22:02:47.268 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T22:02:47.365 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T22:02:47.417 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T22:02:47.509 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T22:02:47.552 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-06T22:02:47.628 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T22:02:47.714 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-06T22:02:47.939 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-06T22:02:48.073 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-06T22:02:48.352 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-06T22:02:48.561 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-06T22:02:48.741 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T22:02:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-06T22:02:49.006 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T22:02:49.214 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-06T22:02:49.276 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T22:02:49.457 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-06T22:02:49.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T22:02:49.665 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-06T22:02:49.732 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-06T22:02:49.854 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T22:02:49.907 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-06T22:02:50.040 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T22:02:50.235 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-06T22:02:50.452 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-06T22:02:50.629 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-06T22:02:50.832 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T22:02:50.864 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-06T22:02:50.964 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-06T22:02:51.001 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T22:02:51.031 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-06T22:02:51.079 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T22:02:51.230 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-06T22:02:51.243 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T22:02:51.308 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-06T22:02:51.518 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-06T22:02:51.589 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-06T22:02:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-06T22:02:51.836 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-06T22:02:52.024 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-06T22:02:52.074 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T22:02:52.266 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T22:02:52.272 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-06T22:02:52.467 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T22:02:52.467 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-06T22:02:52.662 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-06T22:02:52.683 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T22:02:52.852 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T22:02:52.868 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-06T22:02:52.921 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T22:02:53.039 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-06T22:02:53.199 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-06T22:02:53.208 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T22:02:53.287 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-06T22:02:53.439 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-06T22:02:53.550 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-06T22:02:53.594 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T22:02:53.674 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T22:02:53.757 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-06T22:02:53.907 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-06T22:02:53.949 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T22:02:54.038 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-06T22:02:54.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-06T22:02:54.243 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T22:02:54.298 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-06T22:02:54.405 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T22:02:54.463 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-06T22:02:54.610 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-06T22:02:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T22:02:54.805 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-06T22:02:54.919 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T22:02:54.921 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:02:55.003 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-06T22:02:55.056 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T22:02:55.134 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:02:55.166 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-06T22:02:55.212 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T22:02:55.394 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-06T22:02:55.428 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T22:02:55.606 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T22:02:55.619 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-06T22:02:55.738 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:02:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T22:02:55.983 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T22:02:56.021 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T22:02:56.066 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-06T22:02:56.067 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T22:02:56.184 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T22:02:56.191 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T22:02:56.383 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-06T22:02:56.408 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T22:02:56.625 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-06T22:02:56.648 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T22:02:56.858 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T22:02:56.894 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-06T22:02:57.075 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T22:02:57.128 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-06T22:02:57.292 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T22:02:57.459 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T22:02:57.463 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-06T22:02:57.701 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-06T22:02:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T22:02:57.928 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-06T22:02:58.129 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-06T22:02:58.316 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-06T22:02:58.489 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-06T22:02:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T22:02:58.875 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-06T22:02:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T22:02:59.188 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-06T22:02:59.444 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-06T22:02:59.493 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T22:02:59.694 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-06T22:02:59.796 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T22:02:59.876 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T22:03:00.052 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T22:03:00.067 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T22:03:00.254 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T22:03:00.347 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T22:03:00.616 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T22:03:00.771 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T22:03:00.930 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T22:03:01.043 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T22:03:01.135 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T22:03:01.209 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T22:03:01.301 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T22:03:01.343 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T22:03:01.451 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T22:03:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T22:03:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T22:03:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T22:03:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T22:03:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T22:03:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T22:03:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T22:03:03.134 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T22:03:03.319 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T22:03:03.586 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T22:03:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T22:03:05.204 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:03:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T22:03:05.459 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:03:05.548 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T22:03:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T22:03:06.051 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:03:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T22:03:06.376 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T22:03:06.460 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T22:03:06.472 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T22:03:06.642 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T22:03:07.388 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T22:03:07.564 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T22:03:07.658 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T22:03:07.856 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T22:03:13.022 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:03:13.247 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:03:13.997 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T22:03:14.281 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T22:03:14.410 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T22:03:14.555 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T22:03:15.024 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-10-06T22:03:15.025 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:03:15.348 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T22:03:16.949 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T22:03:17.112 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T22:03:17.332 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T22:03:17.497 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T22:03:17.950 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T22:03:18.081 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T22:03:18.098 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T22:03:19.755 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T22:03:19.956 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T22:03:20.094 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T22:03:20.304 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T22:03:20.480 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T22:03:20.702 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T22:03:20.970 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T22:03:21.931 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T22:03:22.105 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T22:03:22.195 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T22:03:22.520 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T22:03:22.716 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T22:03:22.752 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T22:03:22.888 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T22:03:22.899 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T22:03:24.232 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T22:03:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T22:03:24.448 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T22:03:24.579 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T22:03:24.622 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T22:03:24.754 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T22:03:25.115 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T22:03:25.279 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T22:03:25.481 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T22:03:25.481 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /run 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-10-06T22:03:25.482 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:03:25.845 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T22:03:27.456 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T22:03:27.614 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T22:03:27.834 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T22:03:27.997 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T22:03:28.451 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T22:03:28.589 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T22:03:28.605 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T22:03:29.919 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T22:03:30.481 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T22:03:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T22:03:30.780 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T22:03:30.802 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T22:03:30.996 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T22:03:31.065 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-06T22:03:31.204 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T22:03:31.231 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 117/134 2023-10-06T22:03:31.425 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T22:03:31.499 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-06T22:03:31.653 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T22:03:31.689 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T22:03:31.849 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T22:03:32.057 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-06T22:03:32.223 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T22:03:32.370 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T22:03:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T22:03:32.821 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T22:03:32.912 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T22:03:33.282 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T22:03:33.528 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T22:03:33.563 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T22:03:33.704 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T22:03:33.715 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T22:03:34.573 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:03:34.811 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:03:34.828 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:03:34.969 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T22:03:35.081 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T22:03:35.244 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T22:03:35.249 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T22:03:35.285 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T22:03:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T22:03:35.472 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T22:03:35.541 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T22:03:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T22:03:35.864 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T22:03:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T22:03:36.003 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T22:03:36.003 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /sys 2023-10-06T22:03:36.003 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /proc 2023-10-06T22:03:36.003 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /dev 2023-10-06T22:03:36.003 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /run 2023-10-06T22:03:36.003 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /mnt 2023-10-06T22:03:36.004 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /var/tmp 2023-10-06T22:03:36.004 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /home 2023-10-06T22:03:36.004 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /tmp 2023-10-06T22:03:36.004 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /dev 2023-10-06T22:03:36.004 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:03:36.044 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T22:03:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T22:03:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T22:03:36.319 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T22:03:36.331 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T22:03:36.377 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T22:03:36.469 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T22:03:36.476 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T22:03:36.604 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-06T22:03:36.604 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:03:36.797 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:03:36.797 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T22:03:37.007 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T22:03:37.209 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:03:37.747 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:03:37.747 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T22:03:37.748 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T22:03:37.749 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T22:03:37.750 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T22:03:37.751 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T22:03:37.752 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-06T22:03:37.753 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-06T22:03:37.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-06T22:03:37.755 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-06T22:03:37.756 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-06T22:03:37.757 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:03:37.758 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T22:03:38.260 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T22:03:38.444 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T22:03:38.728 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T22:03:38.918 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T22:03:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T22:03:39.710 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T22:03:39.752 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T22:03:40.177 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:03:40.177 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:47 ago on Fri 06 Oct 2023 10:01:53 PM UTC. 2023-10-06T22:03:40.787 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T22:03:41.095 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T22:03:41.269 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T22:03:41.461 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T22:03:42.371 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-06T22:03:42.682 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-06T22:03:42.702 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T22:03:42.880 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T22:03:42.881 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 119/134 2023-10-06T22:03:42.992 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T22:03:43.065 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T22:03:43.201 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T22:03:43.265 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T22:03:43.387 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T22:03:43.400 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T22:03:43.685 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T22:03:43.897 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-10-06T22:03:43.909 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T22:03:44.109 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T22:03:44.110 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.111 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:44.112 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T22:03:44.113 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T22:03:44.114 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T22:03:44.115 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T22:03:44.115 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T22:03:44.115 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T22:03:44.115 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T22:03:44.115 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T22:03:44.116 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T22:03:44.117 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T22:03:44.118 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T22:03:44.119 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T22:03:44.120 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T22:03:44.121 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.122 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.122 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:44.122 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T22:03:44.122 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T22:03:44.122 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T22:03:44.122 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T22:03:44.122 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T22:03:44.123 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T22:03:44.123 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-10-06T22:03:44.123 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-10-06T22:03:44.318 DEBUG:teuthology.parallel:result is None 2023-10-06T22:03:45.096 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T22:03:45.333 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T22:03:45.472 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T22:03:45.653 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:03:45.849 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T22:03:45.919 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:03:45.936 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:03:46.067 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T22:03:46.137 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T22:03:46.144 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T22:03:46.350 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T22:03:46.379 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T22:03:46.410 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T22:03:46.755 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T22:03:47.087 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T22:03:47.277 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T22:03:47.286 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T22:03:47.472 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-06T22:03:47.676 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T22:03:47.759 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T22:03:47.913 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T22:03:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T22:03:48.007 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T22:03:48.007 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:03:48.102 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T22:03:48.166 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T22:03:48.213 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:03:48.214 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T22:03:48.333 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T22:03:48.417 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T22:03:48.457 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T22:03:48.583 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T22:03:48.659 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:03:49.032 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T22:03:49.216 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T22:03:49.220 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T22:03:49.221 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T22:03:49.222 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T22:03:49.223 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T22:03:49.223 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T22:03:49.223 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T22:03:49.223 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T22:03:49.223 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T22:03:49.223 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T22:03:49.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T22:03:49.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T22:03:49.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T22:03:49.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T22:03:49.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T22:03:49.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T22:03:49.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T22:03:49.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T22:03:49.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T22:03:49.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-06T22:03:49.236 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-06T22:03:49.237 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-06T22:03:49.238 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-06T22:03:49.239 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-06T22:03:49.240 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-06T22:03:49.241 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T22:03:49.241 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T22:03:49.241 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T22:03:49.241 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T22:03:49.241 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:03:49.241 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T22:03:51.918 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:03:51.918 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:56 ago on Fri 06 Oct 2023 10:01:55 PM UTC. 2023-10-06T22:03:54.661 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T22:03:55.984 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T22:03:56.330 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-06T22:03:56.531 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 117/134 2023-10-06T22:03:56.782 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-06T22:03:56.973 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T22:03:57.249 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-06T22:03:57.479 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-06T22:03:57.670 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T22:03:57.833 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T22:03:59.581 INFO:teuthology.orchestra.run.smithi190.stdout:Installed products updated. 2023-10-06T22:03:59.797 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:03:59.797 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2023-10-06T22:03:59.797 INFO:teuthology.orchestra.run.smithi190.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T22:03:59.797 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.798 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.799 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-3.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T22:03:59.800 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.801 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T22:03:59.802 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T22:03:59.803 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T22:03:59.804 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T22:03:59.805 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T22:03:59.806 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T22:03:59.807 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T22:03:59.808 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T22:03:59.809 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T22:03:59.809 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:03:59.809 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-10-06T22:04:00.011 DEBUG:teuthology.parallel:result is None 2023-10-06T22:04:01.370 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:04:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:04:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T22:04:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T22:04:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T22:04:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T22:04:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T22:04:03.072 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T22:04:03.182 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-06T22:04:03.389 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T22:04:03.418 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-06T22:04:03.647 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T22:04:03.670 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-06T22:04:03.834 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-06T22:04:03.835 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T22:04:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T22:04:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:04:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:04:04.797 INFO:teuthology.orchestra.run.smithi057.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T22:04:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T22:04:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T22:04:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T22:04:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T22:04:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T22:04:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T22:04:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T22:04:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T22:04:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T22:04:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T22:04:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T22:04:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-06T22:04:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-06T22:04:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-06T22:04:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-06T22:04:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T22:04:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T22:04:08.150 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T22:04:08.150 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:02:13 ago on Fri 06 Oct 2023 10:01:55 PM UTC. 2023-10-06T22:04:11.664 INFO:teuthology.orchestra.run.smithi057.stdout:Installed products updated. 2023-10-06T22:04:11.907 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T22:04:11.908 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.909 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T22:04:11.910 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: jq-1.6-3.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T22:04:11.911 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T22:04:11.912 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:04:11.913 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T22:04:11.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T22:04:11.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T22:04:11.916 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T22:04:11.917 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T22:04:11.917 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T22:04:11.917 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T22:04:11.917 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.917 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.917 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T22:04:11.917 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T22:04:11.918 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.918 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T22:04:11.918 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T22:04:11.918 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T22:04:11.918 INFO:teuthology.orchestra.run.smithi057.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T22:04:11.918 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.919 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T22:04:11.920 INFO:teuthology.orchestra.run.smithi057.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T22:04:11.920 INFO:teuthology.orchestra.run.smithi057.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T22:04:11.920 INFO:teuthology.orchestra.run.smithi057.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T22:04:11.920 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T22:04:11.921 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T22:04:11.921 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T22:04:11.921 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:04:11.921 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-06T22:04:12.225 DEBUG:teuthology.parallel:result is None 2023-10-06T22:04:12.225 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:04:12.385 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T22:04:12.441 INFO:teuthology.orchestra.run.smithi057.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T22:04:12.441 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T22:04:12.441 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T22:04:12.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:04:12.657 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T22:04:12.686 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T22:04:12.687 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T22:04:12.687 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T22:04:12.687 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:04:12.901 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T22:04:12.930 INFO:teuthology.orchestra.run.smithi190.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T22:04:12.930 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T22:04:12.930 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T22:04:12.930 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T22:04:12.930 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:04:12.931 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T22:04:13.026 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:04:13.027 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T22:04:13.075 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:04:13.075 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T22:04:13.123 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T22:04:13.123 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:04:13.124 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T22:04:13.196 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T22:04:13.310 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:04:13.310 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T22:04:13.342 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T22:04:13.412 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:04:13.412 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T22:04:13.444 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T22:04:13.515 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T22:04:13.515 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:04:13.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T22:04:13.584 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T22:04:13.692 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:04:13.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T22:04:13.724 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T22:04:13.795 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:04:13.795 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T22:04:13.826 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T22:04:13.899 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T22:04:13.900 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:04:13.900 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T22:04:13.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T22:04:14.079 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:04:14.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T22:04:14.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T22:04:14.182 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:04:14.183 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T22:04:14.214 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T22:04:14.284 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-06T22:04:14.370 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_client_delegate_inos_pct': 50, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-06T22:04:14.370 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:04:14.371 INFO:tasks.cephadm:Cluster fsid is 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:04:14.371 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-06T22:04:14.371 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.57', 'mon.b': '172.21.15.156', 'mon.c': '172.21.15.190'} 2023-10-06T22:04:14.371 INFO:tasks.cephadm:First mon is mon.a on smithi057 2023-10-06T22:04:14.371 INFO:tasks.cephadm:First mgr is x 2023-10-06T22:04:14.371 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-06T22:04:14.371 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname $(hostname -s) 2023-10-06T22:04:14.454 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2023-10-06T22:04:14.486 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2023-10-06T22:04:14.519 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-06T22:04:14.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:04:14.659 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73047/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6616-g14d8d032', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6616.g14d8d032'}, 'url': 'https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-06 10:46:00.441906', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-cephfs-mon', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-06T22:04:14.798 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref rishabh-cephfs-mon, sha1 14d8d032507701c76ca224e769ee65a96b660a8c from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:04:14.798 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm 2023-10-06T22:04:14.799 DEBUG:teuthology.orchestra.run.smithi057:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:15.253 INFO:teuthology.orchestra.run.smithi057.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 22:04 /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:15.253 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:15.680 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 22:04 /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:15.680 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:16.267 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 22:04 /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:16.268 DEBUG:teuthology.orchestra.run.smithi057:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:16.319 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:16.341 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T22:04:16.376 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c on all hosts... 2023-10-06T22:04:16.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T22:04:16.381 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T22:04:16.383 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T22:04:17.646 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T22:04:18.171 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T22:04:20.598 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T22:04:51.166 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-10-06T22:04:51.167 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T22:04:51.167 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T22:04:51.167 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2023-10-06T22:04:51.167 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T22:04:51.167 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-10-06T22:04:51.167 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-10-06T22:04:53.872 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:04:53.872 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T22:04:53.872 INFO:teuthology.orchestra.run.smithi057.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T22:04:53.872 INFO:teuthology.orchestra.run.smithi057.stdout: "repo_digests": [ 2023-10-06T22:04:53.872 INFO:teuthology.orchestra.run.smithi057.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T22:04:53.872 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:04:53.872 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:04:54.386 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-10-06T22:04:54.386 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T22:04:54.386 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T22:04:54.386 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2023-10-06T22:04:54.387 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T22:04:54.387 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2023-10-06T22:04:54.387 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-10-06T22:04:54.406 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph 2023-10-06T22:04:54.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2023-10-06T22:04:54.519 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2023-10-06T22:04:54.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /etc/ceph 2023-10-06T22:04:54.619 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2023-10-06T22:04:54.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2023-10-06T22:04:54.679 INFO:tasks.cephadm:Writing seed config... 2023-10-06T22:04:54.680 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-10-06T22:04:54.680 INFO:tasks.cephadm: override: [client] debug client = 20 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-10-06T22:04:54.681 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mds] mds_client_delegate_inos_pct = 50 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-06T22:04:54.682 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-06T22:04:54.683 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-10-06T22:04:54.684 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-06T22:04:54.685 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-06T22:04:54.685 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:04:54.685 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-06T22:04:54.728 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 495b2048-6494-11ee-8db5-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_client_delegate_inos_pct = 50 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-10-06T22:04:54.728 DEBUG:teuthology.orchestra.run.smithi057:mon.a> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-06T22:04:54.773 DEBUG:teuthology.orchestra.run.smithi057:mgr.x> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-06T22:04:54.818 INFO:tasks.cephadm:Bootstrapping... 2023-10-06T22:04:54.818 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c -v bootstrap --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.57 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:04:54.840 INFO:journalctl@ceph.mon.a.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:04:54.890 INFO:journalctl@ceph.mgr.x.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:04:55.058 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-06T22:04:55.058 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c', '-v', 'bootstrap', '--fsid', '495b2048-6494-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.57', '--skip-admin-label'] 2023-10-06T22:04:55.231 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.1.1 2023-10-06T22:04:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-06T22:04:55.232 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying podman|docker is present... 2023-10-06T22:04:55.403 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.1.1 2023-10-06T22:04:55.403 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying lvm2 is present... 2023-10-06T22:04:55.403 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying time synchronization is in place... 2023-10-06T22:04:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-06T22:04:55.417 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-06T22:04:55.430 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-06T22:04:55.430 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-06T22:04:55.444 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2023-10-06T22:04:55.457 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2023-10-06T22:04:55.458 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2023-10-06T22:04:55.458 INFO:teuthology.orchestra.run.smithi057.stdout:Repeating the final host check... 2023-10-06T22:04:55.629 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.1.1 2023-10-06T22:04:55.629 INFO:teuthology.orchestra.run.smithi057.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-06T22:04:55.629 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl is present 2023-10-06T22:04:55.629 INFO:teuthology.orchestra.run.smithi057.stdout:lvcreate is present 2023-10-06T22:04:55.642 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-06T22:04:55.643 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-06T22:04:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-06T22:04:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-06T22:04:55.669 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2023-10-06T22:04:55.683 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2023-10-06T22:04:55.683 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2023-10-06T22:04:55.683 INFO:teuthology.orchestra.run.smithi057.stdout:Host looks OK 2023-10-06T22:04:55.683 INFO:teuthology.orchestra.run.smithi057.stdout:Cluster fsid: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:04:55.684 INFO:teuthology.orchestra.run.smithi057.stdout:Acquiring lock 140716557262128 on /run/cephadm/495b2048-6494-11ee-8db5-212e2dc638e7.lock 2023-10-06T22:04:55.684 INFO:teuthology.orchestra.run.smithi057.stdout:Lock 140716557262128 acquired on /run/cephadm/495b2048-6494-11ee-8db5-212e2dc638e7.lock 2023-10-06T22:04:55.684 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 3300 ... 2023-10-06T22:04:55.685 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 6789 ... 2023-10-06T22:04:55.685 INFO:teuthology.orchestra.run.smithi057.stdout:Base mon IP(s) is [172.21.15.57:3300, 172.21.15.57:6789], mon addrv is [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2023-10-06T22:04:55.692 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-10-06T22:04:55.692 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.57 metric 100 2023-10-06T22:04:55.699 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-06T22:04:55.699 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-10-06T22:04:55.699 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2023-10-06T22:04:55.706 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2023-10-06T22:04:55.707 INFO:teuthology.orchestra.run.smithi057.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-06T22:04:55.708 INFO:teuthology.orchestra.run.smithi057.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-06T22:04:55.709 INFO:teuthology.orchestra.run.smithi057.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T22:04:56.182 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14 2023-10-06T22:04:56.182 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T22:04:56.183 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Getting image source signatures 2023-10-06T22:04:56.183 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:a951a90ba2bd822dba777faf493a1625bb5a2f5c5042f0fdc315d633d2c600c8 2023-10-06T22:04:56.183 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:be9ed931e0f930233626f1c4d225977fbcc33e056e1a8fc67c7f63553d9455a2 2023-10-06T22:04:56.183 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying config sha256:c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14 2023-10-06T22:04:56.183 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-06T22:04:56.183 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Storing signatures 2023-10-06T22:04:57.163 INFO:teuthology.orchestra.run.smithi057.stdout:ceph: stdout ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:04:57.163 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph version: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:04:57.164 INFO:teuthology.orchestra.run.smithi057.stdout:Extracting ceph user uid/gid from container image... 2023-10-06T22:04:57.940 INFO:teuthology.orchestra.run.smithi057.stdout:stat: stdout 167 167 2023-10-06T22:04:57.940 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial keys... 2023-10-06T22:04:58.716 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCKhCBlrazbEhAA4G1Q7tpdU1j0ZiT/Tao+ag== 2023-10-06T22:04:59.490 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCLhCBlEUdsBRAAjlKo/mLhqyq2ud6Rau7Syg== 2023-10-06T22:05:00.368 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCLhCBlZVgZNBAAIhr0ykUfkb8umMNlrLTNfg== 2023-10-06T22:05:00.369 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial monmap... 2023-10-06T22:05:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-06T22:05:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-06T22:05:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-06T22:05:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool for a [v2:172.21.15.57:3300,v1:172.21.15.57:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-06T22:05:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = pacific 2023-10-06T22:05:01.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: set fsid to 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:01.247 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-06T22:05:01.247 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:05:01.247 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mon... 2023-10-06T22:05:02.028 INFO:teuthology.orchestra.run.smithi057.stdout:create mon.a on 2023-10-06T22:05:02.285 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-06T22:05:02.502 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-06T22:05:02.906 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-495b2048-6494-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-495b2048-6494-11ee-8db5-212e2dc638e7.target. 2023-10-06T22:05:02.907 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-495b2048-6494-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-495b2048-6494-11ee-8db5-212e2dc638e7.target. 2023-10-06T22:05:03.304 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a 2023-10-06T22:05:03.304 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-06T22:05:03.507 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-495b2048-6494-11ee-8db5-212e2dc638e7.target.wants/ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-495b2048-6494-11ee-8db5-212e2dc638e7@.service. 2023-10-06T22:05:03.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:03 smithi057 systemd[1]: Starting Ceph mon.a for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:05:04.294 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:04 smithi057 podman[97952]: 2023-10-06T22:05:04.294 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:04 smithi057 bash[97952]: 5fe2e026ddb842b1c54c23964cdde6e995801897cead7854bbb57c93c661beae 2023-10-06T22:05:04.320 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T22:05:04.320 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-06T22:05:04.336 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T22:05:04.337 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-06T22:05:04.337 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-06T22:05:04.337 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2023-10-06T22:05:04.337 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon to start... 2023-10-06T22:05:04.337 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon... 2023-10-06T22:05:04.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:04 smithi057 ceph-mon[97992]: mkfs 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:04.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:04 smithi057 ceph-mon[97992]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T22:05:04.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:04 smithi057 systemd[1]: Started Ceph mon.a for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:05:05.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout cluster: 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout id: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout services: 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.667083s) 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout data: 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-06T22:05:05.323 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pgs: 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.866+0000 7f00e31d7700 1 Processor -- start 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.866+0000 7f00e31d7700 1 -- start start 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.866+0000 7f00e31d7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc07a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.866+0000 7f00e31d7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc07a720 con 0x7f00dc07bcf0 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.867+0000 7f00e0f73700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc07a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.867+0000 7f00e0f73700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc07a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58934/0 (socket says 172.21.15.57:58934) 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.867+0000 7f00e0f73700 1 -- 172.21.15.57:0/944053471 learned_addr learned my addr 172.21.15.57:0/944053471 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.868+0000 7f00e0f73700 1 -- 172.21.15.57:0/944053471 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc07a860 con 0x7f00dc07bcf0 2023-10-06T22:05:05.324 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.868+0000 7f00e0f73700 1 --2- 172.21.15.57:0/944053471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc07a150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f00d00098a0 tx=0x7f00d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572e0a7742ab8d11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.868+0000 7f00db7fe700 1 -- 172.21.15.57:0/944053471 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d0004160 con 0x7f00dc07bcf0 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.868+0000 7f00db7fe700 1 -- 172.21.15.57:0/944053471 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d00042c0 con 0x7f00dc07bcf0 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.868+0000 7f00db7fe700 1 -- 172.21.15.57:0/944053471 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d00045a0 con 0x7f00dc07bcf0 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.869+0000 7f00e31d7700 1 -- 172.21.15.57:0/944053471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 msgr2=0x7f00dc07a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.869+0000 7f00e31d7700 1 --2- 172.21.15.57:0/944053471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc07a150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f00d00098a0 tx=0x7f00d0009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.869+0000 7f00e31d7700 1 -- 172.21.15.57:0/944053471 shutdown_connections 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.869+0000 7f00e31d7700 1 --2- 172.21.15.57:0/944053471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc07a150 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.869+0000 7f00e31d7700 1 -- 172.21.15.57:0/944053471 >> 172.21.15.57:0/944053471 conn(0x7f00dc101e80 msgr2=0x7f00dc1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.869+0000 7f00e31d7700 1 -- 172.21.15.57:0/944053471 shutdown_connections 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.869+0000 7f00e31d7700 1 -- 172.21.15.57:0/944053471 wait complete. 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.870+0000 7f00e31d7700 1 Processor -- start 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.870+0000 7f00e31d7700 1 -- start start 2023-10-06T22:05:05.325 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.870+0000 7f00e31d7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.870+0000 7f00e31d7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc19c470 con 0x7f00dc07bcf0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.871+0000 7f00e0f73700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.871+0000 7f00e0f73700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc19bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58936/0 (socket says 172.21.15.57:58936) 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.871+0000 7f00e0f73700 1 -- 172.21.15.57:0/3319623070 learned_addr learned my addr 172.21.15.57:0/3319623070 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.871+0000 7f00e0f73700 1 -- 172.21.15.57:0/3319623070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009550 con 0x7f00dc07bcf0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.871+0000 7f00e0f73700 1 --2- 172.21.15.57:0/3319623070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc19bf30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f00dc07b1c0 tx=0x7f00d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.871+0000 7f00d9ffb700 1 -- 172.21.15.57:0/3319623070 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d0011630 con 0x7f00dc07bcf0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.872+0000 7f00d9ffb700 1 -- 172.21.15.57:0/3319623070 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d0011790 con 0x7f00dc07bcf0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.872+0000 7f00d9ffb700 1 -- 172.21.15.57:0/3319623070 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d0011a30 con 0x7f00dc07bcf0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.872+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc19c670 con 0x7f00dc07bcf0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.872+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc19ca90 con 0x7f00dc07bcf0 2023-10-06T22:05:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.872+0000 7f00d9ffb700 1 -- 172.21.15.57:0/3319623070 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f00d0022070 con 0x7f00dc07bcf0 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.872+0000 7f00d9ffb700 1 -- 172.21.15.57:0/3319623070 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f00d001c450 con 0x7f00dc07bcf0 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.873+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc195e10 con 0x7f00dc07bcf0 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.876+0000 7f00d9ffb700 1 -- 172.21.15.57:0/3319623070 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f00d001b450 con 0x7f00dc07bcf0 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.952+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f00dc07abd0 con 0x7f00dc07bcf0 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.952+0000 7f00d9ffb700 1 -- 172.21.15.57:0/3319623070 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f00d001b5f0 con 0x7f00dc07bcf0 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.953+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 msgr2=0x7f00dc19bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.953+0000 7f00e31d7700 1 --2- 172.21.15.57:0/3319623070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc19bf30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f00dc07b1c0 tx=0x7f00d0004750 comp rx=0 tx=0).stop 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.953+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 shutdown_connections 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.953+0000 7f00e31d7700 1 --2- 172.21.15.57:0/3319623070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00dc07bcf0 0x7f00dc19bf30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.953+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 >> 172.21.15.57:0/3319623070 conn(0x7f00dc101e80 msgr2=0x7f00dc103800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:05.327 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.953+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 shutdown_connections 2023-10-06T22:05:05.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:04.953+0000 7f00e31d7700 1 -- 172.21.15.57:0/3319623070 wait complete. 2023-10-06T22:05:05.328 INFO:teuthology.orchestra.run.smithi057.stdout:mon is available 2023-10-06T22:05:05.328 INFO:teuthology.orchestra.run.smithi057.stdout:Assimilating anything we can from ceph.conf... 2023-10-06T22:05:05.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:05 smithi057 ceph-mon[97992]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T22:05:05.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:05 smithi057 ceph-mon[97992]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} removed_ranks: {} 2023-10-06T22:05:05.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:05 smithi057 ceph-mon[97992]: fsmap 2023-10-06T22:05:05.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:05 smithi057 ceph-mon[97992]: osdmap e1: 0 total, 0 up, 0 in 2023-10-06T22:05:05.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:05 smithi057 ceph-mon[97992]: mgrmap e1: no daemons active 2023-10-06T22:05:05.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:05 smithi057 ceph-mon[97992]: from='client.? 172.21.15.57:0/3319623070' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:06.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.863+0000 7f6fad8f2700 1 Processor -- start 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.864+0000 7f6fad8f2700 1 -- start start 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.864+0000 7f6fad8f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa807a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:06.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.864+0000 7f6fad8f2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa807a740 con 0x7f6fa807bd10 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.864+0000 7f6fa6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa807a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.865+0000 7f6fa6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa807a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58944/0 (socket says 172.21.15.57:58944) 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.865+0000 7f6fa6ffd700 1 -- 172.21.15.57:0/3862486042 learned_addr learned my addr 172.21.15.57:0/3862486042 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.865+0000 7f6fa6ffd700 1 -- 172.21.15.57:0/3862486042 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa807a880 con 0x7f6fa807bd10 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.865+0000 7f6fa6ffd700 1 --2- 172.21.15.57:0/3862486042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa807a170 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6f980098a0 tx=0x7f6f98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2a524bca5c756e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.865+0000 7f6fa5ffb700 1 -- 172.21.15.57:0/3862486042 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f98004160 con 0x7f6fa807bd10 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.865+0000 7f6fa5ffb700 1 -- 172.21.15.57:0/3862486042 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f980042c0 con 0x7f6fa807bd10 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.865+0000 7f6fa5ffb700 1 -- 172.21.15.57:0/3862486042 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f980045a0 con 0x7f6fa807bd10 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.866+0000 7f6fad8f2700 1 -- 172.21.15.57:0/3862486042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 msgr2=0x7f6fa807a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.866+0000 7f6fad8f2700 1 --2- 172.21.15.57:0/3862486042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa807a170 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6f980098a0 tx=0x7f6f98009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:06.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.866+0000 7f6fad8f2700 1 -- 172.21.15.57:0/3862486042 shutdown_connections 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.866+0000 7f6fad8f2700 1 --2- 172.21.15.57:0/3862486042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa807a170 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.866+0000 7f6fad8f2700 1 -- 172.21.15.57:0/3862486042 >> 172.21.15.57:0/3862486042 conn(0x7f6fa8101e80 msgr2=0x7f6fa81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.866+0000 7f6fad8f2700 1 -- 172.21.15.57:0/3862486042 shutdown_connections 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.866+0000 7f6fad8f2700 1 -- 172.21.15.57:0/3862486042 wait complete. 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.867+0000 7f6fad8f2700 1 Processor -- start 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.868+0000 7f6fad8f2700 1 -- start start 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.868+0000 7f6fad8f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.868+0000 7f6fad8f2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa819c480 con 0x7f6fa807bd10 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.868+0000 7f6fa6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.868+0000 7f6fa6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa819bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58948/0 (socket says 172.21.15.57:58948) 2023-10-06T22:05:06.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.868+0000 7f6fa6ffd700 1 -- 172.21.15.57:0/4187574618 learned_addr learned my addr 172.21.15.57:0/4187574618 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.868+0000 7f6fa6ffd700 1 -- 172.21.15.57:0/4187574618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98009550 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.869+0000 7f6fa6ffd700 1 --2- 172.21.15.57:0/4187574618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa819bf40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6fa807b1e0 tx=0x7f6f98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.869+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f98011630 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.869+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98011790 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.869+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f98011a30 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.869+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa819c680 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.869+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa819caa0 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.869+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6f98022070 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.870+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6f9801c450 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.870+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa8195e60 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.873+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f6f9801b450 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.946+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6fa807ae20 con 0x7f6fa807bd10 2023-10-06T22:05:06.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.951+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f6f9801b5f0 con 0x7f6fa807bd10 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.951+0000 7f6fac8f0700 1 -- 172.21.15.57:0/4187574618 <== mon.0 v2:172.21.15.57:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f9801b790 con 0x7f6fa807bd10 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.952+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 msgr2=0x7f6fa819bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.952+0000 7f6fad8f2700 1 --2- 172.21.15.57:0/4187574618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa819bf40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6fa807b1e0 tx=0x7f6f98004750 comp rx=0 tx=0).stop 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.952+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 shutdown_connections 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.952+0000 7f6fad8f2700 1 --2- 172.21.15.57:0/4187574618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fa807bd10 0x7f6fa819bf40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.952+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 >> 172.21.15.57:0/4187574618 conn(0x7f6fa8101e80 msgr2=0x7f6fa81037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.952+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 shutdown_connections 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:05.953+0000 7f6fad8f2700 1 -- 172.21.15.57:0/4187574618 wait complete. 2023-10-06T22:05:06.310 INFO:teuthology.orchestra.run.smithi057.stdout:Generating new minimal ceph.conf... 2023-10-06T22:05:06.568 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:06 smithi057 ceph-mon[97992]: from='client.? 172.21.15.57:0/4187574618' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-06T22:05:06.568 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:06 smithi057 ceph-mon[97992]: from='client.? 172.21.15.57:0/4187574618' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-06T22:05:07.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.847+0000 7fc52c70e700 1 Processor -- start 2023-10-06T22:05:07.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.847+0000 7fc52c70e700 1 -- start start 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.847+0000 7fc52c70e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52407a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.847+0000 7fc52c70e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52407a740 con 0x7fc52407bd10 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.847+0000 7fc52a4aa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52407a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.847+0000 7fc52a4aa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52407a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58964/0 (socket says 172.21.15.57:58964) 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.847+0000 7fc52a4aa700 1 -- 172.21.15.57:0/4170001819 learned_addr learned my addr 172.21.15.57:0/4170001819 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.848+0000 7fc52a4aa700 1 -- 172.21.15.57:0/4170001819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52407a880 con 0x7fc52407bd10 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.848+0000 7fc52a4aa700 1 --2- 172.21.15.57:0/4170001819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52407a170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc5200098a0 tx=0x7fc520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07ee574f41bf50d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.848+0000 7fc5294a8700 1 -- 172.21.15.57:0/4170001819 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc520004160 con 0x7fc52407bd10 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.848+0000 7fc5294a8700 1 -- 172.21.15.57:0/4170001819 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5200042c0 con 0x7fc52407bd10 2023-10-06T22:05:07.386 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.848+0000 7fc5294a8700 1 -- 172.21.15.57:0/4170001819 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc520011a00 con 0x7fc52407bd10 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.849+0000 7fc52c70e700 1 -- 172.21.15.57:0/4170001819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 msgr2=0x7fc52407a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.849+0000 7fc52c70e700 1 --2- 172.21.15.57:0/4170001819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52407a170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc5200098a0 tx=0x7fc520009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.849+0000 7fc52c70e700 1 -- 172.21.15.57:0/4170001819 shutdown_connections 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.849+0000 7fc52c70e700 1 --2- 172.21.15.57:0/4170001819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52407a170 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.849+0000 7fc52c70e700 1 -- 172.21.15.57:0/4170001819 >> 172.21.15.57:0/4170001819 conn(0x7fc524101d80 msgr2=0x7fc5241041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.849+0000 7fc52c70e700 1 -- 172.21.15.57:0/4170001819 shutdown_connections 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.849+0000 7fc52c70e700 1 -- 172.21.15.57:0/4170001819 wait complete. 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.850+0000 7fc52c70e700 1 Processor -- start 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.850+0000 7fc52c70e700 1 -- start start 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.851+0000 7fc52c70e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:07.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.851+0000 7fc52c70e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52419cc60 con 0x7fc52407bd10 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.851+0000 7fc52a4aa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.851+0000 7fc52a4aa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58978/0 (socket says 172.21.15.57:58978) 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.851+0000 7fc52a4aa700 1 -- 172.21.15.57:0/2688458169 learned_addr learned my addr 172.21.15.57:0/2688458169 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.851+0000 7fc52a4aa700 1 -- 172.21.15.57:0/2688458169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc520009550 con 0x7fc52407bd10 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.851+0000 7fc52a4aa700 1 --2- 172.21.15.57:0/2688458169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52419c720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc520009130 tx=0x7fc520011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.852+0000 7fc51b7fe700 1 -- 172.21.15.57:0/2688458169 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc520003b70 con 0x7fc52407bd10 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.852+0000 7fc51b7fe700 1 -- 172.21.15.57:0/2688458169 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc520003cd0 con 0x7fc52407bd10 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.852+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52419ce60 con 0x7fc52407bd10 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.852+0000 7fc51b7fe700 1 -- 172.21.15.57:0/2688458169 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5200197d0 con 0x7fc52407bd10 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.852+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52419d280 con 0x7fc52407bd10 2023-10-06T22:05:07.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.852+0000 7fc51b7fe700 1 -- 172.21.15.57:0/2688458169 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc5200199c0 con 0x7fc52407bd10 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.852+0000 7fc51b7fe700 1 -- 172.21.15.57:0/2688458169 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc52000b720 con 0x7fc52407bd10 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.853+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5240624e0 con 0x7fc52407bd10 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.856+0000 7fc51b7fe700 1 -- 172.21.15.57:0/2688458169 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc52002a060 con 0x7fc52407bd10 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.929+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc52410ce10 con 0x7fc52407bd10 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.929+0000 7fc51b7fe700 1 -- 172.21.15.57:0/2688458169 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fc520019db0 con 0x7fc52407bd10 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.930+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 msgr2=0x7fc52419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.930+0000 7fc52c70e700 1 --2- 172.21.15.57:0/2688458169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52419c720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc520009130 tx=0x7fc520011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.930+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 shutdown_connections 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.930+0000 7fc52c70e700 1 --2- 172.21.15.57:0/2688458169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc52407bd10 0x7fc52419c720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.930+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 >> 172.21.15.57:0/2688458169 conn(0x7fc524101d80 msgr2=0x7fc5241034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.930+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 shutdown_connections 2023-10-06T22:05:07.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:06.930+0000 7fc52c70e700 1 -- 172.21.15.57:0/2688458169 wait complete. 2023-10-06T22:05:07.390 INFO:teuthology.orchestra.run.smithi057.stdout:Restarting the monitor... 2023-10-06T22:05:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:07 smithi057 ceph-mon[97992]: from='client.? 172.21.15.57:0/2688458169' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:05:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:07 smithi057 systemd[1]: Stopping Ceph mon.a for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:05:09.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a[97968]: 2023-10-06T22:05:09.044+0000 7f6e9c19f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:05:09.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a[97968]: 2023-10-06T22:05:09.044+0000 7f6e9c19f700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-06T22:05:09.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:09 smithi057 bash[98329]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a 2023-10-06T22:05:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:09 smithi057 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-06T22:05:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:09 smithi057 systemd[1]: Stopped Ceph mon.a for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:05:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:09 smithi057 systemd[1]: Starting Ceph mon.a for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:05:10.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 podman[98452]: 2023-10-06T22:05:10.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 bash[98452]: 3c01fa3d03fa959771fd920d5a720a09b82d29c1b0f71ce3ef91e5bbc008d8c6 2023-10-06T22:05:10.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T22:05:10.581 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 7 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: pidfile_write: ignore empty --pid-file 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: load: jerasure load: lrc 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: RocksDB version: 7.9.2 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Git sha 0 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: DB SUMMARY 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: DB Session ID: R53OG8F0VUOOJC6CLSER 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: CURRENT file: CURRENT 2023-10-06T22:05:10.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100127 ; 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.error_if_exists: 0 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.create_if_missing: 0 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.paranoid_checks: 1 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.env: 0x561b378b4da0 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T22:05:10.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.info_log: 0x561b386f95c0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.statistics: (nil) 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.use_fsync: 0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_log_file_size: 0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.allow_fallocate: 1 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.use_direct_reads: 0 2023-10-06T22:05:10.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T22:05:10.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T22:05:10.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.db_log_dir: 2023-10-06T22:05:10.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.wal_dir: 2023-10-06T22:05:10.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T22:05:10.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T22:05:10.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T22:05:10.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T22:05:10.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T22:05:10.586 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.write_buffer_manager: 0x561b386b1360 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T22:05:10.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.unordered_write: 0 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.row_cache: None 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.wal_filter: None 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.two_write_queues: 0 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.wal_compression: 0 2023-10-06T22:05:10.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.atomic_flush: 0 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.log_readahead_size: 0 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T22:05:10.589 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_background_jobs: 2 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_background_compactions: -1 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_subcompactions: 1 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_open_files: -1 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T22:05:10.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_background_flushes: -1 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Compression algorithms supported: 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kZSTD supported: 0 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kXpressCompression supported: 0 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kZlibCompression supported: 1 2023-10-06T22:05:10.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kSnappyCompression supported: 1 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kLZ4Compression supported: 1 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: kBZip2Compression supported: 0 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.merge_operator: 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_filter: None 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T22:05:10.593 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561b386f96c0) 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T22:05:10.594 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 4 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x561b3874f1f0 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2023-10-06T22:05:10.595 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: bloomfilter 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 5 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: max_auto_readahead_size: 262144 2023-10-06T22:05:10.596 INFO:journalctl@ceph.mon.a.smithi057.stdout: prepopulate_block_cache: 0 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout: initial_auto_readahead_size: 8192 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression: NoCompression 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.num_levels: 7 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T22:05:10.597 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T22:05:10.598 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T22:05:10.599 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T22:05:10.600 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T22:05:10.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.table_properties_collectors: 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.inplace_update_support: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.bloom_locality: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.max_successive_merges: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T22:05:10.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.ttl: 2592000 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.enable_blob_files: false 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.min_blob_size: 0 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T22:05:10.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5222aef1-966d-4c97-8d41-84716f864b4f 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696629910513874, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696629910515710, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95247, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 93235, "index_size": 294, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13993, "raw_average_key_size": 50, "raw_value_size": 85615, "raw_average_value_size": 306, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696629910, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5222aef1-966d-4c97-8d41-84716f864b4f", "db_session_id": "R53OG8F0VUOOJC6CLSER", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696629910515845, "job": 1, "event": "recovery_finished"} 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561b387ec000 2023-10-06T22:05:10.604 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: DB pointer 0x561b387d8000 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-06T22:05:10.605 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 94.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 94.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-06T22:05:10.606 INFO:journalctl@ceph.mon.a.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 13.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 13.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache BinnedLRUCache@0x561b3874f1f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-10-06T22:05:10.607 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: starting mon.a rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???) e1 preinit fsid 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???).mds e1 new map 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???).mds e1 print_map 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-06T22:05:10.608 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 systemd[1]: Started Ceph mon.a for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} removed_ranks: {} 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: fsmap 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: osdmap e1: 0 total, 0 up, 0 in 2023-10-06T22:05:10.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:10 smithi057 ceph-mon[98492]: mgrmap e1: no daemons active 2023-10-06T22:05:10.610 INFO:teuthology.orchestra.run.smithi057.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-06T22:05:11.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.086+0000 7f4df5c3b700 1 Processor -- start 2023-10-06T22:05:11.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.086+0000 7f4df5c3b700 1 -- start start 2023-10-06T22:05:11.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.086+0000 7f4df5c3b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df0106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.086+0000 7f4df5c3b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0107280 con 0x7f4df0106890 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.087+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df0106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.087+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df0106cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58980/0 (socket says 172.21.15.57:58980) 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.087+0000 7f4def7fe700 1 -- 172.21.15.57:0/3798159101 learned_addr learned my addr 172.21.15.57:0/3798159101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.088+0000 7f4def7fe700 1 -- 172.21.15.57:0/3798159101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df0107a90 con 0x7f4df0106890 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.088+0000 7f4def7fe700 1 --2- 172.21.15.57:0/3798159101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df0106cb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4de00098a0 tx=0x7f4de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8480626002d7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.088+0000 7f4dee7fc700 1 -- 172.21.15.57:0/3798159101 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4de0004160 con 0x7f4df0106890 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.088+0000 7f4dee7fc700 1 -- 172.21.15.57:0/3798159101 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4de00042c0 con 0x7f4df0106890 2023-10-06T22:05:11.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.088+0000 7f4dee7fc700 1 -- 172.21.15.57:0/3798159101 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4de0011a00 con 0x7f4df0106890 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.089+0000 7f4df5c3b700 1 -- 172.21.15.57:0/3798159101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 msgr2=0x7f4df0106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.089+0000 7f4df5c3b700 1 --2- 172.21.15.57:0/3798159101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df0106cb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4de00098a0 tx=0x7f4de0009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.089+0000 7f4df5c3b700 1 -- 172.21.15.57:0/3798159101 shutdown_connections 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.089+0000 7f4df5c3b700 1 --2- 172.21.15.57:0/3798159101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df0106cb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.089+0000 7f4df5c3b700 1 -- 172.21.15.57:0/3798159101 >> 172.21.15.57:0/3798159101 conn(0x7f4df0101e50 msgr2=0x7f4df0104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.089+0000 7f4df5c3b700 1 -- 172.21.15.57:0/3798159101 shutdown_connections 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.089+0000 7f4df5c3b700 1 -- 172.21.15.57:0/3798159101 wait complete. 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.090+0000 7f4df5c3b700 1 Processor -- start 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.090+0000 7f4df5c3b700 1 -- start start 2023-10-06T22:05:11.573 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.091+0000 7f4df5c3b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.091+0000 7f4df5c3b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df019aad0 con 0x7f4df0106890 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.091+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.091+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58986/0 (socket says 172.21.15.57:58986) 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.091+0000 7f4def7fe700 1 -- 172.21.15.57:0/411970865 learned_addr learned my addr 172.21.15.57:0/411970865 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.091+0000 7f4def7fe700 1 -- 172.21.15.57:0/411970865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0009550 con 0x7f4df0106890 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4def7fe700 1 --2- 172.21.15.57:0/411970865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df019a590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009130 tx=0x7f4de0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4decff9700 1 -- 172.21.15.57:0/411970865 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4de0003ab0 con 0x7f4df0106890 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4decff9700 1 -- 172.21.15.57:0/411970865 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4de0003c10 con 0x7f4df0106890 2023-10-06T22:05:11.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4decff9700 1 -- 172.21.15.57:0/411970865 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4de00247d0 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df019acd0 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df019b0f0 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4decff9700 1 -- 172.21.15.57:0/411970865 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4de0024d00 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.092+0000 7f4decff9700 1 -- 172.21.15.57:0/411970865 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4de00193c0 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.093+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df00624e0 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.096+0000 7f4decff9700 1 -- 172.21.15.57:0/411970865 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4de001abb0 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.169+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f4df010acf0 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.172+0000 7f4decff9700 1 -- 172.21.15.57:0/411970865 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4de001ad50 con 0x7f4df0106890 2023-10-06T22:05:11.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.173+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 msgr2=0x7f4df019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:11.576 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.173+0000 7f4df5c3b700 1 --2- 172.21.15.57:0/411970865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df019a590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009130 tx=0x7f4de0011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:11.576 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.173+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 shutdown_connections 2023-10-06T22:05:11.576 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.173+0000 7f4df5c3b700 1 --2- 172.21.15.57:0/411970865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4df0106890 0x7f4df019a590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:11.576 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.173+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 >> 172.21.15.57:0/411970865 conn(0x7f4df0101e50 msgr2=0x7f4df0103540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:11.576 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.173+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 shutdown_connections 2023-10-06T22:05:11.576 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:11.173+0000 7f4df5c3b700 1 -- 172.21.15.57:0/411970865 wait complete. 2023-10-06T22:05:11.576 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-06T22:05:11.598 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:05:11.598 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mgr... 2023-10-06T22:05:11.598 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-06T22:05:11.599 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-06T22:05:11.976 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.x 2023-10-06T22:05:11.976 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-10-06T22:05:12.190 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-495b2048-6494-11ee-8db5-212e2dc638e7.target.wants/ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-495b2048-6494-11ee-8db5-212e2dc638e7@.service. 2023-10-06T22:05:12.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:12 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/411970865' entity='client.admin' 2023-10-06T22:05:12.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:12 smithi057 systemd[1]: Starting Ceph mgr.x for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:05:12.978 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:12 smithi057 podman[98722]: 2023-10-06T22:05:12.978 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:12 smithi057 bash[98722]: b055af86806176ac558fcabe56ca00725f4f261b9cd7486118890e659240c1a6 2023-10-06T22:05:13.001 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T22:05:13.001 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-06T22:05:13.018 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T22:05:13.018 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-06T22:05:13.018 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-06T22:05:13.018 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2023-10-06T22:05:13.036 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T22:05:13.036 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-06T22:05:13.053 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T22:05:13.053 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-06T22:05:13.053 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-06T22:05:13.053 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-06T22:05:13.053 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr to start... 2023-10-06T22:05:13.053 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr... 2023-10-06T22:05:13.341 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:12 smithi057 systemd[1]: Started Ceph mgr.x for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:05:13.341 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:13.103+0000 7f7c56eab200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:05:13.341 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:13.221+0000 7f7c56eab200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:05:13.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:13.341+0000 7f7c56eab200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:05:13.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:13.471+0000 7f7c56eab200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:05:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:13 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3513896734' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-06T22:05:14.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:13.716+0000 7f7c56eab200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:05:14.236 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:14.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-06T22:05:14.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-06T22:05:14.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2023-10-06T22:05:14.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:14.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-06T22:05:14.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2023-10-06T22:05:14.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:14.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-06T22:05:14.242 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-06T22:05:14.242 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:14.242 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-06T22:05:14.242 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-06T22:05:14.242 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:14.242 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-06T22:05:14.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:14.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-06T22:05:14.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-06T22:05:14.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-06T22:05:14.253 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-06T22:05:14.253 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-06T22:05:14.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-06T22:05:14.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2023-10-06T22:05:04.289868+0000", 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.620+0000 7f371d46f700 1 Processor -- start 2023-10-06T22:05:14.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.620+0000 7f371d46f700 1 -- start start 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.620+0000 7f371d46f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371807a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.620+0000 7f371d46f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371807a740 con 0x7f371807bd10 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.621+0000 7f3716ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371807a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.621+0000 7f3716ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371807a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44876/0 (socket says 172.21.15.57:44876) 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.621+0000 7f3716ffd700 1 -- 172.21.15.57:0/3482497389 learned_addr learned my addr 172.21.15.57:0/3482497389 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.621+0000 7f3716ffd700 1 -- 172.21.15.57:0/3482497389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371807a880 con 0x7f371807bd10 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.621+0000 7f3716ffd700 1 --2- 172.21.15.57:0/3482497389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371807a170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f37080098a0 tx=0x7f3708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2919259bbf85ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f3715ffb700 1 -- 172.21.15.57:0/3482497389 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3708004160 con 0x7f371807bd10 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f3715ffb700 1 -- 172.21.15.57:0/3482497389 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37080042c0 con 0x7f371807bd10 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f3715ffb700 1 -- 172.21.15.57:0/3482497389 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3708011a00 con 0x7f371807bd10 2023-10-06T22:05:14.257 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f371d46f700 1 -- 172.21.15.57:0/3482497389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 msgr2=0x7f371807a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f371d46f700 1 --2- 172.21.15.57:0/3482497389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371807a170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f37080098a0 tx=0x7f3708009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f371d46f700 1 -- 172.21.15.57:0/3482497389 shutdown_connections 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f371d46f700 1 --2- 172.21.15.57:0/3482497389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371807a170 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.622+0000 7f371d46f700 1 -- 172.21.15.57:0/3482497389 >> 172.21.15.57:0/3482497389 conn(0x7f3718101e80 msgr2=0x7f37181042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.623+0000 7f371d46f700 1 -- 172.21.15.57:0/3482497389 shutdown_connections 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.623+0000 7f371d46f700 1 -- 172.21.15.57:0/3482497389 wait complete. 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.624+0000 7f371d46f700 1 Processor -- start 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.624+0000 7f371d46f700 1 -- start start 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.625+0000 7f371d46f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.625+0000 7f371d46f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371819ccb0 con 0x7f371807bd10 2023-10-06T22:05:14.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.625+0000 7f3716ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.625+0000 7f3716ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371819c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44878/0 (socket says 172.21.15.57:44878) 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.625+0000 7f3716ffd700 1 -- 172.21.15.57:0/3513896734 learned_addr learned my addr 172.21.15.57:0/3513896734 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.625+0000 7f3716ffd700 1 -- 172.21.15.57:0/3513896734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708009550 con 0x7f371807bd10 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.626+0000 7f3716ffd700 1 --2- 172.21.15.57:0/3513896734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371819c770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3708009130 tx=0x7f3708011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.626+0000 7f36fffff700 1 -- 172.21.15.57:0/3513896734 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3708003b70 con 0x7f371807bd10 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.626+0000 7f36fffff700 1 -- 172.21.15.57:0/3513896734 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3708003cd0 con 0x7f371807bd10 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.626+0000 7f36fffff700 1 -- 172.21.15.57:0/3513896734 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37080197d0 con 0x7f371807bd10 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.626+0000 7f371d46f700 1 -- 172.21.15.57:0/3513896734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371819ceb0 con 0x7f371807bd10 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.626+0000 7f371d46f700 1 -- 172.21.15.57:0/3513896734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371819d2d0 con 0x7f371807bd10 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.626+0000 7f36fffff700 1 -- 172.21.15.57:0/3513896734 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f37080199c0 con 0x7f371807bd10 2023-10-06T22:05:14.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.627+0000 7f36fffff700 1 -- 172.21.15.57:0/3513896734 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f370800b720 con 0x7f371807bd10 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.628+0000 7f36fdffb700 1 -- 172.21.15.57:0/3513896734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f80052f0 con 0x7f371807bd10 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.630+0000 7f36fffff700 1 -- 172.21.15.57:0/3513896734 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f370802a060 con 0x7f371807bd10 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.708+0000 7f36fdffb700 1 -- 172.21.15.57:0/3513896734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f36f8005870 con 0x7f371807bd10 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.708+0000 7f36fffff700 1 -- 172.21.15.57:0/3513896734 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3708019db0 con 0x7f371807bd10 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.709+0000 7f36fdffb700 1 -- 172.21.15.57:0/3513896734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 msgr2=0x7f371819c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.709+0000 7f36fdffb700 1 --2- 172.21.15.57:0/3513896734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371819c770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3708009130 tx=0x7f3708011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.709+0000 7f36fdffb700 1 -- 172.21.15.57:0/3513896734 shutdown_connections 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.709+0000 7f36fdffb700 1 --2- 172.21.15.57:0/3513896734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f371807bd10 0x7f371819c770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.709+0000 7f36fdffb700 1 -- 172.21.15.57:0/3513896734 >> 172.21.15.57:0/3513896734 conn(0x7f3718101e80 msgr2=0x7f3718103530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:14.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.709+0000 7f36fdffb700 1 -- 172.21.15.57:0/3513896734 shutdown_connections 2023-10-06T22:05:14.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:13.710+0000 7f36fdffb700 1 -- 172.21.15.57:0/3513896734 wait complete. 2023-10-06T22:05:14.261 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (1/15)... 2023-10-06T22:05:14.515 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:14 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:14.222+0000 7f7c56eab200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:05:14.515 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:14 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:14.392+0000 7f7c56eab200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:05:14.515 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:14 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:14.512+0000 7f7c56eab200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:05:14.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:14 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:14.640+0000 7f7c56eab200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:05:15.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:15 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:15.200+0000 7f7c56eab200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:05:16.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:16 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:16.114+0000 7f7c56eab200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:05:16.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:16 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:16.223+0000 7f7c56eab200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:05:17.301 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:16 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/4172888742' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-06T22:05:17.428 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-06T22:05:17.429 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-06T22:05:17.430 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:17.430 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-06T22:05:17.430 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:17.430 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-06T22:05:17.430 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-06T22:05:17.430 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-06T22:05:17.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-06T22:05:17.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-06T22:05:17.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2023-10-06T22:05:04.289868+0000", 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-06T22:05:17.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.797+0000 7f9dfcba9700 1 Processor -- start 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.797+0000 7f9dfcba9700 1 -- start start 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.797+0000 7f9dfcba9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df8106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.797+0000 7f9dfcba9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8107270 con 0x7f9df8106880 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.797+0000 7f9df659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df8106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.797+0000 7f9df659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df8106ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44888/0 (socket says 172.21.15.57:44888) 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.797+0000 7f9df659c700 1 -- 172.21.15.57:0/3529585691 learned_addr learned my addr 172.21.15.57:0/3529585691 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.798+0000 7f9df659c700 1 -- 172.21.15.57:0/3529585691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8107a80 con 0x7f9df8106880 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.798+0000 7f9df659c700 1 --2- 172.21.15.57:0/3529585691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df8106ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9de80098a0 tx=0x7f9de8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e7c3dc0638e815 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.798+0000 7f9df559a700 1 -- 172.21.15.57:0/3529585691 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de8004160 con 0x7f9df8106880 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.798+0000 7f9df559a700 1 -- 172.21.15.57:0/3529585691 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9de80042c0 con 0x7f9df8106880 2023-10-06T22:05:17.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.798+0000 7f9df559a700 1 -- 172.21.15.57:0/3529585691 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de8011a00 con 0x7f9df8106880 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.799+0000 7f9dfcba9700 1 -- 172.21.15.57:0/3529585691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 msgr2=0x7f9df8106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.799+0000 7f9dfcba9700 1 --2- 172.21.15.57:0/3529585691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df8106ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9de80098a0 tx=0x7f9de8009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.799+0000 7f9dfcba9700 1 -- 172.21.15.57:0/3529585691 shutdown_connections 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.799+0000 7f9dfcba9700 1 --2- 172.21.15.57:0/3529585691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df8106ca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.799+0000 7f9dfcba9700 1 -- 172.21.15.57:0/3529585691 >> 172.21.15.57:0/3529585691 conn(0x7f9df8101e80 msgr2=0x7f9df81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.799+0000 7f9dfcba9700 1 -- 172.21.15.57:0/3529585691 shutdown_connections 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.799+0000 7f9dfcba9700 1 -- 172.21.15.57:0/3529585691 wait complete. 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.801+0000 7f9dfcba9700 1 Processor -- start 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.801+0000 7f9dfcba9700 1 -- start start 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.802+0000 7f9dfcba9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:17.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.802+0000 7f9dfcba9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df819cd00 con 0x7f9df8106880 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.802+0000 7f9df659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.802+0000 7f9df659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44896/0 (socket says 172.21.15.57:44896) 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.802+0000 7f9df659c700 1 -- 172.21.15.57:0/4172888742 learned_addr learned my addr 172.21.15.57:0/4172888742 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.802+0000 7f9df659c700 1 -- 172.21.15.57:0/4172888742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8009550 con 0x7f9df8106880 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.802+0000 7f9df659c700 1 --2- 172.21.15.57:0/4172888742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df819c7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009130 tx=0x7f9de8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.803+0000 7f9de77fe700 1 -- 172.21.15.57:0/4172888742 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de8003ab0 con 0x7f9df8106880 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.803+0000 7f9de77fe700 1 -- 172.21.15.57:0/4172888742 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9de8003c10 con 0x7f9df8106880 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.803+0000 7f9de77fe700 1 -- 172.21.15.57:0/4172888742 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de80247d0 con 0x7f9df8106880 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.803+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df819cf00 con 0x7f9df8106880 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.803+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df819d320 con 0x7f9df8106880 2023-10-06T22:05:17.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.803+0000 7f9de77fe700 1 -- 172.21.15.57:0/4172888742 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9de8024d00 con 0x7f9df8106880 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.803+0000 7f9de77fe700 1 -- 172.21.15.57:0/4172888742 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9de8019450 con 0x7f9df8106880 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.804+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df80624e0 con 0x7f9df8106880 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.807+0000 7f9de77fe700 1 -- 172.21.15.57:0/4172888742 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9de801abb0 con 0x7f9df8106880 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.936+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9df810ace0 con 0x7f9df8106880 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.937+0000 7f9de77fe700 1 -- 172.21.15.57:0/4172888742 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9de801ad50 con 0x7f9df8106880 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.938+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 msgr2=0x7f9df819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.938+0000 7f9dfcba9700 1 --2- 172.21.15.57:0/4172888742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df819c7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009130 tx=0x7f9de8011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.938+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 shutdown_connections 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.938+0000 7f9dfcba9700 1 --2- 172.21.15.57:0/4172888742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9df8106880 0x7f9df819c7c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.938+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 >> 172.21.15.57:0/4172888742 conn(0x7f9df8101e80 msgr2=0x7f9df81035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.938+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 shutdown_connections 2023-10-06T22:05:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:16.938+0000 7f9dfcba9700 1 -- 172.21.15.57:0/4172888742 wait complete. 2023-10-06T22:05:17.445 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (2/15)... 2023-10-06T22:05:17.659 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:17 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:17.298+0000 7f7c56eab200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:05:17.659 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:17 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:17.423+0000 7f7c56eab200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:05:17.659 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:17 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:17.656+0000 7f7c56eab200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:05:18.319 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:18 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:18.035+0000 7f7c56eab200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:05:18.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:18 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:18.316+0000 7f7c56eab200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:05:18.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:18 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:18.515+0000 7f7c56eab200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:05:19.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:18 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:18.870+0000 7f7c56eab200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:05:20.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:19 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:19.849+0000 7f7c56eab200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:05:20.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:20 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:20.029+0000 7f7c56eab200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:05:20.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:20 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:20.285+0000 7f7c56eab200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:05:20.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:20 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:20.436+0000 7f7c56eab200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:05:21.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:20 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:20.725+0000 7f7c56eab200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:05:22.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:21.840+0000 7f7c56eab200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:05:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: Activating manager daemon x 2023-10-06T22:05:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: mgrmap e2: x(active, starting, since 0.00417918s) 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: Manager daemon x is now available 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' 2023-10-06T22:05:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:21 smithi057 ceph-mon[98492]: from='mgr.14100 172.21.15.57:0/527738830' entity='mgr.x' 2023-10-06T22:05:24.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:23 smithi057 ceph-mon[98492]: mgrmap e3: x(active, since 1.01145s) 2023-10-06T22:05:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:24 smithi057 ceph-mon[98492]: mgrmap e4: x(active, since 2s) 2023-10-06T22:05:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:27 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3913835615' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-06T22:05:27.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:27.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-06T22:05:27.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-06T22:05:27.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2023-10-06T22:05:27.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:27.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-06T22:05:27.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2023-10-06T22:05:27.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:27.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-10-06T22:05:27.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-06T22:05:27.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-06T22:05:27.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-06T22:05:27.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-06T22:05:27.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-06T22:05:27.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-06T22:05:27.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:27.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-06T22:05:27.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-06T22:05:27.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-06T22:05:27.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-06T22:05:27.465 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-06T22:05:27.466 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-06T22:05:27.466 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-06T22:05:27.466 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-06T22:05:27.466 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-06T22:05:27.466 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:27.466 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-06T22:05:27.466 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-06T22:05:27.467 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-06T22:05:27.468 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-06T22:05:27.468 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2023-10-06T22:05:27.468 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-06T22:05:27.468 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T22:05:27.468 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:27.468 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-06T22:05:27.468 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T22:05:27.469 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2023-10-06T22:05:04.289868+0000", 2023-10-06T22:05:27.469 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T22:05:27.469 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-06T22:05:27.469 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-06T22:05:27.469 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-06T22:05:27.469 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.738+0000 7fa83eb8e700 1 Processor -- start 2023-10-06T22:05:27.469 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.738+0000 7fa83eb8e700 1 -- start start 2023-10-06T22:05:27.470 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.738+0000 7fa83eb8e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83807a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:27.470 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.738+0000 7fa83eb8e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83807a760 con 0x7fa83807bd30 2023-10-06T22:05:27.470 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.739+0000 7fa83c92a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83807a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:27.470 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.739+0000 7fa83c92a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83807a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41448/0 (socket says 172.21.15.57:41448) 2023-10-06T22:05:27.470 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.739+0000 7fa83c92a700 1 -- 172.21.15.57:0/3291209678 learned_addr learned my addr 172.21.15.57:0/3291209678 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:27.470 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.739+0000 7fa83c92a700 1 -- 172.21.15.57:0/3291209678 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83807a8a0 con 0x7fa83807bd30 2023-10-06T22:05:27.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.739+0000 7fa83c92a700 1 --2- 172.21.15.57:0/3291209678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83807a190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098a0 tx=0x7fa82c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eb62c0d1956628 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:27.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.739+0000 7fa8377fe700 1 -- 172.21.15.57:0/3291209678 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa82c004160 con 0x7fa83807bd30 2023-10-06T22:05:27.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa8377fe700 1 -- 172.21.15.57:0/3291209678 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa82c0042c0 con 0x7fa83807bd30 2023-10-06T22:05:27.471 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa8377fe700 1 -- 172.21.15.57:0/3291209678 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa82c011a00 con 0x7fa83807bd30 2023-10-06T22:05:27.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3291209678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 msgr2=0x7fa83807a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:27.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa83eb8e700 1 --2- 172.21.15.57:0/3291209678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83807a190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098a0 tx=0x7fa82c009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:27.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3291209678 shutdown_connections 2023-10-06T22:05:27.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa83eb8e700 1 --2- 172.21.15.57:0/3291209678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83807a190 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:27.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3291209678 >> 172.21.15.57:0/3291209678 conn(0x7fa838101e80 msgr2=0x7fa8381042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:27.472 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3291209678 shutdown_connections 2023-10-06T22:05:27.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.740+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3291209678 wait complete. 2023-10-06T22:05:27.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.741+0000 7fa83eb8e700 1 Processor -- start 2023-10-06T22:05:27.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.742+0000 7fa83eb8e700 1 -- start start 2023-10-06T22:05:27.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.742+0000 7fa83eb8e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:27.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.742+0000 7fa83eb8e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83819cd30 con 0x7fa83807bd30 2023-10-06T22:05:27.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.742+0000 7fa83c92a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:27.473 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.742+0000 7fa83c92a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41462/0 (socket says 172.21.15.57:41462) 2023-10-06T22:05:27.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.742+0000 7fa83c92a700 1 -- 172.21.15.57:0/3913835615 learned_addr learned my addr 172.21.15.57:0/3913835615 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:27.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.742+0000 7fa83c92a700 1 -- 172.21.15.57:0/3913835615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c009550 con 0x7fa83807bd30 2023-10-06T22:05:27.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.743+0000 7fa83c92a700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83819c7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009130 tx=0x7fa82c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:27.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.743+0000 7fa835ffb700 1 -- 172.21.15.57:0/3913835615 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa82c003ab0 con 0x7fa83807bd30 2023-10-06T22:05:27.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.743+0000 7fa835ffb700 1 -- 172.21.15.57:0/3913835615 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa82c003c10 con 0x7fa83807bd30 2023-10-06T22:05:27.474 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.743+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83819cf30 con 0x7fa83807bd30 2023-10-06T22:05:27.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.743+0000 7fa835ffb700 1 -- 172.21.15.57:0/3913835615 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa82c0247d0 con 0x7fa83807bd30 2023-10-06T22:05:27.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.743+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83819d350 con 0x7fa83807bd30 2023-10-06T22:05:27.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.744+0000 7fa835ffb700 1 -- 172.21.15.57:0/3913835615 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7fa82c01a430 con 0x7fa83807bd30 2023-10-06T22:05:27.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.744+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa838196860 con 0x7fa83807bd30 2023-10-06T22:05:27.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.744+0000 7fa835ffb700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fa828038820 0x7fa82803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:27.475 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.745+0000 7fa835ffb700 1 -- 172.21.15.57:0/3913835615 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa82c04c540 con 0x7fa83807bd30 2023-10-06T22:05:27.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.745+0000 7fa837fff700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fa828038820 0x7fa82803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:27.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.745+0000 7fa837fff700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fa828038820 0x7fa82803ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:27.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:26.751+0000 7fa835ffb700 1 -- 172.21.15.57:0/3913835615 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa82c018530 con 0x7fa83807bd30 2023-10-06T22:05:27.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.043+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa83802d100 con 0x7fa83807bd30 2023-10-06T22:05:27.476 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.043+0000 7fa835ffb700 1 -- 172.21.15.57:0/3913835615 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fa82c018350 con 0x7fa83807bd30 2023-10-06T22:05:27.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.045+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fa828038820 msgr2=0x7fa82803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:27.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.045+0000 7fa83eb8e700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fa828038820 0x7fa82803ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:27.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.045+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 msgr2=0x7fa83819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:27.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.045+0000 7fa83eb8e700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83819c7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009130 tx=0x7fa82c011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:27.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.046+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 shutdown_connections 2023-10-06T22:05:27.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.046+0000 7fa83eb8e700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fa828038820 0x7fa82803ace0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:27.477 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.046+0000 7fa83eb8e700 1 --2- 172.21.15.57:0/3913835615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa83807bd30 0x7fa83819c7f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:27.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.046+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 >> 172.21.15.57:0/3913835615 conn(0x7fa838101e80 msgr2=0x7fa8381035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:27.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.046+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 shutdown_connections 2023-10-06T22:05:27.478 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.046+0000 7fa83eb8e700 1 -- 172.21.15.57:0/3913835615 wait complete. 2023-10-06T22:05:27.478 INFO:teuthology.orchestra.run.smithi057.stdout:mgr is available 2023-10-06T22:05:28.638 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-06T22:05:28.639 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.977+0000 7f92894c0700 1 Processor -- start 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.977+0000 7f92894c0700 1 -- start start 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.977+0000 7f92894c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928407a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.977+0000 7f92894c0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928407a720 con 0x7f928407bcf0 2023-10-06T22:05:28.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.978+0000 7f9282ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928407a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.978+0000 7f9282ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928407a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41472/0 (socket says 172.21.15.57:41472) 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.978+0000 7f9282ffd700 1 -- 172.21.15.57:0/3717863160 learned_addr learned my addr 172.21.15.57:0/3717863160 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.978+0000 7f9282ffd700 1 -- 172.21.15.57:0/3717863160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928407a860 con 0x7f928407bcf0 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.979+0000 7f9282ffd700 1 --2- 172.21.15.57:0/3717863160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928407a150 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f92740098a0 tx=0x7f9274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f5e26e01f47432 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.979+0000 7f9281ffb700 1 -- 172.21.15.57:0/3717863160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9274004160 con 0x7f928407bcf0 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.979+0000 7f9281ffb700 1 -- 172.21.15.57:0/3717863160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92740042c0 con 0x7f928407bcf0 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.979+0000 7f9281ffb700 1 -- 172.21.15.57:0/3717863160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9274011a00 con 0x7f928407bcf0 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.979+0000 7f92894c0700 1 -- 172.21.15.57:0/3717863160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 msgr2=0x7f928407a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.979+0000 7f92894c0700 1 --2- 172.21.15.57:0/3717863160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928407a150 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f92740098a0 tx=0x7f9274009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.980+0000 7f92894c0700 1 -- 172.21.15.57:0/3717863160 shutdown_connections 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.980+0000 7f92894c0700 1 --2- 172.21.15.57:0/3717863160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928407a150 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.980+0000 7f92894c0700 1 -- 172.21.15.57:0/3717863160 >> 172.21.15.57:0/3717863160 conn(0x7f9284101e80 msgr2=0x7f92841042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.980+0000 7f92894c0700 1 -- 172.21.15.57:0/3717863160 shutdown_connections 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.980+0000 7f92894c0700 1 -- 172.21.15.57:0/3717863160 wait complete. 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.981+0000 7f92894c0700 1 Processor -- start 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.981+0000 7f92894c0700 1 -- start start 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.981+0000 7f92894c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.981+0000 7f92894c0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928419ccb0 con 0x7f928407bcf0 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.981+0000 7f9282ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.981+0000 7f9282ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41482/0 (socket says 172.21.15.57:41482) 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f9282ffd700 1 -- 172.21.15.57:0/3483279475 learned_addr learned my addr 172.21.15.57:0/3483279475 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f9282ffd700 1 -- 172.21.15.57:0/3483279475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009550 con 0x7f928407bcf0 2023-10-06T22:05:28.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f9282ffd700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928419c770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9274009130 tx=0x7f9274011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f926bfff700 1 -- 172.21.15.57:0/3483279475 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9274003b70 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f926bfff700 1 -- 172.21.15.57:0/3483279475 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9274003cd0 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f926bfff700 1 -- 172.21.15.57:0/3483279475 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92740197d0 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928419ceb0 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.982+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928419d2d0 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.983+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92840624e0 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.984+0000 7f926bfff700 1 -- 172.21.15.57:0/3483279475 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7f927401abb0 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.984+0000 7f926bfff700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f926c038820 0x7f926c03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.984+0000 7f926bfff700 1 -- 172.21.15.57:0/3483279475 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f927404c9f0 con 0x7f928407bcf0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.984+0000 7f92827fc700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f926c038820 0x7f926c03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:28.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.985+0000 7f92827fc700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f926c038820 0x7f926c03ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9278006fd0 tx=0x7f9278006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:27.989+0000 7f926bfff700 1 -- 172.21.15.57:0/3483279475 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f927401a430 con 0x7f928407bcf0 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.201+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f928410cc30 con 0x7f928407bcf0 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.202+0000 7f926bfff700 1 -- 172.21.15.57:0/3483279475 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9274023370 con 0x7f928407bcf0 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f926c038820 msgr2=0x7f926c03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f926c038820 0x7f926c03ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9278006fd0 tx=0x7f9278006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 msgr2=0x7f928419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928419c770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9274009130 tx=0x7f9274011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 shutdown_connections 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f926c038820 0x7f926c03ace0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 --2- 172.21.15.57:0/3483279475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f928407bcf0 0x7f928419c770 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.204+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 >> 172.21.15.57:0/3483279475 conn(0x7f9284101e80 msgr2=0x7f9284103530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.205+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 shutdown_connections 2023-10-06T22:05:28.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:28.205+0000 7f92894c0700 1 -- 172.21.15.57:0/3483279475 wait complete. 2023-10-06T22:05:28.645 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling cephadm module... 2023-10-06T22:05:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:28 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3483279475' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-06T22:05:29.775 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:29 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2940355851' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-06T22:05:29.776 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:29 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: ignoring --setuser ceph since I am not root 2023-10-06T22:05:29.776 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:29 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: ignoring --setgroup ceph since I am not root 2023-10-06T22:05:29.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.170+0000 7fbffda3b700 1 Processor -- start 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.170+0000 7fbffda3b700 1 -- start start 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.170+0000 7fbffda3b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff807a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.170+0000 7fbffda3b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff807a760 con 0x7fbff807bd30 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.171+0000 7fbff6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff807a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.171+0000 7fbff6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff807a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41498/0 (socket says 172.21.15.57:41498) 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.171+0000 7fbff6ffd700 1 -- 172.21.15.57:0/3415313731 learned_addr learned my addr 172.21.15.57:0/3415313731 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.171+0000 7fbff6ffd700 1 -- 172.21.15.57:0/3415313731 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff807a8a0 con 0x7fbff807bd30 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.172+0000 7fbff6ffd700 1 --2- 172.21.15.57:0/3415313731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff807a190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80098a0 tx=0x7fbfe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63575f296cf52afe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.172+0000 7fbff5ffb700 1 -- 172.21.15.57:0/3415313731 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8004160 con 0x7fbff807bd30 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.172+0000 7fbff5ffb700 1 -- 172.21.15.57:0/3415313731 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfe80042c0 con 0x7fbff807bd30 2023-10-06T22:05:29.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.172+0000 7fbff5ffb700 1 -- 172.21.15.57:0/3415313731 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8011a00 con 0x7fbff807bd30 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.172+0000 7fbffda3b700 1 -- 172.21.15.57:0/3415313731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 msgr2=0x7fbff807a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.172+0000 7fbffda3b700 1 --2- 172.21.15.57:0/3415313731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff807a190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80098a0 tx=0x7fbfe8009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.173+0000 7fbffda3b700 1 -- 172.21.15.57:0/3415313731 shutdown_connections 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.173+0000 7fbffda3b700 1 --2- 172.21.15.57:0/3415313731 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff807a190 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.173+0000 7fbffda3b700 1 -- 172.21.15.57:0/3415313731 >> 172.21.15.57:0/3415313731 conn(0x7fbff8101e50 msgr2=0x7fbff8104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.173+0000 7fbffda3b700 1 -- 172.21.15.57:0/3415313731 shutdown_connections 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.173+0000 7fbffda3b700 1 -- 172.21.15.57:0/3415313731 wait complete. 2023-10-06T22:05:29.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.174+0000 7fbffda3b700 1 Processor -- start 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.174+0000 7fbffda3b700 1 -- start start 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.174+0000 7fbffda3b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.174+0000 7fbffda3b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff819cd00 con 0x7fbff807bd30 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.174+0000 7fbff6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.174+0000 7fbff6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41506/0 (socket says 172.21.15.57:41506) 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbff6ffd700 1 -- 172.21.15.57:0/2940355851 learned_addr learned my addr 172.21.15.57:0/2940355851 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbff6ffd700 1 -- 172.21.15.57:0/2940355851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8009550 con 0x7fbff807bd30 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbff6ffd700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff819c7c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009130 tx=0x7fbfe8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8003ab0 con 0x7fbff807bd30 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfe8003c10 con 0x7fbff807bd30 2023-10-06T22:05:29.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff819cf00 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe80247d0 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.175+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff819d320 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.176+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7fbfe801abb0 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.177+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff80624e0 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.177+0000 7fbffca39700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fbfe0038820 0x7fbfe003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.177+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbfe804c530 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.177+0000 7fbff67fc700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fbfe0038820 0x7fbfe003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.178+0000 7fbff67fc700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fbfe0038820 0x7fbfe003ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006fd0 tx=0x7fbfec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.182+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbfe8018710 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.452+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fbff810ccb0 con 0x7fbff807bd30 2023-10-06T22:05:29.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.504+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fbfe8018350 con 0x7fbff807bd30 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.506+0000 7fbffca39700 1 -- 172.21.15.57:0/2940355851 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fbfe802d770 con 0x7fbff807bd30 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fbfe0038820 msgr2=0x7fbfe003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fbfe0038820 0x7fbfe003ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006fd0 tx=0x7fbfec006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 msgr2=0x7fbff819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff819c7c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009130 tx=0x7fbfe8011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 shutdown_connections 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7fbfe0038820 0x7fbfe003ace0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 --2- 172.21.15.57:0/2940355851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff807bd30 0x7fbff819c7c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 >> 172.21.15.57:0/2940355851 conn(0x7fbff8101e50 msgr2=0x7fbff8103550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 shutdown_connections 2023-10-06T22:05:29.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:29.508+0000 7fbffda3b700 1 -- 172.21.15.57:0/2940355851 wait complete. 2023-10-06T22:05:30.133 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:29 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:29.773+0000 7f6820b9f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:05:30.134 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:29 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:29.881+0000 7f6820b9f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:05:30.134 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:30 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:30.007+0000 7f6820b9f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:05:30.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:30 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:30.131+0000 7f6820b9f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:05:30.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:30 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:30.377+0000 7f6820b9f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:05:30.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:30 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2940355851' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-06T22:05:30.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:30 smithi057 ceph-mon[98492]: mgrmap e5: x(active, since 7s) 2023-10-06T22:05:31.043 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:30 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:30.867+0000 7f6820b9f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:05:31.207 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-06T22:05:31.207 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.471+0000 7f8491033700 1 Processor -- start 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.471+0000 7f8491033700 1 -- start start 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.472+0000 7f8491033700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c07a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.472+0000 7f8491033700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c07a740 con 0x7f848c07bd10 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.472+0000 7f848ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c07a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.472+0000 7f848ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c07a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41522/0 (socket says 172.21.15.57:41522) 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.472+0000 7f848ad9d700 1 -- 172.21.15.57:0/138529903 learned_addr learned my addr 172.21.15.57:0/138529903 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:31.208 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.472+0000 7f848ad9d700 1 -- 172.21.15.57:0/138529903 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c07a880 con 0x7f848c07bd10 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.473+0000 7f848ad9d700 1 --2- 172.21.15.57:0/138529903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c07a170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f847c0098a0 tx=0x7f847c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaf6a7a3a333d00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.473+0000 7f8489d9b700 1 -- 172.21.15.57:0/138529903 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f847c004160 con 0x7f848c07bd10 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.473+0000 7f8489d9b700 1 -- 172.21.15.57:0/138529903 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f847c0042c0 con 0x7f848c07bd10 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.473+0000 7f8489d9b700 1 -- 172.21.15.57:0/138529903 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f847c011a00 con 0x7f848c07bd10 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.474+0000 7f8491033700 1 -- 172.21.15.57:0/138529903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 msgr2=0x7f848c07a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.474+0000 7f8491033700 1 --2- 172.21.15.57:0/138529903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c07a170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f847c0098a0 tx=0x7f847c009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.474+0000 7f8491033700 1 -- 172.21.15.57:0/138529903 shutdown_connections 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.474+0000 7f8491033700 1 --2- 172.21.15.57:0/138529903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c07a170 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.474+0000 7f8491033700 1 -- 172.21.15.57:0/138529903 >> 172.21.15.57:0/138529903 conn(0x7f848c101e80 msgr2=0x7f848c1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.474+0000 7f8491033700 1 -- 172.21.15.57:0/138529903 shutdown_connections 2023-10-06T22:05:31.209 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.474+0000 7f8491033700 1 -- 172.21.15.57:0/138529903 wait complete. 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.475+0000 7f8491033700 1 Processor -- start 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.475+0000 7f8491033700 1 -- start start 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.475+0000 7f8491033700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.475+0000 7f8491033700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c19ccb0 con 0x7f848c07bd10 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.476+0000 7f848ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.476+0000 7f848ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c19c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41536/0 (socket says 172.21.15.57:41536) 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.476+0000 7f848ad9d700 1 -- 172.21.15.57:0/2506312093 learned_addr learned my addr 172.21.15.57:0/2506312093 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.476+0000 7f848ad9d700 1 -- 172.21.15.57:0/2506312093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009550 con 0x7f848c07bd10 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.476+0000 7f848ad9d700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c19c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f847c009130 tx=0x7f847c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.476+0000 7f8473fff700 1 -- 172.21.15.57:0/2506312093 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f847c003b70 con 0x7f848c07bd10 2023-10-06T22:05:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.477+0000 7f8473fff700 1 -- 172.21.15.57:0/2506312093 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f847c003cd0 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.477+0000 7f8473fff700 1 -- 172.21.15.57:0/2506312093 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f847c0197d0 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.477+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c19ceb0 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.477+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c19d2d0 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.478+0000 7f8473fff700 1 -- 172.21.15.57:0/2506312093 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f847c01abb0 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.478+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c0624e0 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.478+0000 7f8473fff700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 0x7f847403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.479+0000 7f8473fff700 1 -- 172.21.15.57:0/2506312093 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f847c04ca40 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.479+0000 7f848a59c700 1 -- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 msgr2=0x7f847403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.479+0000 7f848a59c700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 0x7f847403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.483+0000 7f8473fff700 1 -- 172.21.15.57:0/2506312093 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f847c01a430 con 0x7f848c07bd10 2023-10-06T22:05:31.211 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.680+0000 7f848a59c700 1 -- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 msgr2=0x7f847403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.680+0000 7f848a59c700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 0x7f847403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.754+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f848c10cc70 con 0x7f848c07bd10 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.754+0000 7f8473fff700 1 -- 172.21.15.57:0/2506312093 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f847c02db00 con 0x7f848c07bd10 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.756+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 msgr2=0x7f847403ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.756+0000 7f8491033700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 0x7f847403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.756+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 msgr2=0x7f848c19c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.756+0000 7f8491033700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c19c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f847c009130 tx=0x7f847c011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.757+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 shutdown_connections 2023-10-06T22:05:31.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.757+0000 7f8491033700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f8474038870 0x7f847403ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:31.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.757+0000 7f8491033700 1 --2- 172.21.15.57:0/2506312093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f848c07bd10 0x7f848c19c770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:31.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.757+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 >> 172.21.15.57:0/2506312093 conn(0x7f848c101e80 msgr2=0x7f848c103530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:31.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.757+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 shutdown_connections 2023-10-06T22:05:31.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:30.757+0000 7f8491033700 1 -- 172.21.15.57:0/2506312093 wait complete. 2023-10-06T22:05:31.213 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2023-10-06T22:05:31.213 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 5... 2023-10-06T22:05:31.301 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:31 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:31.041+0000 7f6820b9f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:05:31.301 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:31 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:31.163+0000 7f6820b9f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:05:31.558 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:31 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2506312093' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-06T22:05:31.558 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:31 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:31.299+0000 7f6820b9f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:05:32.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:31 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:31.871+0000 7f6820b9f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:05:33.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:32 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:32.744+0000 7f6820b9f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:05:33.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:32 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:32.852+0000 7f6820b9f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:05:34.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:33 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:33.842+0000 7f6820b9f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:05:34.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:33 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:33.965+0000 7f6820b9f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:05:34.575 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:34 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:34.198+0000 7f6820b9f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:05:34.856 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:34 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:34.573+0000 7f6820b9f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:05:35.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:34 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:34.854+0000 7f6820b9f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:05:35.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:35 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:35.053+0000 7f6820b9f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:05:35.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:35 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:35.409+0000 7f6820b9f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:05:36.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:36.369+0000 7f6820b9f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:05:36.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:36.548+0000 7f6820b9f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:05:37.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:36.805+0000 7f6820b9f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:05:37.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:36.955+0000 7f6820b9f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:05:37.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:37.243+0000 7f6820b9f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:05:38.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:38.354+0000 7f6820b9f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:05:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: Active manager daemon x restarted 2023-10-06T22:05:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: Activating manager daemon x 2023-10-06T22:05:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: osdmap e2: 0 total, 0 up, 0 in 2023-10-06T22:05:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: mgrmap e6: x(active, starting, since 0.00863262s) 2023-10-06T22:05:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: Manager daemon x is now available 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:38 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:05:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:39 smithi057 ceph-mon[98492]: Found migration_current of "None". Setting to last migration. 2023-10-06T22:05:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:39 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:05:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:39 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:05:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:39 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T22:05:39.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:39 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T22:05:39.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:39 smithi057 ceph-mon[98492]: mgrmap e7: x(active, since 1.01145s) 2023-10-06T22:05:39.909 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-06T22:05:39.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-06T22:05:39.910 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-06T22:05:39.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-06T22:05:39.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.795+0000 7f61d7b53700 1 Processor -- start 2023-10-06T22:05:39.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.796+0000 7f61d7b53700 1 -- start start 2023-10-06T22:05:39.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.796+0000 7f61d7b53700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d007a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:39.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.796+0000 7f61d7b53700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d007a720 con 0x7f61d007bcf0 2023-10-06T22:05:39.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.796+0000 7f61d58ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d007a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:39.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.796+0000 7f61d58ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d007a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41550/0 (socket says 172.21.15.57:41550) 2023-10-06T22:05:39.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.796+0000 7f61d58ef700 1 -- 172.21.15.57:0/242460732 learned_addr learned my addr 172.21.15.57:0/242460732 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:39.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.797+0000 7f61d58ef700 1 -- 172.21.15.57:0/242460732 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d007a860 con 0x7f61d007bcf0 2023-10-06T22:05:39.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.797+0000 7f61d58ef700 1 --2- 172.21.15.57:0/242460732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d007a150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0098a0 tx=0x7f61cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8fd74328bcd572 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:39.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.797+0000 7f61d48ed700 1 -- 172.21.15.57:0/242460732 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61cc004160 con 0x7f61d007bcf0 2023-10-06T22:05:39.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.797+0000 7f61d48ed700 1 -- 172.21.15.57:0/242460732 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61cc0042c0 con 0x7f61d007bcf0 2023-10-06T22:05:39.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.797+0000 7f61d48ed700 1 -- 172.21.15.57:0/242460732 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61cc011a00 con 0x7f61d007bcf0 2023-10-06T22:05:39.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.798+0000 7f61d7b53700 1 -- 172.21.15.57:0/242460732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 msgr2=0x7f61d007a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:39.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.798+0000 7f61d7b53700 1 --2- 172.21.15.57:0/242460732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d007a150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0098a0 tx=0x7f61cc009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:39.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.798+0000 7f61d7b53700 1 -- 172.21.15.57:0/242460732 shutdown_connections 2023-10-06T22:05:39.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.798+0000 7f61d7b53700 1 --2- 172.21.15.57:0/242460732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d007a150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:39.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.798+0000 7f61d7b53700 1 -- 172.21.15.57:0/242460732 >> 172.21.15.57:0/242460732 conn(0x7f61d0101e80 msgr2=0x7f61d01042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:39.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.798+0000 7f61d7b53700 1 -- 172.21.15.57:0/242460732 shutdown_connections 2023-10-06T22:05:39.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.798+0000 7f61d7b53700 1 -- 172.21.15.57:0/242460732 wait complete. 2023-10-06T22:05:39.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.799+0000 7f61d7b53700 1 Processor -- start 2023-10-06T22:05:39.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.799+0000 7f61d7b53700 1 -- start start 2023-10-06T22:05:39.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.800+0000 7f61d7b53700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:39.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.800+0000 7f61d7b53700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d019ccb0 con 0x7f61d007bcf0 2023-10-06T22:05:39.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.800+0000 7f61d58ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:39.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.800+0000 7f61d58ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41554/0 (socket says 172.21.15.57:41554) 2023-10-06T22:05:39.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.800+0000 7f61d58ef700 1 -- 172.21.15.57:0/3794385180 learned_addr learned my addr 172.21.15.57:0/3794385180 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:39.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.800+0000 7f61d58ef700 1 -- 172.21.15.57:0/3794385180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc009550 con 0x7f61d007bcf0 2023-10-06T22:05:39.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.801+0000 7f61d58ef700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d019c770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009fb0 tx=0x7f61cc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:39.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.801+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61cc003a40 con 0x7f61d007bcf0 2023-10-06T22:05:39.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.801+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61cc003ba0 con 0x7f61d007bcf0 2023-10-06T22:05:39.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.801+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d019ceb0 con 0x7f61d007bcf0 2023-10-06T22:05:39.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.801+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61cc01a750 con 0x7f61d007bcf0 2023-10-06T22:05:39.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.801+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d019d2d0 con 0x7f61d007bcf0 2023-10-06T22:05:39.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.802+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f61cc024430 con 0x7f61d007bcf0 2023-10-06T22:05:39.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.802+0000 7f61c6ffd700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:39.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.802+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 --> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61bc03b3a0 con 0x7f61bc0387d0 2023-10-06T22:05:39.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.803+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f61cc04c540 con 0x7f61d007bcf0 2023-10-06T22:05:39.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.803+0000 7f61d50ee700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 msgr2=0x7f61bc03ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:39.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:31.803+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:05:39.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:32.004+0000 7f61d50ee700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 msgr2=0x7f61bc03ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:39.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:32.004+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T22:05:39.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:32.405+0000 7f61d50ee700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 msgr2=0x7f61bc03ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:39.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:32.405+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T22:05:39.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:33.206+0000 7f61d50ee700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 msgr2=0x7f61bc03ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:39.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:33.206+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T22:05:39.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:34.807+0000 7f61d50ee700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 msgr2=0x7f61bc03ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:39.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:34.807+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T22:05:39.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:38.011+0000 7f61d50ee700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 msgr2=0x7f61bc03ac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1962312800 2023-10-06T22:05:39.919 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:38.011+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T22:05:39.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:38.364+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 6) v1 ==== 45346+0+0 (secure 0 0 0) 0x7f61cc024b60 con 0x7f61d007bcf0 2023-10-06T22:05:39.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:38.364+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 msgr2=0x7f61bc03ac90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:05:39.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:38.364+0000 7f61c6ffd700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:39.920 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.365+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 7) v1 ==== 45473+0+0 (secure 0 0 0) 0x7f61cc01cd60 con 0x7f61d007bcf0 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.366+0000 7f61c6ffd700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f61bc03bff0 0x7f61bc03e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.366+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61bc03b3a0 con 0x7f61bc03bff0 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.366+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f61bc03bff0 0x7f61bc03e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.367+0000 7f61d50ee700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f61bc03bff0 0x7f61bc03e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f61c0003a10 tx=0x7f61c0009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.368+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f61bc03b3a0 con 0x7f61bc03bff0 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.375+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f61d019dab0 con 0x7f61bc03bff0 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.375+0000 7f61c6ffd700 1 -- 172.21.15.57:0/3794385180 <== mgr.14118 v2:172.21.15.57:6800/4144294466 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f61d019dab0 con 0x7f61bc03bff0 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f61bc03bff0 msgr2=0x7f61bc03e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f61bc03bff0 0x7f61bc03e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f61c0003a10 tx=0x7f61c0009290 comp rx=0 tx=0).stop 2023-10-06T22:05:39.921 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 msgr2=0x7f61d019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d019c770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009fb0 tx=0x7f61cc011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 shutdown_connections 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f61bc03bff0 0x7f61bc03e3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d007bcf0 0x7f61d019c770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 --2- 172.21.15.57:0/3794385180 >> [v2:172.21.15.57:6800/1962312800,v1:172.21.15.57:6801/1962312800] conn(0x7f61bc0387d0 0x7f61bc03ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 >> 172.21.15.57:0/3794385180 conn(0x7f61d0101e80 msgr2=0x7f61d0103530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 shutdown_connections 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:39.376+0000 7f61d7b53700 1 -- 172.21.15.57:0/3794385180 wait complete. 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 5 is available 2023-10-06T22:05:39.922 INFO:teuthology.orchestra.run.smithi057.stdout:Setting orchestrator backend to cephadm... 2023-10-06T22:05:40.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:40 smithi057 ceph-mon[98492]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-06T22:05:40.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:40 smithi057 ceph-mon[98492]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-06T22:05:40.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:40 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:40.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:40 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:41.192 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.448+0000 7f74111f8700 1 Processor -- start 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.448+0000 7f74111f8700 1 -- start start 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.448+0000 7f74111f8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c07a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.448+0000 7f74111f8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c07a760 con 0x7f740c07bd30 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.449+0000 7f740bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c07a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.449+0000 7f740bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c07a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34160/0 (socket says 172.21.15.57:34160) 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.449+0000 7f740bfff700 1 -- 172.21.15.57:0/1538687495 learned_addr learned my addr 172.21.15.57:0/1538687495 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.449+0000 7f740bfff700 1 -- 172.21.15.57:0/1538687495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c07a8a0 con 0x7f740c07bd30 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.449+0000 7f740bfff700 1 --2- 172.21.15.57:0/1538687495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c07a190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0098a0 tx=0x7f73fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d35d8493f3fea6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:41.193 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f740affd700 1 -- 172.21.15.57:0/1538687495 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73fc004160 con 0x7f740c07bd30 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f740affd700 1 -- 172.21.15.57:0/1538687495 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f73fc0042c0 con 0x7f740c07bd30 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f740affd700 1 -- 172.21.15.57:0/1538687495 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73fc011a00 con 0x7f740c07bd30 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f74111f8700 1 -- 172.21.15.57:0/1538687495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 msgr2=0x7f740c07a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f74111f8700 1 --2- 172.21.15.57:0/1538687495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c07a190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0098a0 tx=0x7f73fc009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f74111f8700 1 -- 172.21.15.57:0/1538687495 shutdown_connections 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f74111f8700 1 --2- 172.21.15.57:0/1538687495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c07a190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f74111f8700 1 -- 172.21.15.57:0/1538687495 >> 172.21.15.57:0/1538687495 conn(0x7f740c101c20 msgr2=0x7f740c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.450+0000 7f74111f8700 1 -- 172.21.15.57:0/1538687495 shutdown_connections 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.451+0000 7f74111f8700 1 -- 172.21.15.57:0/1538687495 wait complete. 2023-10-06T22:05:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.451+0000 7f74111f8700 1 Processor -- start 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.452+0000 7f74111f8700 1 -- start start 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.452+0000 7f74111f8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.452+0000 7f74111f8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19cad0 con 0x7f740c07bd30 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.452+0000 7f740bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.452+0000 7f740bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34172/0 (socket says 172.21.15.57:34172) 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.452+0000 7f740bfff700 1 -- 172.21.15.57:0/1323914389 learned_addr learned my addr 172.21.15.57:0/1323914389 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.453+0000 7f740bfff700 1 -- 172.21.15.57:0/1323914389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc009550 con 0x7f740c07bd30 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.453+0000 7f740bfff700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c19c590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73fc006af0 tx=0x7f73fc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.453+0000 7f74097fa700 1 -- 172.21.15.57:0/1323914389 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73fc003a40 con 0x7f740c07bd30 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.453+0000 7f74097fa700 1 -- 172.21.15.57:0/1323914389 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f73fc003ba0 con 0x7f740c07bd30 2023-10-06T22:05:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.453+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c19ccd0 con 0x7f740c07bd30 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.453+0000 7f74097fa700 1 -- 172.21.15.57:0/1323914389 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73fc01a700 con 0x7f740c07bd30 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.453+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c19d0f0 con 0x7f740c07bd30 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.454+0000 7f74097fa700 1 -- 172.21.15.57:0/1323914389 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f73fc024ad0 con 0x7f740c07bd30 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.454+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c0624e0 con 0x7f740c07bd30 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.455+0000 7f74097fa700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f73f4038820 0x7f73f403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.455+0000 7f74097fa700 1 -- 172.21.15.57:0/1323914389 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f73fc04c930 con 0x7f740c07bd30 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.455+0000 7f740b7fe700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f73f4038820 0x7f73f403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.456+0000 7f740b7fe700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f73f4038820 0x7f73f403ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7400006fd0 tx=0x7f7400006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.461+0000 7f74097fa700 1 -- 172.21.15.57:0/1323914389 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f73fc018b80 con 0x7f740c07bd30 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.685+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f740c103fa0 con 0x7f73f4038820 2023-10-06T22:05:41.196 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.695+0000 7f74097fa700 1 -- 172.21.15.57:0/1323914389 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f740c103fa0 con 0x7f73f4038820 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.698+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f73f4038820 msgr2=0x7f73f403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.698+0000 7f74111f8700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f73f4038820 0x7f73f403ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7400006fd0 tx=0x7f7400006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.698+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 msgr2=0x7f740c19c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.698+0000 7f74111f8700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c19c590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73fc006af0 tx=0x7f73fc011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.699+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 shutdown_connections 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.699+0000 7f74111f8700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f73f4038820 0x7f73f403ace0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.699+0000 7f74111f8700 1 --2- 172.21.15.57:0/1323914389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f740c07bd30 0x7f740c19c590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.699+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 >> 172.21.15.57:0/1323914389 conn(0x7f740c101c20 msgr2=0x7f740c103380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.699+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 shutdown_connections 2023-10-06T22:05:41.197 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:40.699+0000 7f74111f8700 1 -- 172.21.15.57:0/1323914389 wait complete. 2023-10-06T22:05:41.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:41 smithi057 ceph-mon[98492]: mgrmap e8: x(active, since 2s) 2023-10-06T22:05:41.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:41 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:41.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:41 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:05:41.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:41 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:05:42.377 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.719+0000 7fa537be1700 1 Processor -- start 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.719+0000 7fa537be1700 1 -- start start 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.719+0000 7fa537be1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53007a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.719+0000 7fa537be1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53007a740 con 0x7fa53007bd10 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.719+0000 7fa53597d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53007a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.719+0000 7fa53597d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53007a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34176/0 (socket says 172.21.15.57:34176) 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.719+0000 7fa53597d700 1 -- 172.21.15.57:0/949713178 learned_addr learned my addr 172.21.15.57:0/949713178 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.720+0000 7fa53597d700 1 -- 172.21.15.57:0/949713178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53007a880 con 0x7fa53007bd10 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.720+0000 7fa53597d700 1 --2- 172.21.15.57:0/949713178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53007a170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0098a0 tx=0x7fa52c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61ef126ba5c1666 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.720+0000 7fa53497b700 1 -- 172.21.15.57:0/949713178 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa52c004160 con 0x7fa53007bd10 2023-10-06T22:05:42.378 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.720+0000 7fa53497b700 1 -- 172.21.15.57:0/949713178 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa52c0042c0 con 0x7fa53007bd10 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.720+0000 7fa53497b700 1 -- 172.21.15.57:0/949713178 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa52c011a00 con 0x7fa53007bd10 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.721+0000 7fa537be1700 1 -- 172.21.15.57:0/949713178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 msgr2=0x7fa53007a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.721+0000 7fa537be1700 1 --2- 172.21.15.57:0/949713178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53007a170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0098a0 tx=0x7fa52c009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.721+0000 7fa537be1700 1 -- 172.21.15.57:0/949713178 shutdown_connections 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.721+0000 7fa537be1700 1 --2- 172.21.15.57:0/949713178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53007a170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.721+0000 7fa537be1700 1 -- 172.21.15.57:0/949713178 >> 172.21.15.57:0/949713178 conn(0x7fa530101e80 msgr2=0x7fa5301042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.721+0000 7fa537be1700 1 -- 172.21.15.57:0/949713178 shutdown_connections 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.721+0000 7fa537be1700 1 -- 172.21.15.57:0/949713178 wait complete. 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.722+0000 7fa537be1700 1 Processor -- start 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.722+0000 7fa537be1700 1 -- start start 2023-10-06T22:05:42.379 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.723+0000 7fa537be1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.723+0000 7fa537be1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53019ccb0 con 0x7fa53007bd10 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.723+0000 7fa53597d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.723+0000 7fa53597d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34182/0 (socket says 172.21.15.57:34182) 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.723+0000 7fa53597d700 1 -- 172.21.15.57:0/2265833723 learned_addr learned my addr 172.21.15.57:0/2265833723 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.723+0000 7fa53597d700 1 -- 172.21.15.57:0/2265833723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c009550 con 0x7fa53007bd10 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.723+0000 7fa53597d700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53019c770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009130 tx=0x7fa52c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.724+0000 7fa526ffd700 1 -- 172.21.15.57:0/2265833723 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa52c003b70 con 0x7fa53007bd10 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.724+0000 7fa526ffd700 1 -- 172.21.15.57:0/2265833723 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa52c003cd0 con 0x7fa53007bd10 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.724+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53019ceb0 con 0x7fa53007bd10 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.724+0000 7fa526ffd700 1 -- 172.21.15.57:0/2265833723 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa52c0197d0 con 0x7fa53007bd10 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.724+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53019d2d0 con 0x7fa53007bd10 2023-10-06T22:05:42.380 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.725+0000 7fa526ffd700 1 -- 172.21.15.57:0/2265833723 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fa52c01a430 con 0x7fa53007bd10 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.725+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5300624e0 con 0x7fa53007bd10 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.725+0000 7fa526ffd700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fa51c0388c0 0x7fa51c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.725+0000 7fa526ffd700 1 -- 172.21.15.57:0/2265833723 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa52c04caf0 con 0x7fa53007bd10 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.726+0000 7fa53517c700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fa51c0388c0 0x7fa51c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.726+0000 7fa53517c700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fa51c0388c0 0x7fa51c03ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa520006fd0 tx=0x7fa520006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.731+0000 7fa526ffd700 1 -- 172.21.15.57:0/2265833723 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa52c02d9a0 con 0x7fa53007bd10 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.939+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa530104190 con 0x7fa51c0388c0 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.940+0000 7fa526ffd700 1 -- 172.21.15.57:0/2265833723 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa530104190 con 0x7fa51c0388c0 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fa51c0388c0 msgr2=0x7fa51c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fa51c0388c0 0x7fa51c03ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa520006fd0 tx=0x7fa520006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 msgr2=0x7fa53019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:42.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53019c770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009130 tx=0x7fa52c011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:42.382 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 shutdown_connections 2023-10-06T22:05:42.382 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fa51c0388c0 0x7fa51c03ad80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:42.382 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 --2- 172.21.15.57:0/2265833723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53007bd10 0x7fa53019c770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:42.382 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 >> 172.21.15.57:0/2265833723 conn(0x7fa530101e80 msgr2=0x7fa530103530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:42.382 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 shutdown_connections 2023-10-06T22:05:42.382 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:41.942+0000 7fa537be1700 1 -- 172.21.15.57:0/2265833723 wait complete. 2023-10-06T22:05:42.382 INFO:teuthology.orchestra.run.smithi057.stdout:Generating ssh key... 2023-10-06T22:05:42.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:42 smithi057 ceph-mon[98492]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:05:42.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:42 smithi057 ceph-mon[98492]: [06/Oct/2023:22:05:40] ENGINE Bus STARTING 2023-10-06T22:05:42.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:42 smithi057 ceph-mon[98492]: [06/Oct/2023:22:05:40] ENGINE Serving on https://172.21.15.57:7150 2023-10-06T22:05:42.617 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:42 smithi057 ceph-mon[98492]: [06/Oct/2023:22:05:41] ENGINE Serving on http://172.21.15.57:8765 2023-10-06T22:05:42.617 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:42 smithi057 ceph-mon[98492]: [06/Oct/2023:22:05:41] ENGINE Bus STARTED 2023-10-06T22:05:43.876 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-mon[98492]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: Generating public/private rsa key pair. 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: Your identification has been saved in /tmp/tmpvuux87hq/key. 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: Your public key has been saved in /tmp/tmpvuux87hq/key.pub. 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: The key fingerprint is: 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: SHA256:js9PjcsyTNH2kcK6OEokhdLs43MKuMEfz+t74CUVy7g ceph-495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: The key's randomart image is: 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: +---[RSA 3072]----+ 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: | | 2023-10-06T22:05:43.877 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: | o . . | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: |. + . o oo . | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: | o . . +. = o | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: | + . o S+ o . | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: |o. + E +o o. | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: |+.o.+ =+..o . | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: | +.=+.o+=o . | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: |. ..oB+.o++ | 2023-10-06T22:05:43.878 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: +----[SHA256]-----+ 2023-10-06T22:05:43.958 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.875+0000 7f49f952a700 1 Processor -- start 2023-10-06T22:05:43.959 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.875+0000 7f49f952a700 1 -- start start 2023-10-06T22:05:43.959 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.875+0000 7f49f952a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f407a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:43.959 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.875+0000 7f49f952a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f407a760 con 0x7f49f407bd30 2023-10-06T22:05:43.959 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.876+0000 7f49f2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f407a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:43.959 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.876+0000 7f49f2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f407a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57058/0 (socket says 172.21.15.57:57058) 2023-10-06T22:05:43.960 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.876+0000 7f49f2ffd700 1 -- 172.21.15.57:0/4276133275 learned_addr learned my addr 172.21.15.57:0/4276133275 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:43.960 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.876+0000 7f49f2ffd700 1 -- 172.21.15.57:0/4276133275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f407a8a0 con 0x7f49f407bd30 2023-10-06T22:05:43.960 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.876+0000 7f49f2ffd700 1 --2- 172.21.15.57:0/4276133275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f407a190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49e40098a0 tx=0x7f49e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26be26aa00c0d62e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:43.960 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f1ffb700 1 -- 172.21.15.57:0/4276133275 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e4004160 con 0x7f49f407bd30 2023-10-06T22:05:43.961 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f1ffb700 1 -- 172.21.15.57:0/4276133275 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49e40042c0 con 0x7f49f407bd30 2023-10-06T22:05:43.961 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f1ffb700 1 -- 172.21.15.57:0/4276133275 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e4011a00 con 0x7f49f407bd30 2023-10-06T22:05:43.961 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f952a700 1 -- 172.21.15.57:0/4276133275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 msgr2=0x7f49f407a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:43.961 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f952a700 1 --2- 172.21.15.57:0/4276133275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f407a190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49e40098a0 tx=0x7f49e4009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:43.961 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f952a700 1 -- 172.21.15.57:0/4276133275 shutdown_connections 2023-10-06T22:05:43.961 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f952a700 1 --2- 172.21.15.57:0/4276133275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f407a190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:43.962 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f952a700 1 -- 172.21.15.57:0/4276133275 >> 172.21.15.57:0/4276133275 conn(0x7f49f4101e80 msgr2=0x7f49f41042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:43.962 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.877+0000 7f49f952a700 1 -- 172.21.15.57:0/4276133275 shutdown_connections 2023-10-06T22:05:43.962 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.878+0000 7f49f952a700 1 -- 172.21.15.57:0/4276133275 wait complete. 2023-10-06T22:05:43.962 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.878+0000 7f49f952a700 1 Processor -- start 2023-10-06T22:05:43.962 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.879+0000 7f49f952a700 1 -- start start 2023-10-06T22:05:43.962 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.879+0000 7f49f952a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:43.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.879+0000 7f49f952a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f419cd30 con 0x7f49f407bd30 2023-10-06T22:05:43.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.879+0000 7f49f2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:43.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.879+0000 7f49f2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57068/0 (socket says 172.21.15.57:57068) 2023-10-06T22:05:43.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.879+0000 7f49f2ffd700 1 -- 172.21.15.57:0/4210002203 learned_addr learned my addr 172.21.15.57:0/4210002203 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:43.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.880+0000 7f49f2ffd700 1 -- 172.21.15.57:0/4210002203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4009550 con 0x7f49f407bd30 2023-10-06T22:05:43.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.880+0000 7f49f2ffd700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f419c7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009130 tx=0x7f49e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:43.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.880+0000 7f49dbfff700 1 -- 172.21.15.57:0/4210002203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e4003ab0 con 0x7f49f407bd30 2023-10-06T22:05:43.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.880+0000 7f49dbfff700 1 -- 172.21.15.57:0/4210002203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49e4003c10 con 0x7f49f407bd30 2023-10-06T22:05:43.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.880+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f419cf30 con 0x7f49f407bd30 2023-10-06T22:05:43.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.880+0000 7f49dbfff700 1 -- 172.21.15.57:0/4210002203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e40247d0 con 0x7f49f407bd30 2023-10-06T22:05:43.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.880+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f419d350 con 0x7f49f407bd30 2023-10-06T22:05:43.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.881+0000 7f49dbfff700 1 -- 172.21.15.57:0/4210002203 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f49e401a430 con 0x7f49f407bd30 2023-10-06T22:05:43.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.881+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f4196860 con 0x7f49f407bd30 2023-10-06T22:05:43.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.882+0000 7f49dbfff700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f49dc038870 0x7f49dc03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:43.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.882+0000 7f49dbfff700 1 -- 172.21.15.57:0/4210002203 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f49e404c9b0 con 0x7f49f407bd30 2023-10-06T22:05:43.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.882+0000 7f49f27fc700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f49dc038870 0x7f49dc03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:43.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.882+0000 7f49f27fc700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f49dc038870 0x7f49dc03ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49e8006fd0 tx=0x7f49e8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:43.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:42.887+0000 7f49dbfff700 1 -- 172.21.15.57:0/4210002203 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f49e4018b40 con 0x7f49f407bd30 2023-10-06T22:05:43.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.095+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f402d170 con 0x7f49dc038870 2023-10-06T22:05:43.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.455+0000 7f49dbfff700 1 -- 172.21.15.57:0/4210002203 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f49f402d170 con 0x7f49dc038870 2023-10-06T22:05:43.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f49dc038870 msgr2=0x7f49dc03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:43.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f49dc038870 0x7f49dc03ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49e8006fd0 tx=0x7f49e8006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:43.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 msgr2=0x7f49f419c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:43.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f419c7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009130 tx=0x7f49e4011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:43.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 shutdown_connections 2023-10-06T22:05:43.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f49dc038870 0x7f49dc03ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:43.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 --2- 172.21.15.57:0/4210002203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49f407bd30 0x7f49f419c7f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:43.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 >> 172.21.15.57:0/4210002203 conn(0x7f49f4101e80 msgr2=0x7f49f41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:43.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 shutdown_connections 2023-10-06T22:05:43.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:43.457+0000 7f49f952a700 1 -- 172.21.15.57:0/4210002203 wait complete. 2023-10-06T22:05:44.703 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:44 smithi057 ceph-mon[98492]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:05:44.703 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:44 smithi057 ceph-mon[98492]: Generating ssh key... 2023-10-06T22:05:44.703 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:44 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:44.703 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:44 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:45.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyI+7/QRB8oCobgUhQzSeETfpk8/490d/CUJyKdscgxAQR7GZ6xahRFrOPo80/LcwlLzVjOx0N0qQZ2EnfA4e4A6GI8aU1pEz6jae/SOBc+ocrroY5zKBAMZmuI8XaWI4PvBuBPGmlEGE8TAU4dBRi9i2anX4BRMU+jTPEgbpRQiP8/eDABCNq034Wfb0JPrbgC2J+DQw3aEGZrzimUB7yWxMJuUkyIBEFCICE7nq5NwJlSNNPrgaxShaAI4Uv3BaNp2ADTYEdf24Zt7dRM6TZmxwiz0vgql3TdrmGhCW+WLiPH02EjQoDz3R0VlIhecAn156JqOBcxDTeSIXXU4cHhf8KKSzUo6rho6BPRpSC4Pr/jcw6ljWzWqUHRG9uNqWvI0Y97rLi5wtFOY6pNwzPyE1dmisGs9MJgRXykGoqhvvfMAxSVLTKIyTctbqcrQJz9hF4KJHwnD4zI3s69MWKvmeJoD6XxJpX9E5pc8oY+fZ0A+JYKTij5PQJOKWITbU= ceph-495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:05:45.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.452+0000 7f0e89d16700 1 Processor -- start 2023-10-06T22:05:45.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.452+0000 7f0e89d16700 1 -- start start 2023-10-06T22:05:45.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.452+0000 7f0e89d16700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e84106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:45.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.452+0000 7f0e89d16700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84107270 con 0x7f0e84106880 2023-10-06T22:05:45.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.453+0000 7f0e837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e84106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:45.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.453+0000 7f0e837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e84106ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57074/0 (socket says 172.21.15.57:57074) 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.453+0000 7f0e837fe700 1 -- 172.21.15.57:0/4221697875 learned_addr learned my addr 172.21.15.57:0/4221697875 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.453+0000 7f0e837fe700 1 -- 172.21.15.57:0/4221697875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84107a80 con 0x7f0e84106880 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.453+0000 7f0e837fe700 1 --2- 172.21.15.57:0/4221697875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e84106ca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e740098a0 tx=0x7f0e74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc97fe884b75ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.453+0000 7f0e827fc700 1 -- 172.21.15.57:0/4221697875 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e74004160 con 0x7f0e84106880 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e827fc700 1 -- 172.21.15.57:0/4221697875 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e740042c0 con 0x7f0e84106880 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e827fc700 1 -- 172.21.15.57:0/4221697875 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e74011a00 con 0x7f0e84106880 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e89d16700 1 -- 172.21.15.57:0/4221697875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 msgr2=0x7f0e84106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e89d16700 1 --2- 172.21.15.57:0/4221697875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e84106ca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e740098a0 tx=0x7f0e74009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e89d16700 1 -- 172.21.15.57:0/4221697875 shutdown_connections 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e89d16700 1 --2- 172.21.15.57:0/4221697875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e84106ca0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:45.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e89d16700 1 -- 172.21.15.57:0/4221697875 >> 172.21.15.57:0/4221697875 conn(0x7f0e84101e80 msgr2=0x7f0e841042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e89d16700 1 -- 172.21.15.57:0/4221697875 shutdown_connections 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.454+0000 7f0e89d16700 1 -- 172.21.15.57:0/4221697875 wait complete. 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.455+0000 7f0e89d16700 1 Processor -- start 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.456+0000 7f0e89d16700 1 -- start start 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.456+0000 7f0e89d16700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e8407bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.456+0000 7f0e89d16700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8407c020 con 0x7f0e84106880 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.456+0000 7f0e837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e8407bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.456+0000 7f0e837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e8407bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57076/0 (socket says 172.21.15.57:57076) 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.456+0000 7f0e837fe700 1 -- 172.21.15.57:0/4256738326 learned_addr learned my addr 172.21.15.57:0/4256738326 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.456+0000 7f0e837fe700 1 -- 172.21.15.57:0/4256738326 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74009550 con 0x7f0e84106880 2023-10-06T22:05:45.141 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.457+0000 7f0e837fe700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e8407bae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009130 tx=0x7f0e74011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.457+0000 7f0e80ff9700 1 -- 172.21.15.57:0/4256738326 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e74003ab0 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.457+0000 7f0e80ff9700 1 -- 172.21.15.57:0/4256738326 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e74003c10 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.457+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8407a130 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.457+0000 7f0e80ff9700 1 -- 172.21.15.57:0/4256738326 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e740247d0 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.457+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8407a550 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.458+0000 7f0e80ff9700 1 -- 172.21.15.57:0/4256738326 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f0e7401a430 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.458+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e840624e0 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.459+0000 7f0e80ff9700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0e6c038870 0x7f0e6c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.459+0000 7f0e80ff9700 1 -- 172.21.15.57:0/4256738326 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0e7404c9d0 con 0x7f0e84106880 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.459+0000 7f0e82ffd700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0e6c038870 0x7f0e6c03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:45.142 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.459+0000 7f0e82ffd700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0e6c038870 0x7f0e6c03ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e78006fd0 tx=0x7f0e78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.464+0000 7f0e80ff9700 1 -- 172.21.15.57:0/4256738326 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0e74018b40 con 0x7f0e84106880 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.675+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e84194320 con 0x7f0e6c038870 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.676+0000 7f0e80ff9700 1 -- 172.21.15.57:0/4256738326 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f0e84194320 con 0x7f0e6c038870 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.678+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0e6c038870 msgr2=0x7f0e6c03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.678+0000 7f0e89d16700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0e6c038870 0x7f0e6c03ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e78006fd0 tx=0x7f0e78006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.678+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 msgr2=0x7f0e8407bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.678+0000 7f0e89d16700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e8407bae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009130 tx=0x7f0e74011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.679+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 shutdown_connections 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.679+0000 7f0e89d16700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0e6c038870 0x7f0e6c03ad30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.679+0000 7f0e89d16700 1 --2- 172.21.15.57:0/4256738326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84106880 0x7f0e8407bae0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:45.143 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.679+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 >> 172.21.15.57:0/4256738326 conn(0x7f0e84101e80 msgr2=0x7f0e84104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:45.144 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.679+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 shutdown_connections 2023-10-06T22:05:45.144 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:44.679+0000 7f0e89d16700 1 -- 172.21.15.57:0/4256738326 wait complete. 2023-10-06T22:05:45.144 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-06T22:05:45.144 INFO:teuthology.orchestra.run.smithi057.stdout:Adding key to root@localhost authorized_keys... 2023-10-06T22:05:45.144 INFO:teuthology.orchestra.run.smithi057.stdout:Adding host smithi057... 2023-10-06T22:05:46.574 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:46 smithi057 ceph-mon[98492]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:05:47.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:47 smithi057 ceph-mon[98492]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:05:48.729 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:48 smithi057 ceph-mon[98492]: Deploying cephadm binary to smithi057 2023-10-06T22:05:49.731 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Added host 'smithi057' with addr '172.21.15.57' 2023-10-06T22:05:49.731 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.655+0000 7f67a7495700 1 Processor -- start 2023-10-06T22:05:49.731 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.655+0000 7f67a7495700 1 -- start start 2023-10-06T22:05:49.731 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.655+0000 7f67a7495700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a0106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:49.731 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.656+0000 7f67a7495700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0107280 con 0x7f67a0106890 2023-10-06T22:05:49.731 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.656+0000 7f67a5231700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a0106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.656+0000 7f67a5231700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a0106cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57082/0 (socket says 172.21.15.57:57082) 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.656+0000 7f67a5231700 1 -- 172.21.15.57:0/966293451 learned_addr learned my addr 172.21.15.57:0/966293451 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.656+0000 7f67a5231700 1 -- 172.21.15.57:0/966293451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a0107a90 con 0x7f67a0106890 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f67a5231700 1 --2- 172.21.15.57:0/966293451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a0106cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f679c0098a0 tx=0x7f679c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba756865fa9aaa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f6797fff700 1 -- 172.21.15.57:0/966293451 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f679c004160 con 0x7f67a0106890 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f6797fff700 1 -- 172.21.15.57:0/966293451 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f679c0042c0 con 0x7f67a0106890 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f6797fff700 1 -- 172.21.15.57:0/966293451 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f679c011a00 con 0x7f67a0106890 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f67a7495700 1 -- 172.21.15.57:0/966293451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 msgr2=0x7f67a0106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f67a7495700 1 --2- 172.21.15.57:0/966293451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a0106cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f679c0098a0 tx=0x7f679c009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f67a7495700 1 -- 172.21.15.57:0/966293451 shutdown_connections 2023-10-06T22:05:49.732 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f67a7495700 1 --2- 172.21.15.57:0/966293451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a0106cb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.657+0000 7f67a7495700 1 -- 172.21.15.57:0/966293451 >> 172.21.15.57:0/966293451 conn(0x7f67a0101e50 msgr2=0x7f67a0104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.658+0000 7f67a7495700 1 -- 172.21.15.57:0/966293451 shutdown_connections 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.658+0000 7f67a7495700 1 -- 172.21.15.57:0/966293451 wait complete. 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.659+0000 7f67a7495700 1 Processor -- start 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.659+0000 7f67a7495700 1 -- start start 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.659+0000 7f67a7495700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.659+0000 7f67a7495700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a019cce0 con 0x7f67a0106890 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.659+0000 7f67a5231700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.659+0000 7f67a5231700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57090/0 (socket says 172.21.15.57:57090) 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.659+0000 7f67a5231700 1 -- 172.21.15.57:0/917475341 learned_addr learned my addr 172.21.15.57:0/917475341 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.660+0000 7f67a5231700 1 -- 172.21.15.57:0/917475341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009550 con 0x7f67a0106890 2023-10-06T22:05:49.733 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.660+0000 7f67a5231700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a019c7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f679c009130 tx=0x7f679c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:49.734 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.660+0000 7f67967fc700 1 -- 172.21.15.57:0/917475341 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f679c003ab0 con 0x7f67a0106890 2023-10-06T22:05:49.734 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.660+0000 7f67967fc700 1 -- 172.21.15.57:0/917475341 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f679c003c10 con 0x7f67a0106890 2023-10-06T22:05:49.734 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.660+0000 7f67967fc700 1 -- 172.21.15.57:0/917475341 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f679c0247d0 con 0x7f67a0106890 2023-10-06T22:05:49.734 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.660+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a019cee0 con 0x7f67a0106890 2023-10-06T22:05:49.734 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.660+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a019d300 con 0x7f67a0106890 2023-10-06T22:05:49.734 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.661+0000 7f67967fc700 1 -- 172.21.15.57:0/917475341 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f679c01abb0 con 0x7f67a0106890 2023-10-06T22:05:49.734 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.661+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a00624e0 con 0x7f67a0106890 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.662+0000 7f67967fc700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f678c038870 0x7f678c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.662+0000 7f67967fc700 1 -- 172.21.15.57:0/917475341 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f679c04c980 con 0x7f67a0106890 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.662+0000 7f67a4a30700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f678c038870 0x7f678c03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.663+0000 7f67a4a30700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f678c038870 0x7f678c03ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6790006fd0 tx=0x7f6790006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.667+0000 7f67967fc700 1 -- 172.21.15.57:0/917475341 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f679c018530 con 0x7f67a0106890 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:45.882+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}) v1 -- 0x7f67a0104160 con 0x7f678c038870 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.318+0000 7f67967fc700 1 -- 172.21.15.57:0/917475341 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f67a0104160 con 0x7f678c038870 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f678c038870 msgr2=0x7f678c03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:49.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f678c038870 0x7f678c03ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6790006fd0 tx=0x7f6790006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 msgr2=0x7f67a019c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a019c7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f679c009130 tx=0x7f679c011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 shutdown_connections 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f678c038870 0x7f678c03ad30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 --2- 172.21.15.57:0/917475341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a0106890 0x7f67a019c7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.321+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 >> 172.21.15.57:0/917475341 conn(0x7f67a0101e50 msgr2=0x7f67a0103500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.322+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 shutdown_connections 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:49.322+0000 7f67a7495700 1 -- 172.21.15.57:0/917475341 wait complete. 2023-10-06T22:05:49.736 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mon service... 2023-10-06T22:05:50.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:50 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:50.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:50 smithi057 ceph-mon[98492]: Added host smithi057 2023-10-06T22:05:50.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:50 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:05:51.016 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.290+0000 7fd5d600a700 1 Processor -- start 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.290+0000 7fd5d600a700 1 -- start start 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.291+0000 7fd5d600a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d0106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.291+0000 7fd5d600a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d0107280 con 0x7fd5d0106890 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.291+0000 7fd5cf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d0106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.291+0000 7fd5cf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d0106cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57096/0 (socket says 172.21.15.57:57096) 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.291+0000 7fd5cf7fe700 1 -- 172.21.15.57:0/1372965123 learned_addr learned my addr 172.21.15.57:0/1372965123 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.292+0000 7fd5cf7fe700 1 -- 172.21.15.57:0/1372965123 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d0107a90 con 0x7fd5d0106890 2023-10-06T22:05:51.017 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.292+0000 7fd5cf7fe700 1 --2- 172.21.15.57:0/1372965123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d0106cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00098a0 tx=0x7fd5c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fdb60a2f8063ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.292+0000 7fd5ce7fc700 1 -- 172.21.15.57:0/1372965123 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5c0004160 con 0x7fd5d0106890 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.292+0000 7fd5ce7fc700 1 -- 172.21.15.57:0/1372965123 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5c00042c0 con 0x7fd5d0106890 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.292+0000 7fd5ce7fc700 1 -- 172.21.15.57:0/1372965123 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5c0011a00 con 0x7fd5d0106890 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.293+0000 7fd5d600a700 1 -- 172.21.15.57:0/1372965123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 msgr2=0x7fd5d0106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.293+0000 7fd5d600a700 1 --2- 172.21.15.57:0/1372965123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d0106cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00098a0 tx=0x7fd5c0009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.293+0000 7fd5d600a700 1 -- 172.21.15.57:0/1372965123 shutdown_connections 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.293+0000 7fd5d600a700 1 --2- 172.21.15.57:0/1372965123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d0106cb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.293+0000 7fd5d600a700 1 -- 172.21.15.57:0/1372965123 >> 172.21.15.57:0/1372965123 conn(0x7fd5d0101e50 msgr2=0x7fd5d0104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.293+0000 7fd5d600a700 1 -- 172.21.15.57:0/1372965123 shutdown_connections 2023-10-06T22:05:51.018 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.293+0000 7fd5d600a700 1 -- 172.21.15.57:0/1372965123 wait complete. 2023-10-06T22:05:51.019 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.294+0000 7fd5d600a700 1 Processor -- start 2023-10-06T22:05:51.019 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.294+0000 7fd5d600a700 1 -- start start 2023-10-06T22:05:51.019 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.294+0000 7fd5d600a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:51.019 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.294+0000 7fd5d600a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d019cce0 con 0x7fd5d0106890 2023-10-06T22:05:51.019 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5cf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:51.019 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5cf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57104/0 (socket says 172.21.15.57:57104) 2023-10-06T22:05:51.019 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5cf7fe700 1 -- 172.21.15.57:0/3469791951 learned_addr learned my addr 172.21.15.57:0/3469791951 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5cf7fe700 1 -- 172.21.15.57:0/3469791951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0009550 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5cf7fe700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d019c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009130 tx=0x7fd5c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5ccff9700 1 -- 172.21.15.57:0/3469791951 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5c0003ab0 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5ccff9700 1 -- 172.21.15.57:0/3469791951 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5c0003c10 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d019cee0 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.295+0000 7fd5ccff9700 1 -- 172.21.15.57:0/3469791951 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5c00247d0 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.296+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d019d300 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.297+0000 7fd5ccff9700 1 -- 172.21.15.57:0/3469791951 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fd5c002d3f0 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.297+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d00624e0 con 0x7fd5d0106890 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.297+0000 7fd5ccff9700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fd5b8038870 0x7fd5b803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:51.020 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.297+0000 7fd5ccff9700 1 -- 172.21.15.57:0/3469791951 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd5c004cb70 con 0x7fd5d0106890 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.297+0000 7fd5ceffd700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fd5b8038870 0x7fd5b803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.298+0000 7fd5ceffd700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fd5b8038870 0x7fd5b803ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4006fd0 tx=0x7fd5c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.302+0000 7fd5ccff9700 1 -- 172.21.15.57:0/3469791951 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5c0018b00 con 0x7fd5d0106890 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.528+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd5d0104160 con 0x7fd5b8038870 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.535+0000 7fd5ccff9700 1 -- 172.21.15.57:0/3469791951 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd5d0104160 con 0x7fd5b8038870 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fd5b8038870 msgr2=0x7fd5b803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fd5b8038870 0x7fd5b803ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4006fd0 tx=0x7fd5c4006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 msgr2=0x7fd5d019c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d019c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009130 tx=0x7fd5c0011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 shutdown_connections 2023-10-06T22:05:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7fd5b8038870 0x7fd5b803ad30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:51.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 --2- 172.21.15.57:0/3469791951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d0106890 0x7fd5d019c7a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:51.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 >> 172.21.15.57:0/3469791951 conn(0x7fd5d0101e50 msgr2=0x7fd5d0103500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:51.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 shutdown_connections 2023-10-06T22:05:51.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:50.538+0000 7fd5d600a700 1 -- 172.21.15.57:0/3469791951 wait complete. 2023-10-06T22:05:51.022 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mgr service... 2023-10-06T22:05:51.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:51 smithi057 ceph-mon[98492]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:05:51.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:51 smithi057 ceph-mon[98492]: Saving service mon spec with placement count:5 2023-10-06T22:05:51.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:51 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:52.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-06T22:05:52.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.736+0000 7f3f686de700 1 Processor -- start 2023-10-06T22:05:52.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.737+0000 7f3f686de700 1 -- start start 2023-10-06T22:05:52.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.737+0000 7f3f686de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f60106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:52.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.737+0000 7f3f686de700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f60107280 con 0x7f3f60106890 2023-10-06T22:05:52.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.737+0000 7f3f6647a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f60106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.737+0000 7f3f6647a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f60106cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57120/0 (socket says 172.21.15.57:57120) 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.737+0000 7f3f6647a700 1 -- 172.21.15.57:0/1036290574 learned_addr learned my addr 172.21.15.57:0/1036290574 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.738+0000 7f3f6647a700 1 -- 172.21.15.57:0/1036290574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f60107a90 con 0x7f3f60106890 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.738+0000 7f3f6647a700 1 --2- 172.21.15.57:0/1036290574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f60106cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0098a0 tx=0x7f3f5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00dd6225895fd31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.738+0000 7f3f65478700 1 -- 172.21.15.57:0/1036290574 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c004160 con 0x7f3f60106890 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.738+0000 7f3f65478700 1 -- 172.21.15.57:0/1036290574 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f5c0042c0 con 0x7f3f60106890 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.738+0000 7f3f65478700 1 -- 172.21.15.57:0/1036290574 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c011a00 con 0x7f3f60106890 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.739+0000 7f3f686de700 1 -- 172.21.15.57:0/1036290574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 msgr2=0x7f3f60106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.739+0000 7f3f686de700 1 --2- 172.21.15.57:0/1036290574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f60106cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0098a0 tx=0x7f3f5c009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:52.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.739+0000 7f3f686de700 1 -- 172.21.15.57:0/1036290574 shutdown_connections 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.739+0000 7f3f686de700 1 --2- 172.21.15.57:0/1036290574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f60106cb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.739+0000 7f3f686de700 1 -- 172.21.15.57:0/1036290574 >> 172.21.15.57:0/1036290574 conn(0x7f3f60101e50 msgr2=0x7f3f60104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.739+0000 7f3f686de700 1 -- 172.21.15.57:0/1036290574 shutdown_connections 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.739+0000 7f3f686de700 1 -- 172.21.15.57:0/1036290574 wait complete. 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.740+0000 7f3f686de700 1 Processor -- start 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.740+0000 7f3f686de700 1 -- start start 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.741+0000 7f3f686de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f6007bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.741+0000 7f3f686de700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6007c020 con 0x7f3f60106890 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.741+0000 7f3f6647a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f6007bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:52.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.741+0000 7f3f6647a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f6007bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57122/0 (socket says 172.21.15.57:57122) 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.741+0000 7f3f6647a700 1 -- 172.21.15.57:0/2645864649 learned_addr learned my addr 172.21.15.57:0/2645864649 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.741+0000 7f3f6647a700 1 -- 172.21.15.57:0/2645864649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c009550 con 0x7f3f60106890 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.741+0000 7f3f6647a700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f6007bae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009130 tx=0x7f3f5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.742+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c003ab0 con 0x7f3f60106890 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.742+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6007a130 con 0x7f3f60106890 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.742+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f5c003c10 con 0x7f3f60106890 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.742+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c0247d0 con 0x7f3f60106890 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.742+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6007a550 con 0x7f3f60106890 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.743+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f3f5c01a430 con 0x7f3f60106890 2023-10-06T22:05:52.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.743+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f600624e0 con 0x7f3f60106890 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.743+0000 7f3f577fe700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3f4c038820 0x7f3f4c03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.743+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3f5c04ca10 con 0x7f3f60106890 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.743+0000 7f3f65c79700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3f4c038820 0x7f3f4c03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.744+0000 7f3f65c79700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3f4c038820 0x7f3f4c03ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f50006fd0 tx=0x7f3f50006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.749+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f5c018530 con 0x7f3f60106890 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.976+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3f6007ac90 con 0x7f3f4c038820 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.979+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mon.0 v2:172.21.15.57:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f5c018530 con 0x7f3f60106890 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.986+0000 7f3f577fe700 1 -- 172.21.15.57:0/2645864649 <== mgr.14118 v2:172.21.15.57:6800/4144294466 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3f6007ac90 con 0x7f3f4c038820 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3f4c038820 msgr2=0x7f3f4c03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3f4c038820 0x7f3f4c03ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f50006fd0 tx=0x7f3f50006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 msgr2=0x7f3f6007bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f6007bae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009130 tx=0x7f3f5c011ad0 comp rx=0 tx=0).stop 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 shutdown_connections 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3f4c038820 0x7f3f4c03ace0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 --2- 172.21.15.57:0/2645864649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f60106890 0x7f3f6007bae0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 >> 172.21.15.57:0/2645864649 conn(0x7f3f60101e50 msgr2=0x7f3f60103540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 shutdown_connections 2023-10-06T22:05:52.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:51.988+0000 7f3f686de700 1 -- 172.21.15.57:0/2645864649 wait complete. 2023-10-06T22:05:53.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:52 smithi057 ceph-mon[98492]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:05:53.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:52 smithi057 ceph-mon[98492]: Saving service mgr spec with placement count:2 2023-10-06T22:05:53.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:52 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:53.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:52 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:53.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:52 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.954+0000 7f39a351b700 1 Processor -- start 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.954+0000 7f39a351b700 1 -- start start 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.954+0000 7f39a351b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c07a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.954+0000 7f39a351b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c07a760 con 0x7f399c07bd30 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.955+0000 7f39a12b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c07a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.955+0000 7f39a12b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c07a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39654/0 (socket says 172.21.15.57:39654) 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.955+0000 7f39a12b7700 1 -- 172.21.15.57:0/328518986 learned_addr learned my addr 172.21.15.57:0/328518986 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:53.683 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.955+0000 7f39a12b7700 1 -- 172.21.15.57:0/328518986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c07a8a0 con 0x7f399c07bd30 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.955+0000 7f39a12b7700 1 --2- 172.21.15.57:0/328518986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c07a190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39980098a0 tx=0x7f3998009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17faf49759b4948b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f3993fff700 1 -- 172.21.15.57:0/328518986 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3998004160 con 0x7f399c07bd30 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f3993fff700 1 -- 172.21.15.57:0/328518986 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39980042c0 con 0x7f399c07bd30 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f3993fff700 1 -- 172.21.15.57:0/328518986 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39980119b0 con 0x7f399c07bd30 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f39a351b700 1 -- 172.21.15.57:0/328518986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 msgr2=0x7f399c07a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f39a351b700 1 --2- 172.21.15.57:0/328518986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c07a190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39980098a0 tx=0x7f3998009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f39a351b700 1 -- 172.21.15.57:0/328518986 shutdown_connections 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f39a351b700 1 --2- 172.21.15.57:0/328518986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c07a190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f39a351b700 1 -- 172.21.15.57:0/328518986 >> 172.21.15.57:0/328518986 conn(0x7f399c101e80 msgr2=0x7f399c1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.956+0000 7f39a351b700 1 -- 172.21.15.57:0/328518986 shutdown_connections 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.957+0000 7f39a351b700 1 -- 172.21.15.57:0/328518986 wait complete. 2023-10-06T22:05:53.684 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.957+0000 7f39a351b700 1 Processor -- start 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.958+0000 7f39a351b700 1 -- start start 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.958+0000 7f39a351b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.958+0000 7f39a351b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c19cd30 con 0x7f399c07bd30 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.958+0000 7f39a12b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.958+0000 7f39a12b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39656/0 (socket says 172.21.15.57:39656) 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.958+0000 7f39a12b7700 1 -- 172.21.15.57:0/1567842763 learned_addr learned my addr 172.21.15.57:0/1567842763 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.959+0000 7f39a12b7700 1 -- 172.21.15.57:0/1567842763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998009550 con 0x7f399c07bd30 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.959+0000 7f39a12b7700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c19c7f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3998003e60 tx=0x7f3998011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.959+0000 7f39927fc700 1 -- 172.21.15.57:0/1567842763 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3998003a60 con 0x7f399c07bd30 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.959+0000 7f39927fc700 1 -- 172.21.15.57:0/1567842763 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3998003bc0 con 0x7f399c07bd30 2023-10-06T22:05:53.685 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.959+0000 7f39927fc700 1 -- 172.21.15.57:0/1567842763 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39980196c0 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.959+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c19cf30 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.959+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c19d350 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.960+0000 7f39927fc700 1 -- 172.21.15.57:0/1567842763 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f399801ab60 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.960+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c196880 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.961+0000 7f39927fc700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3988038800 0x7f398803acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.961+0000 7f39927fc700 1 -- 172.21.15.57:0/1567842763 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f399804cb50 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.961+0000 7f39a0ab6700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3988038800 0x7f398803acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.961+0000 7f39a0ab6700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3988038800 0x7f398803acc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f398c006fd0 tx=0x7f398c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:52.966+0000 7f39927fc700 1 -- 172.21.15.57:0/1567842763 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3998010790 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.183+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f399c0624e0 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.188+0000 7f39927fc700 1 -- 172.21.15.57:0/1567842763 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f3998010380 con 0x7f399c07bd30 2023-10-06T22:05:53.686 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.190+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3988038800 msgr2=0x7f398803acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.190+0000 7f39a351b700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3988038800 0x7f398803acc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f398c006fd0 tx=0x7f398c006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.190+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 msgr2=0x7f399c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.190+0000 7f39a351b700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c19c7f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3998003e60 tx=0x7f3998011a30 comp rx=0 tx=0).stop 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.191+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 shutdown_connections 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.191+0000 7f39a351b700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f3988038800 0x7f398803acc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.191+0000 7f39a351b700 1 --2- 172.21.15.57:0/1567842763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f399c07bd30 0x7f399c19c7f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.191+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 >> 172.21.15.57:0/1567842763 conn(0x7f399c101e80 msgr2=0x7f399c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.191+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 shutdown_connections 2023-10-06T22:05:53.687 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:53.191+0000 7f39a351b700 1 -- 172.21.15.57:0/1567842763 wait complete. 2023-10-06T22:05:54.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:54 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1567842763' entity='client.admin' 2023-10-06T22:05:55.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.228+0000 7f0882721700 1 Processor -- start 2023-10-06T22:05:55.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.228+0000 7f0882721700 1 -- start start 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.229+0000 7f0882721700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c07a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.229+0000 7f0882721700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c07a780 con 0x7f087c07bd50 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.229+0000 7f087bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c07a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.229+0000 7f087bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c07a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39670/0 (socket says 172.21.15.57:39670) 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.229+0000 7f087bfff700 1 -- 172.21.15.57:0/1829968662 learned_addr learned my addr 172.21.15.57:0/1829968662 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.229+0000 7f087bfff700 1 -- 172.21.15.57:0/1829968662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c07a8c0 con 0x7f087c07bd50 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.230+0000 7f087bfff700 1 --2- 172.21.15.57:0/1829968662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c07a1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f086c0098a0 tx=0x7f086c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308f7db6f6cfb102 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.230+0000 7f087affd700 1 -- 172.21.15.57:0/1829968662 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f086c004160 con 0x7f087c07bd50 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.230+0000 7f087affd700 1 -- 172.21.15.57:0/1829968662 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086c0042c0 con 0x7f087c07bd50 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.230+0000 7f087affd700 1 -- 172.21.15.57:0/1829968662 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f086c0119b0 con 0x7f087c07bd50 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.230+0000 7f0882721700 1 -- 172.21.15.57:0/1829968662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 msgr2=0x7f087c07a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:55.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.231+0000 7f0882721700 1 --2- 172.21.15.57:0/1829968662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c07a1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f086c0098a0 tx=0x7f086c009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.231+0000 7f0882721700 1 -- 172.21.15.57:0/1829968662 shutdown_connections 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.231+0000 7f0882721700 1 --2- 172.21.15.57:0/1829968662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c07a1b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.231+0000 7f0882721700 1 -- 172.21.15.57:0/1829968662 >> 172.21.15.57:0/1829968662 conn(0x7f087c101e50 msgr2=0x7f087c104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.231+0000 7f0882721700 1 -- 172.21.15.57:0/1829968662 shutdown_connections 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.231+0000 7f0882721700 1 -- 172.21.15.57:0/1829968662 wait complete. 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.232+0000 7f0882721700 1 Processor -- start 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.232+0000 7f0882721700 1 -- start start 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.232+0000 7f0882721700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.232+0000 7f0882721700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c19cd00 con 0x7f087c07bd50 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f087bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:55.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f087bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39676/0 (socket says 172.21.15.57:39676) 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f087bfff700 1 -- 172.21.15.57:0/1887314616 learned_addr learned my addr 172.21.15.57:0/1887314616 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f087bfff700 1 -- 172.21.15.57:0/1887314616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c009550 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f087bfff700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c19c7c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f086c006af0 tx=0x7f086c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f08797fa700 1 -- 172.21.15.57:0/1887314616 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f086c003a40 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f08797fa700 1 -- 172.21.15.57:0/1887314616 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086c003ba0 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f08797fa700 1 -- 172.21.15.57:0/1887314616 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f086c019690 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.233+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c19cf00 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.234+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c19d320 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.235+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c0624e0 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.235+0000 7f08797fa700 1 -- 172.21.15.57:0/1887314616 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f086c01aa60 con 0x7f087c07bd50 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.235+0000 7f08797fa700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0864038820 0x7f086403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:55.071 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.235+0000 7f08797fa700 1 -- 172.21.15.57:0/1887314616 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f086c04c9d0 con 0x7f087c07bd50 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.235+0000 7f087b7fe700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0864038820 0x7f086403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.236+0000 7f087b7fe700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0864038820 0x7f086403ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0870006fd0 tx=0x7f0870006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.240+0000 7f08797fa700 1 -- 172.21.15.57:0/1887314616 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f086c01a430 con 0x7f087c07bd50 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.467+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f087c19fd60 con 0x7f087c07bd50 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.471+0000 7f08797fa700 1 -- 172.21.15.57:0/1887314616 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f086c018350 con 0x7f087c07bd50 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.473+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0864038820 msgr2=0x7f086403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.473+0000 7f0882721700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0864038820 0x7f086403ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0870006fd0 tx=0x7f0870006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.473+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 msgr2=0x7f087c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.473+0000 7f0882721700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c19c7c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f086c006af0 tx=0x7f086c011a10 comp rx=0 tx=0).stop 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.474+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 shutdown_connections 2023-10-06T22:05:55.072 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.474+0000 7f0882721700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f0864038820 0x7f086403ace0 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0870006fd0 tx=0x7f0870006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:55.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.474+0000 7f0882721700 1 --2- 172.21.15.57:0/1887314616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087c07bd50 0x7f087c19c7c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:55.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.474+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 >> 172.21.15.57:0/1887314616 conn(0x7f087c101e50 msgr2=0x7f087c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:55.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.474+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 shutdown_connections 2023-10-06T22:05:55.073 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:54.474+0000 7f0882721700 1 -- 172.21.15.57:0/1887314616 wait complete. 2023-10-06T22:05:55.073 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling the dashboard module... 2023-10-06T22:05:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:55 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1887314616' entity='client.admin' 2023-10-06T22:05:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:55 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:56.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:56 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3029165841' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-06T22:05:56.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:56 smithi057 ceph-mon[98492]: from='mgr.14118 172.21.15.57:0/531592784' entity='mgr.x' 2023-10-06T22:05:56.861 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: ignoring --setuser ceph since I am not root 2023-10-06T22:05:56.861 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: ignoring --setgroup ceph since I am not root 2023-10-06T22:05:56.862 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:56.858+0000 7fbd63068200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:05:57.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.629+0000 7feaf98fb700 1 Processor -- start 2023-10-06T22:05:57.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.629+0000 7feaf98fb700 1 -- start start 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.630+0000 7feaf98fb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf407a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.630+0000 7feaf98fb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf407a740 con 0x7feaf407bd10 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.630+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf407a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.630+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf407a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39678/0 (socket says 172.21.15.57:39678) 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.630+0000 7feaf2ffd700 1 -- 172.21.15.57:0/1827793762 learned_addr learned my addr 172.21.15.57:0/1827793762 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.631+0000 7feaf2ffd700 1 -- 172.21.15.57:0/1827793762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf407a880 con 0x7feaf407bd10 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.631+0000 7feaf2ffd700 1 --2- 172.21.15.57:0/1827793762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf407a170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feae40098a0 tx=0x7feae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fc6329a1b4975a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.631+0000 7feaf1ffb700 1 -- 172.21.15.57:0/1827793762 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae4004160 con 0x7feaf407bd10 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.631+0000 7feaf1ffb700 1 -- 172.21.15.57:0/1827793762 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feae40042c0 con 0x7feaf407bd10 2023-10-06T22:05:57.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.631+0000 7feaf1ffb700 1 -- 172.21.15.57:0/1827793762 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae40119b0 con 0x7feaf407bd10 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.632+0000 7feaf98fb700 1 -- 172.21.15.57:0/1827793762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 msgr2=0x7feaf407a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.632+0000 7feaf98fb700 1 --2- 172.21.15.57:0/1827793762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf407a170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feae40098a0 tx=0x7feae4009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.632+0000 7feaf98fb700 1 -- 172.21.15.57:0/1827793762 shutdown_connections 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.632+0000 7feaf98fb700 1 --2- 172.21.15.57:0/1827793762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf407a170 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.632+0000 7feaf98fb700 1 -- 172.21.15.57:0/1827793762 >> 172.21.15.57:0/1827793762 conn(0x7feaf4101e50 msgr2=0x7feaf4104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.632+0000 7feaf98fb700 1 -- 172.21.15.57:0/1827793762 shutdown_connections 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.632+0000 7feaf98fb700 1 -- 172.21.15.57:0/1827793762 wait complete. 2023-10-06T22:05:57.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.633+0000 7feaf98fb700 1 Processor -- start 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.633+0000 7feaf98fb700 1 -- start start 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.634+0000 7feaf98fb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.634+0000 7feaf98fb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419cc80 con 0x7feaf407bd10 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.634+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.634+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39692/0 (socket says 172.21.15.57:39692) 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.634+0000 7feaf2ffd700 1 -- 172.21.15.57:0/3029165841 learned_addr learned my addr 172.21.15.57:0/3029165841 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.634+0000 7feaf2ffd700 1 -- 172.21.15.57:0/3029165841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4009550 con 0x7feaf407bd10 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.634+0000 7feaf2ffd700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf419c740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feae4003f00 tx=0x7feae4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.635+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae4003b20 con 0x7feaf407bd10 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.635+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feae4003c80 con 0x7feaf407bd10 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.635+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae4019770 con 0x7feaf407bd10 2023-10-06T22:05:57.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.635+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf419ce80 con 0x7feaf407bd10 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.635+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf419d2a0 con 0x7feaf407bd10 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.636+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7feae401ab60 con 0x7feaf407bd10 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.636+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf410ccd0 con 0x7feaf407bd10 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.637+0000 7feaf88f9700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7feadc038870 0x7feadc03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.637+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feae404cb50 con 0x7feaf407bd10 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.637+0000 7feaf27fc700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7feadc038870 0x7feadc03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.638+0000 7feaf27fc700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7feadc038870 0x7feadc03ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feae8006fd0 tx=0x7feae8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.642+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feae401a430 con 0x7feaf407bd10 2023-10-06T22:05:57.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:55.925+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7feaf410cee0 con 0x7feaf407bd10 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.541+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7feae402db40 con 0x7feaf407bd10 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.543+0000 7feaf88f9700 1 -- 172.21.15.57:0/3029165841 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7feae402d350 con 0x7feaf407bd10 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.545+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7feadc038870 msgr2=0x7feadc03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.545+0000 7feaf98fb700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7feadc038870 0x7feadc03ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feae8006fd0 tx=0x7feae8006e40 comp rx=0 tx=0).stop 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.545+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 msgr2=0x7feaf419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.545+0000 7feaf98fb700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf419c740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feae4003f00 tx=0x7feae4011a30 comp rx=0 tx=0).stop 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.545+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 shutdown_connections 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.546+0000 7feaf98fb700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7feadc038870 0x7feadc03ad30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.546+0000 7feaf98fb700 1 --2- 172.21.15.57:0/3029165841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaf407bd10 0x7feaf419c740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.546+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 >> 172.21.15.57:0/3029165841 conn(0x7feaf4101e50 msgr2=0x7feaf4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.546+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 shutdown_connections 2023-10-06T22:05:57.060 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:56.546+0000 7feaf98fb700 1 -- 172.21.15.57:0/3029165841 wait complete. 2023-10-06T22:05:57.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:56.977+0000 7fbd63068200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:05:57.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:57.103+0000 7fbd63068200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:05:57.545 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:57.235+0000 7fbd63068200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:05:57.545 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:57.480+0000 7fbd63068200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:05:57.813 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:57 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3029165841' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-06T22:05:57.813 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:57 smithi057 ceph-mon[98492]: mgrmap e9: x(active, since 18s) 2023-10-06T22:05:58.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:58.014+0000 7fbd63068200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:05:58.445 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:58.190+0000 7fbd63068200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:05:58.445 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:58.312+0000 7fbd63068200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:05:58.556 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-06T22:05:58.556 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-06T22:05:58.556 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T22:05:58.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-06T22:05:58.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-06T22:05:58.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-06T22:05:58.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.692+0000 7f7daff34700 1 Processor -- start 2023-10-06T22:05:58.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.692+0000 7f7daff34700 1 -- start start 2023-10-06T22:05:58.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.692+0000 7f7daff34700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da807a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:58.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.692+0000 7f7daff34700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da807a720 con 0x7f7da807bcf0 2023-10-06T22:05:58.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.693+0000 7f7dadcd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da807a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:58.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.693+0000 7f7dadcd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da807a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39720/0 (socket says 172.21.15.57:39720) 2023-10-06T22:05:58.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.693+0000 7f7dadcd0700 1 -- 172.21.15.57:0/214644484 learned_addr learned my addr 172.21.15.57:0/214644484 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:58.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.693+0000 7f7dadcd0700 1 -- 172.21.15.57:0/214644484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da807a860 con 0x7f7da807bcf0 2023-10-06T22:05:58.559 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.693+0000 7f7dadcd0700 1 --2- 172.21.15.57:0/214644484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da807a150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7da40098a0 tx=0x7f7da4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed0970bc59a9492 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:58.559 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.694+0000 7f7daccce700 1 -- 172.21.15.57:0/214644484 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7da4004160 con 0x7f7da807bcf0 2023-10-06T22:05:58.559 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.694+0000 7f7daccce700 1 -- 172.21.15.57:0/214644484 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7da40042c0 con 0x7f7da807bcf0 2023-10-06T22:05:58.559 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.694+0000 7f7daccce700 1 -- 172.21.15.57:0/214644484 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7da40119b0 con 0x7f7da807bcf0 2023-10-06T22:05:58.560 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.694+0000 7f7daff34700 1 -- 172.21.15.57:0/214644484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 msgr2=0x7f7da807a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:58.560 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.694+0000 7f7daff34700 1 --2- 172.21.15.57:0/214644484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da807a150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7da40098a0 tx=0x7f7da4009bb0 comp rx=0 tx=0).stop 2023-10-06T22:05:58.560 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.695+0000 7f7daff34700 1 -- 172.21.15.57:0/214644484 shutdown_connections 2023-10-06T22:05:58.560 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.695+0000 7f7daff34700 1 --2- 172.21.15.57:0/214644484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da807a150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:58.560 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.695+0000 7f7daff34700 1 -- 172.21.15.57:0/214644484 >> 172.21.15.57:0/214644484 conn(0x7f7da8101e80 msgr2=0x7f7da81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:58.561 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.695+0000 7f7daff34700 1 -- 172.21.15.57:0/214644484 shutdown_connections 2023-10-06T22:05:58.561 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.695+0000 7f7daff34700 1 -- 172.21.15.57:0/214644484 wait complete. 2023-10-06T22:05:58.561 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.696+0000 7f7daff34700 1 Processor -- start 2023-10-06T22:05:58.561 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.696+0000 7f7daff34700 1 -- start start 2023-10-06T22:05:58.561 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.696+0000 7f7daff34700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.696+0000 7f7daff34700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da819cce0 con 0x7f7da807bcf0 2023-10-06T22:05:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.697+0000 7f7dadcd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:05:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.697+0000 7f7dadcd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39734/0 (socket says 172.21.15.57:39734) 2023-10-06T22:05:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.697+0000 7f7dadcd0700 1 -- 172.21.15.57:0/3089731154 learned_addr learned my addr 172.21.15.57:0/3089731154 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:05:58.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.697+0000 7f7dadcd0700 1 -- 172.21.15.57:0/3089731154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da4009550 con 0x7f7da807bcf0 2023-10-06T22:05:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.698+0000 7f7dadcd0700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da819c7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7da4003f00 tx=0x7f7da4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:05:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.698+0000 7f7d9effd700 1 -- 172.21.15.57:0/3089731154 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7da4003b20 con 0x7f7da807bcf0 2023-10-06T22:05:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.698+0000 7f7d9effd700 1 -- 172.21.15.57:0/3089731154 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7da4003c80 con 0x7f7da807bcf0 2023-10-06T22:05:58.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.698+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da819cee0 con 0x7f7da807bcf0 2023-10-06T22:05:58.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.698+0000 7f7d9effd700 1 -- 172.21.15.57:0/3089731154 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7da4019770 con 0x7f7da807bcf0 2023-10-06T22:05:58.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.698+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da819d300 con 0x7f7da807bcf0 2023-10-06T22:05:58.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.700+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da810cc80 con 0x7f7da807bcf0 2023-10-06T22:05:58.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.700+0000 7f7d9effd700 1 -- 172.21.15.57:0/3089731154 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f7da401ab60 con 0x7f7da807bcf0 2023-10-06T22:05:58.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.701+0000 7f7d9effd700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 0x7f7d9403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:05:58.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.701+0000 7f7d9effd700 1 -- 172.21.15.57:0/3089731154 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7da404cbb0 con 0x7f7da807bcf0 2023-10-06T22:05:58.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.701+0000 7f7dad4cf700 1 -- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 msgr2=0x7f7d9403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:05:58.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.701+0000 7f7dad4cf700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 0x7f7d9403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:05:58.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.706+0000 7f7d9effd700 1 -- 172.21.15.57:0/3089731154 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7da402d7c0 con 0x7f7da807bcf0 2023-10-06T22:05:58.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.902+0000 7f7dad4cf700 1 -- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 msgr2=0x7f7d9403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:05:58.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:57.902+0000 7f7dad4cf700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 0x7f7d9403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T22:05:58.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.004+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7da810cef0 con 0x7f7da807bcf0 2023-10-06T22:05:58.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.005+0000 7f7d9effd700 1 -- 172.21.15.57:0/3089731154 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f7da4023e40 con 0x7f7da807bcf0 2023-10-06T22:05:58.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 msgr2=0x7f7d9403ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:05:58.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 0x7f7d9403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:58.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 msgr2=0x7f7da819c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:05:58.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da819c7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7da4003f00 tx=0x7f7da4011a30 comp rx=0 tx=0).stop 2023-10-06T22:05:58.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 shutdown_connections 2023-10-06T22:05:58.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f7d940388c0 0x7f7d9403ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:58.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 --2- 172.21.15.57:0/3089731154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7da807bcf0 0x7f7da819c7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:05:58.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 >> 172.21.15.57:0/3089731154 conn(0x7f7da8101e80 msgr2=0x7f7da8103530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:05:58.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 shutdown_connections 2023-10-06T22:05:58.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:58.007+0000 7f7daff34700 1 -- 172.21.15.57:0/3089731154 wait complete. 2023-10-06T22:05:58.569 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2023-10-06T22:05:58.569 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 9... 2023-10-06T22:05:58.824 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:05:58 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3089731154' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-06T22:05:58.824 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:58.442+0000 7fbd63068200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:05:59.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:59.023+0000 7fbd63068200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:06:00.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:05:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:05:59.904+0000 7fbd63068200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:06:00.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:00.013+0000 7fbd63068200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:06:01.361 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:01.004+0000 7fbd63068200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:06:01.361 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:01.127+0000 7fbd63068200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:06:01.361 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:01.358+0000 7fbd63068200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:06:02.020 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:01.734+0000 7fbd63068200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:06:02.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:02.018+0000 7fbd63068200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:06:02.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:02.218+0000 7fbd63068200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:06:02.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:02.576+0000 7fbd63068200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:06:03.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:03.541+0000 7fbd63068200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:06:03.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:03.721+0000 7fbd63068200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:06:04.419 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:03.979+0000 7fbd63068200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:06:04.419 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:04.130+0000 7fbd63068200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:06:04.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:04.416+0000 7fbd63068200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:06:05.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:06:05.525+0000 7fbd63068200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:06:05.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: Active manager daemon x restarted 2023-10-06T22:06:05.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: Activating manager daemon x 2023-10-06T22:06:05.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: osdmap e3: 0 total, 0 up, 0 in 2023-10-06T22:06:05.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: mgrmap e10: x(active, starting, since 0.00832544s) 2023-10-06T22:06:05.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:06:05.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T22:06:05.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T22:06:05.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T22:06:05.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T22:06:05.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: Manager daemon x is now available 2023-10-06T22:06:05.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:05.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T22:06:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T22:06:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:06 smithi057 ceph-mon[98492]: mgrmap e11: x(active, since 1.01638s) 2023-10-06T22:06:07.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.094+0000 7f3523c5e700 1 Processor -- start 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.094+0000 7f3523c5e700 1 -- start start 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.094+0000 7f3523c5e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.094+0000 7f3523c5e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c107270 con 0x7f351c106880 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.095+0000 7f35219fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.095+0000 7f35219fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c106ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39738/0 (socket says 172.21.15.57:39738) 2023-10-06T22:06:07.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.095+0000 7f35219fa700 1 -- 172.21.15.57:0/3165827834 learned_addr learned my addr 172.21.15.57:0/3165827834 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.095+0000 7f35219fa700 1 -- 172.21.15.57:0/3165827834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c107a80 con 0x7f351c106880 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.096+0000 7f35219fa700 1 --2- 172.21.15.57:0/3165827834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c106ca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35180098a0 tx=0x7f3518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253555da0c5a087b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.096+0000 7f35209f8700 1 -- 172.21.15.57:0/3165827834 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3518004160 con 0x7f351c106880 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.096+0000 7f35209f8700 1 -- 172.21.15.57:0/3165827834 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35180042c0 con 0x7f351c106880 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.096+0000 7f35209f8700 1 -- 172.21.15.57:0/3165827834 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35180119b0 con 0x7f351c106880 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.097+0000 7f3523c5e700 1 -- 172.21.15.57:0/3165827834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 msgr2=0x7f351c106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.097+0000 7f3523c5e700 1 --2- 172.21.15.57:0/3165827834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c106ca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35180098a0 tx=0x7f3518009bb0 comp rx=0 tx=0).stop 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.097+0000 7f3523c5e700 1 -- 172.21.15.57:0/3165827834 shutdown_connections 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.097+0000 7f3523c5e700 1 --2- 172.21.15.57:0/3165827834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c106ca0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.097+0000 7f3523c5e700 1 -- 172.21.15.57:0/3165827834 >> 172.21.15.57:0/3165827834 conn(0x7f351c101e80 msgr2=0x7f351c1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:07.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.097+0000 7f3523c5e700 1 -- 172.21.15.57:0/3165827834 shutdown_connections 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.097+0000 7f3523c5e700 1 -- 172.21.15.57:0/3165827834 wait complete. 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.098+0000 7f3523c5e700 1 Processor -- start 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.098+0000 7f3523c5e700 1 -- start start 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.099+0000 7f3523c5e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.099+0000 7f3523c5e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c19ab10 con 0x7f351c106880 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.099+0000 7f35219fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.099+0000 7f35219fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39752/0 (socket says 172.21.15.57:39752) 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.099+0000 7f35219fa700 1 -- 172.21.15.57:0/2174034125 learned_addr learned my addr 172.21.15.57:0/2174034125 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.099+0000 7f35219fa700 1 -- 172.21.15.57:0/2174034125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009550 con 0x7f351c106880 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.100+0000 7f35219fa700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c19a5d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3518003e60 tx=0x7f3518011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.100+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3518003a60 con 0x7f351c106880 2023-10-06T22:06:07.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.100+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3518003bc0 con 0x7f351c106880 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.100+0000 7f3523c5e700 1 -- 172.21.15.57:0/2174034125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c19ad10 con 0x7f351c106880 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.100+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35180196c0 con 0x7f351c106880 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.100+0000 7f3523c5e700 1 -- 172.21.15.57:0/2174034125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c19b130 con 0x7f351c106880 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.101+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f3518019930 con 0x7f351c106880 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.102+0000 7f3512ffd700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.102+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 --> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f351c0624e0 con 0x7f35080388a0 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.102+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f351804cc70 con 0x7f351c106880 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.102+0000 7f35211f9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 msgr2=0x7f350803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.102+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.303+0000 7f35211f9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 msgr2=0x7f350803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.303+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T22:06:07.056 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.704+0000 7f35211f9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 msgr2=0x7f350803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:05:59.704+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:00.505+0000 7f35211f9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 msgr2=0x7f350803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:00.505+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:02.106+0000 7f35211f9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 msgr2=0x7f350803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:02.106+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:05.309+0000 7f35211f9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 msgr2=0x7f350803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/4144294466 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:05.309+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:05.534+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 10) v1 ==== 45359+0+0 (secure 0 0 0) 0x7f351802ca50 con 0x7f351c106880 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:05.534+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 msgr2=0x7f350803ad60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:05.534+0000 7f3512ffd700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.540+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 11) v1 ==== 45486+0+0 (secure 0 0 0) 0x7f351804e730 con 0x7f351c106880 2023-10-06T22:06:07.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.540+0000 7f3512ffd700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f350803bfd0 0x7f350803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.540+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f351c0624e0 con 0x7f350803bfd0 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.541+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f350803bfd0 0x7f350803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.542+0000 7f35211f9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f350803bfd0 0x7f350803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f350c003a10 tx=0x7f350c009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.543+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f351c0624e0 con 0x7f350803bfd0 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.550+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f351c19b910 con 0x7f350803bfd0 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.550+0000 7f3512ffd700 1 -- 172.21.15.57:0/2174034125 <== mgr.14150 v2:172.21.15.57:6800/409950778 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f351c19b910 con 0x7f350803bfd0 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.550+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f350803bfd0 msgr2=0x7f350803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.550+0000 7f3510ff9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f350803bfd0 0x7f350803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f350c003a10 tx=0x7f350c009270 comp rx=0 tx=0).stop 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.550+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 msgr2=0x7f351c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.550+0000 7f3510ff9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c19a5d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3518003e60 tx=0x7f3518011a30 comp rx=0 tx=0).stop 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.551+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 shutdown_connections 2023-10-06T22:06:07.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.551+0000 7f3510ff9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f350803bfd0 0x7f350803e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:07.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.551+0000 7f3510ff9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f351c106880 0x7f351c19a5d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:07.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.551+0000 7f3510ff9700 1 --2- 172.21.15.57:0/2174034125 >> [v2:172.21.15.57:6800/4144294466,v1:172.21.15.57:6801/4144294466] conn(0x7f35080388a0 0x7f350803ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:07.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.551+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 >> 172.21.15.57:0/2174034125 conn(0x7f351c101e80 msgr2=0x7f351c103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:07.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.551+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 shutdown_connections 2023-10-06T22:06:07.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:06.551+0000 7f3510ff9700 1 -- 172.21.15.57:0/2174034125 wait complete. 2023-10-06T22:06:07.059 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 9 is available 2023-10-06T22:06:07.059 INFO:teuthology.orchestra.run.smithi057.stdout:Generating a dashboard self-signed certificate... 2023-10-06T22:06:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:07 smithi057 ceph-mon[98492]: [06/Oct/2023:22:06:06] ENGINE Bus STARTING 2023-10-06T22:06:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:07 smithi057 ceph-mon[98492]: [06/Oct/2023:22:06:06] ENGINE Serving on https://172.21.15.57:7150 2023-10-06T22:06:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:07 smithi057 ceph-mon[98492]: [06/Oct/2023:22:06:06] ENGINE Serving on http://172.21.15.57:8765 2023-10-06T22:06:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:07 smithi057 ceph-mon[98492]: [06/Oct/2023:22:06:06] ENGINE Bus STARTED 2023-10-06T22:06:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:07 smithi057 ceph-mon[98492]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-06T22:06:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:07 smithi057 ceph-mon[98492]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-06T22:06:08.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-06T22:06:08.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.840+0000 7fedcccd7700 1 Processor -- start 2023-10-06T22:06:08.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.840+0000 7fedcccd7700 1 -- start start 2023-10-06T22:06:08.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.840+0000 7fedcccd7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc807a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:08.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.840+0000 7fedcccd7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc807a780 con 0x7fedc807bd50 2023-10-06T22:06:08.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.841+0000 7fedc659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc807a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:08.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.841+0000 7fedc659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc807a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34474/0 (socket says 172.21.15.57:34474) 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.841+0000 7fedc659c700 1 -- 172.21.15.57:0/582292427 learned_addr learned my addr 172.21.15.57:0/582292427 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.841+0000 7fedc659c700 1 -- 172.21.15.57:0/582292427 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc807a8c0 con 0x7fedc807bd50 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.842+0000 7fedc659c700 1 --2- 172.21.15.57:0/582292427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc807a1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a137d362ab513488 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.842+0000 7fedc559a700 1 -- 172.21.15.57:0/582292427 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedb8004160 con 0x7fedc807bd50 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.842+0000 7fedc559a700 1 -- 172.21.15.57:0/582292427 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedb80042c0 con 0x7fedc807bd50 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.842+0000 7fedc559a700 1 -- 172.21.15.57:0/582292427 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedb80119b0 con 0x7fedc807bd50 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.842+0000 7fedcccd7700 1 -- 172.21.15.57:0/582292427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 msgr2=0x7fedc807a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.842+0000 7fedcccd7700 1 --2- 172.21.15.57:0/582292427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc807a1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).stop 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.843+0000 7fedcccd7700 1 -- 172.21.15.57:0/582292427 shutdown_connections 2023-10-06T22:06:08.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.843+0000 7fedcccd7700 1 --2- 172.21.15.57:0/582292427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc807a1b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.843+0000 7fedcccd7700 1 -- 172.21.15.57:0/582292427 >> 172.21.15.57:0/582292427 conn(0x7fedc8101e80 msgr2=0x7fedc81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.843+0000 7fedcccd7700 1 -- 172.21.15.57:0/582292427 shutdown_connections 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.843+0000 7fedcccd7700 1 -- 172.21.15.57:0/582292427 wait complete. 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.844+0000 7fedcccd7700 1 Processor -- start 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.844+0000 7fedcccd7700 1 -- start start 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.844+0000 7fedcccd7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.844+0000 7fedcccd7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc819ccf0 con 0x7fedc807bd50 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.844+0000 7fedc659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.844+0000 7fedc659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34484/0 (socket says 172.21.15.57:34484) 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.844+0000 7fedc659c700 1 -- 172.21.15.57:0/3807151631 learned_addr learned my addr 172.21.15.57:0/3807151631 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.845+0000 7fedc659c700 1 -- 172.21.15.57:0/3807151631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009550 con 0x7fedc807bd50 2023-10-06T22:06:08.643 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.845+0000 7fedc659c700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc819c7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedb8006af0 tx=0x7fedb8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.845+0000 7fedb77fe700 1 -- 172.21.15.57:0/3807151631 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedb8003a40 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.845+0000 7fedb77fe700 1 -- 172.21.15.57:0/3807151631 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedb8003ba0 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.845+0000 7fedb77fe700 1 -- 172.21.15.57:0/3807151631 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedb8019690 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.845+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc819cef0 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.845+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc819d310 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.846+0000 7fedb77fe700 1 -- 172.21.15.57:0/3807151631 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7fedb801aa60 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.846+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc80624e0 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.847+0000 7fedb77fe700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedb0038870 0x7fedb003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.847+0000 7fedb77fe700 1 -- 172.21.15.57:0/3807151631 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fedb804cc70 con 0x7fedc807bd50 2023-10-06T22:06:08.644 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.847+0000 7fedc5d9b700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedb0038870 0x7fedb003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.848+0000 7fedc5d9b700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedb0038870 0x7fedb003ad30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:07.853+0000 7fedb77fe700 1 -- 172.21.15.57:0/3807151631 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fedb802d430 con 0x7fedc807bd50 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.085+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fedc81041d0 con 0x7fedb0038870 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.152+0000 7fedb77fe700 1 -- 172.21.15.57:0/3807151631 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fedc81041d0 con 0x7fedb0038870 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedb0038870 msgr2=0x7fedb003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedb0038870 0x7fedb003ad30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 msgr2=0x7fedc819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc819c7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedb8006af0 tx=0x7fedb8011a10 comp rx=0 tx=0).stop 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 shutdown_connections 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedb0038870 0x7fedb003ad30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:08.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 --2- 172.21.15.57:0/3807151631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fedc807bd50 0x7fedc819c7b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:08.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 >> 172.21.15.57:0/3807151631 conn(0x7fedc8101e80 msgr2=0x7fedc81035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:08.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 shutdown_connections 2023-10-06T22:06:08.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:08.155+0000 7fedcccd7700 1 -- 172.21.15.57:0/3807151631 wait complete. 2023-10-06T22:06:08.646 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial admin user... 2023-10-06T22:06:08.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:08 smithi057 ceph-mon[98492]: mgrmap e12: x(active, since 2s) 2023-10-06T22:06:08.912 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:08.912 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:08.912 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:09.880 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:09 smithi057 ceph-mon[98492]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:09.880 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:10.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$OgtBC2tU8/ByVYfjJed4muKe3ykAoAPpFqLcKJDLiQSl.9DZspuOq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696629969, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.199+0000 7f810ad72700 1 Processor -- start 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.199+0000 7f810ad72700 1 -- start start 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.200+0000 7f810ad72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f8104106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.200+0000 7f810ad72700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8104107280 con 0x7f8104106890 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.200+0000 7f8108b0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f8104106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.200+0000 7f8108b0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f8104106cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34498/0 (socket says 172.21.15.57:34498) 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.200+0000 7f8108b0e700 1 -- 172.21.15.57:0/194711733 learned_addr learned my addr 172.21.15.57:0/194711733 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.201+0000 7f8108b0e700 1 -- 172.21.15.57:0/194711733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8104107a90 con 0x7f8104106890 2023-10-06T22:06:10.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.201+0000 7f8108b0e700 1 --2- 172.21.15.57:0/194711733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f8104106cb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80f80098a0 tx=0x7f80f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2a6977f83da27a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.201+0000 7f81037fe700 1 -- 172.21.15.57:0/194711733 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80f8004160 con 0x7f8104106890 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.201+0000 7f81037fe700 1 -- 172.21.15.57:0/194711733 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80f80042c0 con 0x7f8104106890 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.201+0000 7f81037fe700 1 -- 172.21.15.57:0/194711733 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80f80119b0 con 0x7f8104106890 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.202+0000 7f810ad72700 1 -- 172.21.15.57:0/194711733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 msgr2=0x7f8104106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.202+0000 7f810ad72700 1 --2- 172.21.15.57:0/194711733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f8104106cb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80f80098a0 tx=0x7f80f8009bb0 comp rx=0 tx=0).stop 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.202+0000 7f810ad72700 1 -- 172.21.15.57:0/194711733 shutdown_connections 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.202+0000 7f810ad72700 1 --2- 172.21.15.57:0/194711733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f8104106cb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.202+0000 7f810ad72700 1 -- 172.21.15.57:0/194711733 >> 172.21.15.57:0/194711733 conn(0x7f8104101e50 msgr2=0x7f8104104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.202+0000 7f810ad72700 1 -- 172.21.15.57:0/194711733 shutdown_connections 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.202+0000 7f810ad72700 1 -- 172.21.15.57:0/194711733 wait complete. 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.203+0000 7f810ad72700 1 Processor -- start 2023-10-06T22:06:10.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.203+0000 7f810ad72700 1 -- start start 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.203+0000 7f810ad72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f810407bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.203+0000 7f810ad72700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810407c020 con 0x7f8104106890 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.204+0000 7f8108b0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f810407bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.204+0000 7f8108b0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f810407bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34500/0 (socket says 172.21.15.57:34500) 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.204+0000 7f8108b0e700 1 -- 172.21.15.57:0/2512864619 learned_addr learned my addr 172.21.15.57:0/2512864619 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.204+0000 7f8108b0e700 1 -- 172.21.15.57:0/2512864619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f8009550 con 0x7f8104106890 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.204+0000 7f8108b0e700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f810407bae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80f8006af0 tx=0x7f80f8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.205+0000 7f8101ffb700 1 -- 172.21.15.57:0/2512864619 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80f8003a40 con 0x7f8104106890 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.205+0000 7f8101ffb700 1 -- 172.21.15.57:0/2512864619 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80f8003ba0 con 0x7f8104106890 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.205+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810407a130 con 0x7f8104106890 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.205+0000 7f8101ffb700 1 -- 172.21.15.57:0/2512864619 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80f8019690 con 0x7f8104106890 2023-10-06T22:06:10.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.205+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810407a550 con 0x7f8104106890 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.206+0000 7f8101ffb700 1 -- 172.21.15.57:0/2512864619 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f80f801aa60 con 0x7f8104106890 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.206+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810410acc0 con 0x7f8104106890 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.206+0000 7f8101ffb700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f80f40388c0 0x7f80f403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.206+0000 7f8101ffb700 1 -- 172.21.15.57:0/2512864619 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f80f804cc70 con 0x7f8104106890 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.207+0000 7f8103fff700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f80f40388c0 0x7f80f403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.207+0000 7f8103fff700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f80f40388c0 0x7f80f403ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f80f0006fd0 tx=0x7f80f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.212+0000 7f8101ffb700 1 -- 172.21.15.57:0/2512864619 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f80f80237a0 con 0x7f8104106890 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.467+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8104062300 con 0x7f80f40388c0 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.717+0000 7f8101ffb700 1 -- 172.21.15.57:0/2512864619 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8104062300 con 0x7f80f40388c0 2023-10-06T22:06:10.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.719+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f80f40388c0 msgr2=0x7f80f403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f80f40388c0 0x7f80f403ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f80f0006fd0 tx=0x7f80f0006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 msgr2=0x7f810407bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f810407bae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80f8006af0 tx=0x7f80f8011a10 comp rx=0 tx=0).stop 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 shutdown_connections 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f80f40388c0 0x7f80f403ad80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 --2- 172.21.15.57:0/2512864619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8104106890 0x7f810407bae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 >> 172.21.15.57:0/2512864619 conn(0x7f8104101e50 msgr2=0x7f8104103570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 shutdown_connections 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:09.720+0000 7f810ad72700 1 -- 172.21.15.57:0/2512864619 wait complete. 2023-10-06T22:06:10.129 INFO:teuthology.orchestra.run.smithi057.stdout:Fetching dashboard port number... 2023-10-06T22:06:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:10 smithi057 ceph-mon[98492]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:10.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:10 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 8443 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.677+0000 7fe2c8d56700 1 Processor -- start 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.678+0000 7fe2c8d56700 1 -- start start 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.678+0000 7fe2c8d56700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c407a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.678+0000 7fe2c8d56700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c407a740 con 0x7fe2c407bd10 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.678+0000 7fe2c259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c407a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.678+0000 7fe2c259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c407a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34504/0 (socket says 172.21.15.57:34504) 2023-10-06T22:06:11.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.678+0000 7fe2c259c700 1 -- 172.21.15.57:0/1759148980 learned_addr learned my addr 172.21.15.57:0/1759148980 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.679+0000 7fe2c259c700 1 -- 172.21.15.57:0/1759148980 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c407a880 con 0x7fe2c407bd10 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.679+0000 7fe2c259c700 1 --2- 172.21.15.57:0/1759148980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c407a170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0098a0 tx=0x7fe2ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74321ea53c59675 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.679+0000 7fe2c159a700 1 -- 172.21.15.57:0/1759148980 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac004160 con 0x7fe2c407bd10 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.679+0000 7fe2c159a700 1 -- 172.21.15.57:0/1759148980 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2ac0042c0 con 0x7fe2c407bd10 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.679+0000 7fe2c159a700 1 -- 172.21.15.57:0/1759148980 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0119b0 con 0x7fe2c407bd10 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.680+0000 7fe2c8d56700 1 -- 172.21.15.57:0/1759148980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 msgr2=0x7fe2c407a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.680+0000 7fe2c8d56700 1 --2- 172.21.15.57:0/1759148980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c407a170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0098a0 tx=0x7fe2ac009bb0 comp rx=0 tx=0).stop 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.680+0000 7fe2c8d56700 1 -- 172.21.15.57:0/1759148980 shutdown_connections 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.680+0000 7fe2c8d56700 1 --2- 172.21.15.57:0/1759148980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c407a170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:11.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.680+0000 7fe2c8d56700 1 -- 172.21.15.57:0/1759148980 >> 172.21.15.57:0/1759148980 conn(0x7fe2c4101e50 msgr2=0x7fe2c4104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.681+0000 7fe2c8d56700 1 -- 172.21.15.57:0/1759148980 shutdown_connections 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.681+0000 7fe2c8d56700 1 -- 172.21.15.57:0/1759148980 wait complete. 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.682+0000 7fe2c8d56700 1 Processor -- start 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.682+0000 7fe2c8d56700 1 -- start start 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.682+0000 7fe2c8d56700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.682+0000 7fe2c8d56700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c419cc80 con 0x7fe2c407bd10 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.683+0000 7fe2c259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.683+0000 7fe2c259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34508/0 (socket says 172.21.15.57:34508) 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.683+0000 7fe2c259c700 1 -- 172.21.15.57:0/522889837 learned_addr learned my addr 172.21.15.57:0/522889837 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.683+0000 7fe2c259c700 1 -- 172.21.15.57:0/522889837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac009550 con 0x7fe2c407bd10 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.683+0000 7fe2c259c700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c419c740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac003f00 tx=0x7fe2ac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:11.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.684+0000 7fe2bb7fe700 1 -- 172.21.15.57:0/522889837 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac003b20 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.684+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c419ce80 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.684+0000 7fe2bb7fe700 1 -- 172.21.15.57:0/522889837 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2ac003c80 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.684+0000 7fe2bb7fe700 1 -- 172.21.15.57:0/522889837 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac019770 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.684+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c419d2a0 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.685+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c410ccd0 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.685+0000 7fe2bb7fe700 1 -- 172.21.15.57:0/522889837 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7fe2ac01ab60 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.686+0000 7fe2bb7fe700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2b0038870 0x7fe2b003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.686+0000 7fe2bb7fe700 1 -- 172.21.15.57:0/522889837 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe2ac04cc90 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.686+0000 7fe2c1d9b700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2b0038870 0x7fe2b003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.686+0000 7fe2c1d9b700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2b0038870 0x7fe2b003ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4006fd0 tx=0x7fe2b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.696+0000 7fe2bb7fe700 1 -- 172.21.15.57:0/522889837 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe2ac00b3e0 con 0x7fe2c407bd10 2023-10-06T22:06:11.616 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.920+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fe2c410cee0 con 0x7fe2c407bd10 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.921+0000 7fe2bb7fe700 1 -- 172.21.15.57:0/522889837 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fe2ac018350 con 0x7fe2c407bd10 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.923+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2b0038870 msgr2=0x7fe2b003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.923+0000 7fe2c8d56700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2b0038870 0x7fe2b003ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4006fd0 tx=0x7fe2b4006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.923+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 msgr2=0x7fe2c419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.923+0000 7fe2c8d56700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c419c740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac003f00 tx=0x7fe2ac011a30 comp rx=0 tx=0).stop 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.924+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 shutdown_connections 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.924+0000 7fe2c8d56700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2b0038870 0x7fe2b003ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.924+0000 7fe2c8d56700 1 --2- 172.21.15.57:0/522889837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2c407bd10 0x7fe2c419c740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.924+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 >> 172.21.15.57:0/522889837 conn(0x7fe2c4101e50 msgr2=0x7fe2c4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.924+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 shutdown_connections 2023-10-06T22:06:11.617 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:10.924+0000 7fe2c8d56700 1 -- 172.21.15.57:0/522889837 wait complete. 2023-10-06T22:06:11.628 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T22:06:11.629 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-06T22:06:11.645 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T22:06:11.645 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-06T22:06:11.645 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-06T22:06:11.645 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-06T22:06:11.647 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph Dashboard is now available at: 2023-10-06T22:06:11.647 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:11.648 INFO:teuthology.orchestra.run.smithi057.stdout: URL: https://smithi057.front.sepia.ceph.com:8443/ 2023-10-06T22:06:11.648 INFO:teuthology.orchestra.run.smithi057.stdout: User: admin 2023-10-06T22:06:11.648 INFO:teuthology.orchestra.run.smithi057.stdout: Password: h217xyoeg5 2023-10-06T22:06:11.648 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:11.648 INFO:teuthology.orchestra.run.smithi057.stdout:Saving cluster configuration to /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config directory 2023-10-06T22:06:11.649 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling autotune for osd_memory_target 2023-10-06T22:06:11.905 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:11 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/522889837' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-06T22:06:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:12 smithi057 ceph-mon[98492]: mgrmap e13: x(active, since 6s) 2023-10-06T22:06:12.832 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.172+0000 7f5de9ea0700 1 Processor -- start 2023-10-06T22:06:12.832 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.173+0000 7f5de9ea0700 1 -- start start 2023-10-06T22:06:12.832 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.173+0000 7f5de9ea0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de407a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.173+0000 7f5de9ea0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de407a760 con 0x7f5de407bd30 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.174+0000 7f5de37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de407a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.174+0000 7f5de37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de407a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35408/0 (socket says 172.21.15.57:35408) 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.174+0000 7f5de37fe700 1 -- 172.21.15.57:0/125165822 learned_addr learned my addr 172.21.15.57:0/125165822 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.174+0000 7f5de37fe700 1 -- 172.21.15.57:0/125165822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de407a8a0 con 0x7f5de407bd30 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.174+0000 7f5de37fe700 1 --2- 172.21.15.57:0/125165822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de407a190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40098a0 tx=0x7f5dd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2ddb6f7c8edea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.174+0000 7f5de27fc700 1 -- 172.21.15.57:0/125165822 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dd4004160 con 0x7f5de407bd30 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de27fc700 1 -- 172.21.15.57:0/125165822 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd40042c0 con 0x7f5de407bd30 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de27fc700 1 -- 172.21.15.57:0/125165822 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dd40119b0 con 0x7f5de407bd30 2023-10-06T22:06:12.833 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de9ea0700 1 -- 172.21.15.57:0/125165822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 msgr2=0x7f5de407a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de9ea0700 1 --2- 172.21.15.57:0/125165822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de407a190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40098a0 tx=0x7f5dd4009bb0 comp rx=0 tx=0).stop 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de9ea0700 1 -- 172.21.15.57:0/125165822 shutdown_connections 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de9ea0700 1 --2- 172.21.15.57:0/125165822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de407a190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de9ea0700 1 -- 172.21.15.57:0/125165822 >> 172.21.15.57:0/125165822 conn(0x7f5de4101e50 msgr2=0x7f5de4104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de9ea0700 1 -- 172.21.15.57:0/125165822 shutdown_connections 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.175+0000 7f5de9ea0700 1 -- 172.21.15.57:0/125165822 wait complete. 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.176+0000 7f5de9ea0700 1 Processor -- start 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.177+0000 7f5de9ea0700 1 -- start start 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.177+0000 7f5de9ea0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:12.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.177+0000 7f5de9ea0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de419cd20 con 0x7f5de407bd30 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.177+0000 7f5de37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.177+0000 7f5de37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35420/0 (socket says 172.21.15.57:35420) 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.177+0000 7f5de37fe700 1 -- 172.21.15.57:0/3707596690 learned_addr learned my addr 172.21.15.57:0/3707596690 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.177+0000 7f5de37fe700 1 -- 172.21.15.57:0/3707596690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd4009550 con 0x7f5de407bd30 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.178+0000 7f5de37fe700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de419c7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4003e60 tx=0x7f5dd4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.178+0000 7f5de0ff9700 1 -- 172.21.15.57:0/3707596690 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dd4003a60 con 0x7f5de407bd30 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.178+0000 7f5de0ff9700 1 -- 172.21.15.57:0/3707596690 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd4003bc0 con 0x7f5de407bd30 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.178+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de419cf20 con 0x7f5de407bd30 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.178+0000 7f5de0ff9700 1 -- 172.21.15.57:0/3707596690 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dd40196c0 con 0x7f5de407bd30 2023-10-06T22:06:12.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.178+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de419d340 con 0x7f5de407bd30 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.179+0000 7f5de0ff9700 1 -- 172.21.15.57:0/3707596690 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f5dd4019930 con 0x7f5de407bd30 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.179+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de40624e0 con 0x7f5de407bd30 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.180+0000 7f5de0ff9700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5dcc038940 0x7f5dcc03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.180+0000 7f5de0ff9700 1 -- 172.21.15.57:0/3707596690 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5dd404cd30 con 0x7f5de407bd30 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.180+0000 7f5de2ffd700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5dcc038940 0x7f5dcc03ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.181+0000 7f5de2ffd700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5dcc038940 0x7f5dcc03ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8006fd0 tx=0x7f5dd8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.185+0000 7f5de0ff9700 1 -- 172.21.15.57:0/3707596690 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5dd4018ce0 con 0x7f5de407bd30 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.392+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f5de419fd80 con 0x7f5de407bd30 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.392+0000 7f5de0ff9700 1 -- 172.21.15.57:0/3707596690 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f5dd40281c0 con 0x7f5de407bd30 2023-10-06T22:06:12.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5dcc038940 msgr2=0x7f5dcc03ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5dcc038940 0x7f5dcc03ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8006fd0 tx=0x7f5dd8006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 msgr2=0x7f5de419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de419c7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4003e60 tx=0x7f5dd4011a30 comp rx=0 tx=0).stop 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 shutdown_connections 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5dcc038940 0x7f5dcc03ae00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 --2- 172.21.15.57:0/3707596690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de407bd30 0x7f5de419c7e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.394+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 >> 172.21.15.57:0/3707596690 conn(0x7f5de4101e50 msgr2=0x7f5de4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.395+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 shutdown_connections 2023-10-06T22:06:12.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:12.395+0000 7f5de9ea0700 1 -- 172.21.15.57:0/3707596690 wait complete. 2023-10-06T22:06:14.111 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.348+0000 7f5f6895c700 1 Processor -- start 2023-10-06T22:06:14.111 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.348+0000 7f5f6895c700 1 -- start start 2023-10-06T22:06:14.111 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.348+0000 7f5f6895c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f64106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:14.111 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.348+0000 7f5f6895c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f64107280 con 0x7f5f64106890 2023-10-06T22:06:14.111 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.348+0000 7f5f6259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f64106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:14.111 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.349+0000 7f5f6259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f64106cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35422/0 (socket says 172.21.15.57:35422) 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.349+0000 7f5f6259c700 1 -- 172.21.15.57:0/1751873500 learned_addr learned my addr 172.21.15.57:0/1751873500 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.349+0000 7f5f6259c700 1 -- 172.21.15.57:0/1751873500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64107a90 con 0x7f5f64106890 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.349+0000 7f5f6259c700 1 --2- 172.21.15.57:0/1751873500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f64106cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f540098a0 tx=0x7f5f54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7420efb26fd65b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.349+0000 7f5f6159a700 1 -- 172.21.15.57:0/1751873500 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f54004160 con 0x7f5f64106890 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.349+0000 7f5f6159a700 1 -- 172.21.15.57:0/1751873500 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f540042c0 con 0x7f5f64106890 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.349+0000 7f5f6159a700 1 -- 172.21.15.57:0/1751873500 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f540119b0 con 0x7f5f64106890 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.350+0000 7f5f6895c700 1 -- 172.21.15.57:0/1751873500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 msgr2=0x7f5f64106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.350+0000 7f5f6895c700 1 --2- 172.21.15.57:0/1751873500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f64106cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f540098a0 tx=0x7f5f54009bb0 comp rx=0 tx=0).stop 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.350+0000 7f5f6895c700 1 -- 172.21.15.57:0/1751873500 shutdown_connections 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.350+0000 7f5f6895c700 1 --2- 172.21.15.57:0/1751873500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f64106cb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.350+0000 7f5f6895c700 1 -- 172.21.15.57:0/1751873500 >> 172.21.15.57:0/1751873500 conn(0x7f5f64101e50 msgr2=0x7f5f64104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:14.112 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.350+0000 7f5f6895c700 1 -- 172.21.15.57:0/1751873500 shutdown_connections 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.350+0000 7f5f6895c700 1 -- 172.21.15.57:0/1751873500 wait complete. 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.351+0000 7f5f6895c700 1 Processor -- start 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.351+0000 7f5f6895c700 1 -- start start 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.352+0000 7f5f6895c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f6419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.352+0000 7f5f6895c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6419cd20 con 0x7f5f64106890 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.352+0000 7f5f6259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f6419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.352+0000 7f5f6259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f6419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35434/0 (socket says 172.21.15.57:35434) 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.352+0000 7f5f6259c700 1 -- 172.21.15.57:0/1208851754 learned_addr learned my addr 172.21.15.57:0/1208851754 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.352+0000 7f5f6259c700 1 -- 172.21.15.57:0/1208851754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f54009550 con 0x7f5f64106890 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.352+0000 7f5f6259c700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f6419c7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f54003e60 tx=0x7f5f54011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.353+0000 7f5f537fe700 1 -- 172.21.15.57:0/1208851754 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f54003a60 con 0x7f5f64106890 2023-10-06T22:06:14.113 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.353+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6419cf20 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.353+0000 7f5f537fe700 1 -- 172.21.15.57:0/1208851754 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f54003bc0 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.353+0000 7f5f537fe700 1 -- 172.21.15.57:0/1208851754 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f540196c0 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.353+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6419d340 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.354+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f640624e0 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.354+0000 7f5f537fe700 1 -- 172.21.15.57:0/1208851754 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f5f5401ab60 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.355+0000 7f5f537fe700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5f4c0388f0 0x7f5f4c03adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.355+0000 7f5f537fe700 1 -- 172.21.15.57:0/1208851754 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5f5404cd80 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.355+0000 7f5f61d9b700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5f4c0388f0 0x7f5f4c03adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.356+0000 7f5f61d9b700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5f4c0388f0 0x7f5f4c03adb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f58006fd0 tx=0x7f5f58006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.360+0000 7f5f537fe700 1 -- 172.21.15.57:0/1208851754 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5f54010790 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.696+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f5f6419fd80 con 0x7f5f64106890 2023-10-06T22:06:14.114 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.700+0000 7f5f537fe700 1 -- 172.21.15.57:0/1208851754 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f5f54010380 con 0x7f5f64106890 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5f4c0388f0 msgr2=0x7f5f4c03adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5f4c0388f0 0x7f5f4c03adb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f58006fd0 tx=0x7f5f58006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 msgr2=0x7f5f6419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f6419c7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f54003e60 tx=0x7f5f54011a30 comp rx=0 tx=0).stop 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 shutdown_connections 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5f4c0388f0 0x7f5f4c03adb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 --2- 172.21.15.57:0/1208851754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f64106890 0x7f5f6419c7e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 >> 172.21.15.57:0/1208851754 conn(0x7f5f64101e50 msgr2=0x7f5f64103520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 shutdown_connections 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2023-10-06T22:06:13.703+0000 7f5f6895c700 1 -- 172.21.15.57:0/1208851754 wait complete. 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-06T22:06:14.115 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Or, if you are only running a single cluster on this host: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: ceph telemetry on 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:For more information see: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.116 INFO:teuthology.orchestra.run.smithi057.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-06T22:06:14.117 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Bootstrap complete. 2023-10-06T22:06:14.170 INFO:tasks.cephadm:Fetching config... 2023-10-06T22:06:14.170 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:06:14.170 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-06T22:06:14.213 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-06T22:06:14.213 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:06:14.213 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-06T22:06:14.302 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-06T22:06:14.303 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:06:14.303 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-06T22:06:14.373 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-06T22:06:14.374 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:06:14.374 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-06T22:06:14.457 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-06T22:06:14.457 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyI+7/QRB8oCobgUhQzSeETfpk8/490d/CUJyKdscgxAQR7GZ6xahRFrOPo80/LcwlLzVjOx0N0qQZ2EnfA4e4A6GI8aU1pEz6jae/SOBc+ocrroY5zKBAMZmuI8XaWI4PvBuBPGmlEGE8TAU4dBRi9i2anX4BRMU+jTPEgbpRQiP8/eDABCNq034Wfb0JPrbgC2J+DQw3aEGZrzimUB7yWxMJuUkyIBEFCICE7nq5NwJlSNNPrgaxShaAI4Uv3BaNp2ADTYEdf24Zt7dRM6TZmxwiz0vgql3TdrmGhCW+WLiPH02EjQoDz3R0VlIhecAn156JqOBcxDTeSIXXU4cHhf8KKSzUo6rho6BPRpSC4Pr/jcw6ljWzWqUHRG9uNqWvI0Y97rLi5wtFOY6pNwzPyE1dmisGs9MJgRXykGoqhvvfMAxSVLTKIyTctbqcrQJz9hF4KJHwnD4zI3s69MWKvmeJoD6XxJpX9E5pc8oY+fZ0A+JYKTij5PQJOKWITbU= ceph-495b2048-6494-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T22:06:14.609 INFO:teuthology.orchestra.run.smithi057.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyI+7/QRB8oCobgUhQzSeETfpk8/490d/CUJyKdscgxAQR7GZ6xahRFrOPo80/LcwlLzVjOx0N0qQZ2EnfA4e4A6GI8aU1pEz6jae/SOBc+ocrroY5zKBAMZmuI8XaWI4PvBuBPGmlEGE8TAU4dBRi9i2anX4BRMU+jTPEgbpRQiP8/eDABCNq034Wfb0JPrbgC2J+DQw3aEGZrzimUB7yWxMJuUkyIBEFCICE7nq5NwJlSNNPrgaxShaAI4Uv3BaNp2ADTYEdf24Zt7dRM6TZmxwiz0vgql3TdrmGhCW+WLiPH02EjQoDz3R0VlIhecAn156JqOBcxDTeSIXXU4cHhf8KKSzUo6rho6BPRpSC4Pr/jcw6ljWzWqUHRG9uNqWvI0Y97rLi5wtFOY6pNwzPyE1dmisGs9MJgRXykGoqhvvfMAxSVLTKIyTctbqcrQJz9hF4KJHwnD4zI3s69MWKvmeJoD6XxJpX9E5pc8oY+fZ0A+JYKTij5PQJOKWITbU= ceph-495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:06:14.639 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyI+7/QRB8oCobgUhQzSeETfpk8/490d/CUJyKdscgxAQR7GZ6xahRFrOPo80/LcwlLzVjOx0N0qQZ2EnfA4e4A6GI8aU1pEz6jae/SOBc+ocrroY5zKBAMZmuI8XaWI4PvBuBPGmlEGE8TAU4dBRi9i2anX4BRMU+jTPEgbpRQiP8/eDABCNq034Wfb0JPrbgC2J+DQw3aEGZrzimUB7yWxMJuUkyIBEFCICE7nq5NwJlSNNPrgaxShaAI4Uv3BaNp2ADTYEdf24Zt7dRM6TZmxwiz0vgql3TdrmGhCW+WLiPH02EjQoDz3R0VlIhecAn156JqOBcxDTeSIXXU4cHhf8KKSzUo6rho6BPRpSC4Pr/jcw6ljWzWqUHRG9uNqWvI0Y97rLi5wtFOY6pNwzPyE1dmisGs9MJgRXykGoqhvvfMAxSVLTKIyTctbqcrQJz9hF4KJHwnD4zI3s69MWKvmeJoD6XxJpX9E5pc8oY+fZ0A+JYKTij5PQJOKWITbU= ceph-495b2048-6494-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T22:06:14.684 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyI+7/QRB8oCobgUhQzSeETfpk8/490d/CUJyKdscgxAQR7GZ6xahRFrOPo80/LcwlLzVjOx0N0qQZ2EnfA4e4A6GI8aU1pEz6jae/SOBc+ocrroY5zKBAMZmuI8XaWI4PvBuBPGmlEGE8TAU4dBRi9i2anX4BRMU+jTPEgbpRQiP8/eDABCNq034Wfb0JPrbgC2J+DQw3aEGZrzimUB7yWxMJuUkyIBEFCICE7nq5NwJlSNNPrgaxShaAI4Uv3BaNp2ADTYEdf24Zt7dRM6TZmxwiz0vgql3TdrmGhCW+WLiPH02EjQoDz3R0VlIhecAn156JqOBcxDTeSIXXU4cHhf8KKSzUo6rho6BPRpSC4Pr/jcw6ljWzWqUHRG9uNqWvI0Y97rLi5wtFOY6pNwzPyE1dmisGs9MJgRXykGoqhvvfMAxSVLTKIyTctbqcrQJz9hF4KJHwnD4zI3s69MWKvmeJoD6XxJpX9E5pc8oY+fZ0A+JYKTij5PQJOKWITbU= ceph-495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:06:14.697 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyI+7/QRB8oCobgUhQzSeETfpk8/490d/CUJyKdscgxAQR7GZ6xahRFrOPo80/LcwlLzVjOx0N0qQZ2EnfA4e4A6GI8aU1pEz6jae/SOBc+ocrroY5zKBAMZmuI8XaWI4PvBuBPGmlEGE8TAU4dBRi9i2anX4BRMU+jTPEgbpRQiP8/eDABCNq034Wfb0JPrbgC2J+DQw3aEGZrzimUB7yWxMJuUkyIBEFCICE7nq5NwJlSNNPrgaxShaAI4Uv3BaNp2ADTYEdf24Zt7dRM6TZmxwiz0vgql3TdrmGhCW+WLiPH02EjQoDz3R0VlIhecAn156JqOBcxDTeSIXXU4cHhf8KKSzUo6rho6BPRpSC4Pr/jcw6ljWzWqUHRG9uNqWvI0Y97rLi5wtFOY6pNwzPyE1dmisGs9MJgRXykGoqhvvfMAxSVLTKIyTctbqcrQJz9hF4KJHwnD4zI3s69MWKvmeJoD6XxJpX9E5pc8oY+fZ0A+JYKTij5PQJOKWITbU= ceph-495b2048-6494-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T22:06:14.749 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyI+7/QRB8oCobgUhQzSeETfpk8/490d/CUJyKdscgxAQR7GZ6xahRFrOPo80/LcwlLzVjOx0N0qQZ2EnfA4e4A6GI8aU1pEz6jae/SOBc+ocrroY5zKBAMZmuI8XaWI4PvBuBPGmlEGE8TAU4dBRi9i2anX4BRMU+jTPEgbpRQiP8/eDABCNq034Wfb0JPrbgC2J+DQw3aEGZrzimUB7yWxMJuUkyIBEFCICE7nq5NwJlSNNPrgaxShaAI4Uv3BaNp2ADTYEdf24Zt7dRM6TZmxwiz0vgql3TdrmGhCW+WLiPH02EjQoDz3R0VlIhecAn156JqOBcxDTeSIXXU4cHhf8KKSzUo6rho6BPRpSC4Pr/jcw6ljWzWqUHRG9uNqWvI0Y97rLi5wtFOY6pNwzPyE1dmisGs9MJgRXykGoqhvvfMAxSVLTKIyTctbqcrQJz9hF4KJHwnD4zI3s69MWKvmeJoD6XxJpX9E5pc8oY+fZ0A+JYKTij5PQJOKWITbU= ceph-495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:06:14.762 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-06T22:06:14.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:14 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1208851754' entity='client.admin' 2023-10-06T22:06:15.510 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:06:17.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.301+0000 7fa5ec40d700 1 -- 172.21.15.57:0/1680801537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 msgr2=0x7fa5e4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:17.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.301+0000 7fa5ec40d700 1 --2- 172.21.15.57:0/1680801537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4073340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009910 tx=0x7fa5e0009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:17.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.301+0000 7fa5ec40d700 1 -- 172.21.15.57:0/1680801537 shutdown_connections 2023-10-06T22:06:17.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.301+0000 7fa5ec40d700 1 --2- 172.21.15.57:0/1680801537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4073340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:17.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.301+0000 7fa5ec40d700 1 -- 172.21.15.57:0/1680801537 >> 172.21.15.57:0/1680801537 conn(0x7fa5e40fbf80 msgr2=0x7fa5e40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:17.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.301+0000 7fa5ec40d700 1 -- 172.21.15.57:0/1680801537 shutdown_connections 2023-10-06T22:06:17.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.301+0000 7fa5ec40d700 1 -- 172.21.15.57:0/1680801537 wait complete. 2023-10-06T22:06:17.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.302+0000 7fa5ec40d700 1 Processor -- start 2023-10-06T22:06:17.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.302+0000 7fa5ec40d700 1 -- start start 2023-10-06T22:06:17.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.303+0000 7fa5ec40d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:17.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.303+0000 7fa5ec40d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4198280 con 0x7fa5e4074ee0 2023-10-06T22:06:17.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.303+0000 7fa5ea1a9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:17.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.303+0000 7fa5ea1a9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35456/0 (socket says 172.21.15.57:35456) 2023-10-06T22:06:17.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.303+0000 7fa5ea1a9700 1 -- 172.21.15.57:0/3005578214 learned_addr learned my addr 172.21.15.57:0/3005578214 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:17.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.303+0000 7fa5ea1a9700 1 -- 172.21.15.57:0/3005578214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e00095f0 con 0x7fa5e4074ee0 2023-10-06T22:06:17.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.304+0000 7fa5ea1a9700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4197d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0004f40 tx=0x7fa5e00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:17.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.304+0000 7fa5db7fe700 1 -- 172.21.15.57:0/3005578214 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5e001d070 con 0x7fa5e4074ee0 2023-10-06T22:06:17.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.304+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e4198480 con 0x7fa5e4074ee0 2023-10-06T22:06:17.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.304+0000 7fa5db7fe700 1 -- 172.21.15.57:0/3005578214 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5e000bc10 con 0x7fa5e4074ee0 2023-10-06T22:06:17.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.304+0000 7fa5db7fe700 1 -- 172.21.15.57:0/3005578214 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5e0003a70 con 0x7fa5e4074ee0 2023-10-06T22:06:17.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.304+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e4198920 con 0x7fa5e4074ee0 2023-10-06T22:06:17.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.305+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e4191720 con 0x7fa5e4074ee0 2023-10-06T22:06:17.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.305+0000 7fa5db7fe700 1 -- 172.21.15.57:0/3005578214 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fa5e0003ce0 con 0x7fa5e4074ee0 2023-10-06T22:06:17.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.306+0000 7fa5db7fe700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa5d0038910 0x7fa5d003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:17.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.306+0000 7fa5db7fe700 1 -- 172.21.15.57:0/3005578214 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa5e004c6a0 con 0x7fa5e4074ee0 2023-10-06T22:06:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.306+0000 7fa5e99a8700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa5d0038910 0x7fa5d003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:17.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.307+0000 7fa5e99a8700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa5d0038910 0x7fa5d003add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006fd0 tx=0x7fa5d4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:17.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.311+0000 7fa5db7fe700 1 -- 172.21.15.57:0/3005578214 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa5e002aba0 con 0x7fa5e4074ee0 2023-10-06T22:06:17.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.524+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fa5e4108960 con 0x7fa5e4074ee0 2023-10-06T22:06:17.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.527+0000 7fa5db7fe700 1 -- 172.21.15.57:0/3005578214 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa5e0026070 con 0x7fa5e4074ee0 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa5d0038910 msgr2=0x7fa5d003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa5d0038910 0x7fa5d003add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006fd0 tx=0x7fa5d4006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 msgr2=0x7fa5e4197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4197d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0004f40 tx=0x7fa5e00057c0 comp rx=0 tx=0).stop 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 shutdown_connections 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa5d0038910 0x7fa5d003add0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 --2- 172.21.15.57:0/3005578214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e4074ee0 0x7fa5e4197d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 >> 172.21.15.57:0/3005578214 conn(0x7fa5e40fbf80 msgr2=0x7fa5e40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 shutdown_connections 2023-10-06T22:06:17.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:17.530+0000 7fa5ec40d700 1 -- 172.21.15.57:0/3005578214 wait complete. 2023-10-06T22:06:17.991 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-06T22:06:17.991 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-06T22:06:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:06:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:18.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:18.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3005578214' entity='client.admin' 2023-10-06T22:06:18.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:18.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:18.641 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.478+0000 7fe4d55b9700 1 -- 172.21.15.57:0/3250024506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 msgr2=0x7fe4d0100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.478+0000 7fe4d55b9700 1 --2- 172.21.15.57:0/3250024506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0100e20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009910 tx=0x7fe4c0009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.478+0000 7fe4d55b9700 1 -- 172.21.15.57:0/3250024506 shutdown_connections 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.478+0000 7fe4d55b9700 1 --2- 172.21.15.57:0/3250024506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0100e20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.478+0000 7fe4d55b9700 1 -- 172.21.15.57:0/3250024506 >> 172.21.15.57:0/3250024506 conn(0x7fe4d00fbf80 msgr2=0x7fe4d00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.478+0000 7fe4d55b9700 1 -- 172.21.15.57:0/3250024506 shutdown_connections 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.478+0000 7fe4d55b9700 1 -- 172.21.15.57:0/3250024506 wait complete. 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.479+0000 7fe4d55b9700 1 Processor -- start 2023-10-06T22:06:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.479+0000 7fe4d55b9700 1 -- start start 2023-10-06T22:06:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.479+0000 7fe4d55b9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.479+0000 7fe4d55b9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0198260 con 0x7fe4d0100a00 2023-10-06T22:06:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.480+0000 7fe4ceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.480+0000 7fe4ceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35482/0 (socket says 172.21.15.57:35482) 2023-10-06T22:06:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.480+0000 7fe4ceffd700 1 -- 172.21.15.57:0/4242273234 learned_addr learned my addr 172.21.15.57:0/4242273234 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.480+0000 7fe4ceffd700 1 -- 172.21.15.57:0/4242273234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00095f0 con 0x7fe4d0100a00 2023-10-06T22:06:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.480+0000 7fe4ceffd700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0197d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006010 tx=0x7fe4c00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:20.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.481+0000 7fe4b7fff700 1 -- 172.21.15.57:0/4242273234 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4c001d070 con 0x7fe4d0100a00 2023-10-06T22:06:20.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.481+0000 7fe4b7fff700 1 -- 172.21.15.57:0/4242273234 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c00044e0 con 0x7fe4d0100a00 2023-10-06T22:06:20.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.481+0000 7fe4b7fff700 1 -- 172.21.15.57:0/4242273234 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4c00039b0 con 0x7fe4d0100a00 2023-10-06T22:06:20.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.481+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d0198460 con 0x7fe4d0100a00 2023-10-06T22:06:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.481+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d01988a0 con 0x7fe4d0100a00 2023-10-06T22:06:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.482+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d0191910 con 0x7fe4d0100a00 2023-10-06T22:06:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.482+0000 7fe4b7fff700 1 -- 172.21.15.57:0/4242273234 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fe4c000baf0 con 0x7fe4d0100a00 2023-10-06T22:06:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.483+0000 7fe4b7fff700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe4b8038910 0x7fe4b803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.483+0000 7fe4b7fff700 1 -- 172.21.15.57:0/4242273234 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe4c004c980 con 0x7fe4d0100a00 2023-10-06T22:06:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.483+0000 7fe4ce7fc700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe4b8038910 0x7fe4b803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.483+0000 7fe4ce7fc700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe4b8038910 0x7fe4b803add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006fd0 tx=0x7fe4c4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:20.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.487+0000 7fe4b7fff700 1 -- 172.21.15.57:0/4242273234 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4c002a950 con 0x7fe4d0100a00 2023-10-06T22:06:20.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.701+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4d002d170 con 0x7fe4b8038910 2023-10-06T22:06:20.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.705+0000 7fe4b7fff700 1 -- 172.21.15.57:0/4242273234 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe4d002d170 con 0x7fe4b8038910 2023-10-06T22:06:20.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe4b8038910 msgr2=0x7fe4b803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:20.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe4b8038910 0x7fe4b803add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006fd0 tx=0x7fe4c4006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:20.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 msgr2=0x7fe4d0197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:20.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0197d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006010 tx=0x7fe4c00057c0 comp rx=0 tx=0).stop 2023-10-06T22:06:20.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 shutdown_connections 2023-10-06T22:06:20.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe4b8038910 0x7fe4b803add0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:20.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 --2- 172.21.15.57:0/4242273234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d0100a00 0x7fe4d0197d20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:20.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 >> 172.21.15.57:0/4242273234 conn(0x7fe4d00fbf80 msgr2=0x7fe4d00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:20.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 shutdown_connections 2023-10-06T22:06:20.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:20.707+0000 7fe4d55b9700 1 -- 172.21.15.57:0/4242273234 wait complete. 2023-10-06T22:06:21.124 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2023-10-06T22:06:21.124 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:06:21.124 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2023-10-06T22:06:21.146 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:06:21.146 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:21.205 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2023-10-06T22:06:21.205 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi156 2023-10-06T22:06:21.960 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:06:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:21 smithi057 ceph-mon[98492]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:21 smithi057 ceph-mon[98492]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-06T22:06:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:21 smithi057 ceph-mon[98492]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:22.798 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:22 smithi057 ceph-mon[98492]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:22.798 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:22 smithi057 ceph-mon[98492]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:06:23.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.441+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/127057975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 msgr2=0x7f4cd0106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:23.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.441+0000 7f4cd5d7d700 1 --2- 172.21.15.57:0/127057975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0106d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009910 tx=0x7f4cc0009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:23.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.441+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/127057975 shutdown_connections 2023-10-06T22:06:23.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.441+0000 7f4cd5d7d700 1 --2- 172.21.15.57:0/127057975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0106d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:23.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.441+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/127057975 >> 172.21.15.57:0/127057975 conn(0x7f4cd00fc4a0 msgr2=0x7f4cd00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:23.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.441+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/127057975 shutdown_connections 2023-10-06T22:06:23.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.441+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/127057975 wait complete. 2023-10-06T22:06:23.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.442+0000 7f4cd5d7d700 1 Processor -- start 2023-10-06T22:06:23.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.442+0000 7f4cd5d7d700 1 -- start start 2023-10-06T22:06:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.442+0000 7f4cd5d7d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.442+0000 7f4cd5d7d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0198570 con 0x7f4cd0106970 2023-10-06T22:06:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.443+0000 7f4ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.443+0000 7f4ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42376/0 (socket says 172.21.15.57:42376) 2023-10-06T22:06:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.443+0000 7f4ccf7fe700 1 -- 172.21.15.57:0/2719573264 learned_addr learned my addr 172.21.15.57:0/2719573264 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.443+0000 7f4ccf7fe700 1 -- 172.21.15.57:0/2719573264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00095f0 con 0x7f4cd0106970 2023-10-06T22:06:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.443+0000 7f4ccf7fe700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0198030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0006010 tx=0x7f4cc0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.444+0000 7f4cccff9700 1 -- 172.21.15.57:0/2719573264 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc001d070 con 0x7f4cd0106970 2023-10-06T22:06:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.444+0000 7f4cccff9700 1 -- 172.21.15.57:0/2719573264 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc000bbb0 con 0x7f4cd0106970 2023-10-06T22:06:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.444+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd01987d0 con 0x7f4cd0106970 2023-10-06T22:06:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.444+0000 7f4cccff9700 1 -- 172.21.15.57:0/2719573264 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc0003930 con 0x7f4cd0106970 2023-10-06T22:06:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.444+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd0198c50 con 0x7f4cd0106970 2023-10-06T22:06:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.445+0000 7f4cccff9700 1 -- 172.21.15.57:0/2719573264 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f4cc000bd20 con 0x7f4cd0106970 2023-10-06T22:06:23.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.445+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd00624e0 con 0x7f4cd0106970 2023-10-06T22:06:23.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.445+0000 7f4cccff9700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4cb8038910 0x7f4cb803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:23.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.445+0000 7f4cccff9700 1 -- 172.21.15.57:0/2719573264 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4cc004c6f0 con 0x7f4cd0106970 2023-10-06T22:06:23.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.446+0000 7f4cceffd700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4cb8038910 0x7f4cb803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:23.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.446+0000 7f4cceffd700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4cb8038910 0x7f4cb803add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4006fd0 tx=0x7f4cc4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:23.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.451+0000 7f4cccff9700 1 -- 172.21.15.57:0/2719573264 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4cc0029360 con 0x7f4cd0106970 2023-10-06T22:06:23.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:23.665+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cd00008d0 con 0x7f4cb8038910 2023-10-06T22:06:24.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:24.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:24.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:24.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:23 smithi057 ceph-mon[98492]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:24 smithi057 ceph-mon[98492]: Deploying cephadm binary to smithi156 2023-10-06T22:06:26.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.184+0000 7f4cccff9700 1 -- 172.21.15.57:0/2719573264 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4cd00008d0 con 0x7f4cb8038910 2023-10-06T22:06:26.187 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi156' with addr '172.21.15.156' 2023-10-06T22:06:26.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4cb8038910 msgr2=0x7f4cb803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4cb8038910 0x7f4cb803add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4006fd0 tx=0x7f4cc4006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 msgr2=0x7f4cd0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0198030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0006010 tx=0x7f4cc0004ff0 comp rx=0 tx=0).stop 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 shutdown_connections 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4cb8038910 0x7f4cb803add0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 --2- 172.21.15.57:0/2719573264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4cd0106970 0x7f4cd0198030 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 >> 172.21.15.57:0/2719573264 conn(0x7f4cd00fc4a0 msgr2=0x7f4cd0108da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 shutdown_connections 2023-10-06T22:06:26.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:26.187+0000 7f4cd5d7d700 1 -- 172.21.15.57:0/2719573264 wait complete. 2023-10-06T22:06:26.654 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-06T22:06:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:26 smithi057 ceph-mon[98492]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:27.390 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:06:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:27 smithi057 ceph-mon[98492]: Added host smithi156 2023-10-06T22:06:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:27 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:27.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:27 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:28.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.847+0000 7f1a7d178700 1 -- 172.21.15.57:0/2535487476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 msgr2=0x7f1a78106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:28.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.847+0000 7f1a7d178700 1 --2- 172.21.15.57:0/2535487476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78106d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009910 tx=0x7f1a68009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:28.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.848+0000 7f1a7d178700 1 -- 172.21.15.57:0/2535487476 shutdown_connections 2023-10-06T22:06:28.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.848+0000 7f1a7d178700 1 --2- 172.21.15.57:0/2535487476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78106d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:28.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.848+0000 7f1a7d178700 1 -- 172.21.15.57:0/2535487476 >> 172.21.15.57:0/2535487476 conn(0x7f1a780fc4a0 msgr2=0x7f1a780fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:28.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.848+0000 7f1a7d178700 1 -- 172.21.15.57:0/2535487476 shutdown_connections 2023-10-06T22:06:28.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.848+0000 7f1a7d178700 1 -- 172.21.15.57:0/2535487476 wait complete. 2023-10-06T22:06:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.849+0000 7f1a7d178700 1 Processor -- start 2023-10-06T22:06:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.849+0000 7f1a7d178700 1 -- start start 2023-10-06T22:06:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.849+0000 7f1a7d178700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.849+0000 7f1a7d178700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a78196380 con 0x7f1a78106970 2023-10-06T22:06:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.849+0000 7f1a76d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a76d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42384/0 (socket says 172.21.15.57:42384) 2023-10-06T22:06:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a76d9d700 1 -- 172.21.15.57:0/1483523898 learned_addr learned my addr 172.21.15.57:0/1483523898 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:28.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a76d9d700 1 -- 172.21.15.57:0/1483523898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680095f0 con 0x7f1a78106970 2023-10-06T22:06:28.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a76d9d700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78195e40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1a68004f40 tx=0x7f1a68005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:28.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a5ffff700 1 -- 172.21.15.57:0/1483523898 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a6801d070 con 0x7f1a78106970 2023-10-06T22:06:28.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a781965e0 con 0x7f1a78106970 2023-10-06T22:06:28.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a5ffff700 1 -- 172.21.15.57:0/1483523898 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a68004370 con 0x7f1a78106970 2023-10-06T22:06:28.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.850+0000 7f1a5ffff700 1 -- 172.21.15.57:0/1483523898 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a68003970 con 0x7f1a78106970 2023-10-06T22:06:28.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.851+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a78196a60 con 0x7f1a78106970 2023-10-06T22:06:28.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.851+0000 7f1a5ffff700 1 -- 172.21.15.57:0/1483523898 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f1a68003be0 con 0x7f1a78106970 2023-10-06T22:06:28.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.852+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a780624e0 con 0x7f1a78106970 2023-10-06T22:06:28.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.852+0000 7f1a5ffff700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a60038910 0x7f1a6003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:28.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.852+0000 7f1a5ffff700 1 -- 172.21.15.57:0/1483523898 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1a6804c6e0 con 0x7f1a78106970 2023-10-06T22:06:28.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.852+0000 7f1a7659c700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a60038910 0x7f1a6003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:28.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.853+0000 7f1a7659c700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a60038910 0x7f1a6003add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c006fd0 tx=0x7f1a6c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:28.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:28.857+0000 7f1a5ffff700 1 -- 172.21.15.57:0/1483523898 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1a6802a590 con 0x7f1a78106970 2023-10-06T22:06:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:28 smithi057 ceph-mon[98492]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:29.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.071+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a780008d0 con 0x7f1a60038910 2023-10-06T22:06:29.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.072+0000 7f1a5ffff700 1 -- 172.21.15.57:0/1483523898 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f1a780008d0 con 0x7f1a60038910 2023-10-06T22:06:29.089 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:29.089 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2023-10-06T22:06:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a60038910 msgr2=0x7f1a6003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a60038910 0x7f1a6003add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c006fd0 tx=0x7f1a6c006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 msgr2=0x7f1a78195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78195e40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1a68004f40 tx=0x7f1a68005020 comp rx=0 tx=0).stop 2023-10-06T22:06:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 shutdown_connections 2023-10-06T22:06:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a60038910 0x7f1a6003add0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 --2- 172.21.15.57:0/1483523898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a78106970 0x7f1a78195e40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 >> 172.21.15.57:0/1483523898 conn(0x7f1a780fc4a0 msgr2=0x7f1a78108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 shutdown_connections 2023-10-06T22:06:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:29.075+0000 7f1a7d178700 1 -- 172.21.15.57:0/1483523898 wait complete. 2023-10-06T22:06:29.566 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2023-10-06T22:06:29.566 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:06:29.566 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2023-10-06T22:06:29.587 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:06:29.587 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:29.646 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2023-10-06T22:06:29.646 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi190 2023-10-06T22:06:30.379 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:06:30.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:30 smithi057 ceph-mon[98492]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:06:30.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:30 smithi057 ceph-mon[98492]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.866+0000 7fa442383700 1 -- 172.21.15.57:0/2773926364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 msgr2=0x7fa43c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.866+0000 7fa442383700 1 --2- 172.21.15.57:0/2773926364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1031c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009910 tx=0x7fa42c009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.867+0000 7fa442383700 1 -- 172.21.15.57:0/2773926364 shutdown_connections 2023-10-06T22:06:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.867+0000 7fa442383700 1 --2- 172.21.15.57:0/2773926364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1031c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.867+0000 7fa442383700 1 -- 172.21.15.57:0/2773926364 >> 172.21.15.57:0/2773926364 conn(0x7fa43c0fe670 msgr2=0x7fa43c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.867+0000 7fa442383700 1 -- 172.21.15.57:0/2773926364 shutdown_connections 2023-10-06T22:06:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.867+0000 7fa442383700 1 -- 172.21.15.57:0/2773926364 wait complete. 2023-10-06T22:06:31.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.868+0000 7fa442383700 1 Processor -- start 2023-10-06T22:06:31.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.868+0000 7fa442383700 1 -- start start 2023-10-06T22:06:31.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.868+0000 7fa442383700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:31.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.868+0000 7fa442383700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c1985f0 con 0x7fa43c102de0 2023-10-06T22:06:31.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa43bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:31.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa43bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33820/0 (socket says 172.21.15.57:33820) 2023-10-06T22:06:31.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa43bfff700 1 -- 172.21.15.57:0/1034840344 learned_addr learned my addr 172.21.15.57:0/1034840344 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:31.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa43bfff700 1 -- 172.21.15.57:0/1034840344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c0095f0 con 0x7fa43c102de0 2023-10-06T22:06:31.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa43bfff700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1980b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa42c006010 tx=0x7fa42c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:31.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa4397fa700 1 -- 172.21.15.57:0/1034840344 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa42c01d070 con 0x7fa43c102de0 2023-10-06T22:06:31.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa4397fa700 1 -- 172.21.15.57:0/1034840344 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa42c00bc10 con 0x7fa43c102de0 2023-10-06T22:06:31.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.869+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c1987f0 con 0x7fa43c102de0 2023-10-06T22:06:31.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.870+0000 7fa4397fa700 1 -- 172.21.15.57:0/1034840344 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa42c003910 con 0x7fa43c102de0 2023-10-06T22:06:31.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.870+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c198c10 con 0x7fa43c102de0 2023-10-06T22:06:31.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.871+0000 7fa4397fa700 1 -- 172.21.15.57:0/1034840344 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fa42c00bd80 con 0x7fa43c102de0 2023-10-06T22:06:31.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.871+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c0624e0 con 0x7fa43c102de0 2023-10-06T22:06:31.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.871+0000 7fa4397fa700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa4240388c0 0x7fa42403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:31.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.871+0000 7fa4397fa700 1 -- 172.21.15.57:0/1034840344 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa42c04c6f0 con 0x7fa43c102de0 2023-10-06T22:06:31.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.871+0000 7fa43b7fe700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa4240388c0 0x7fa42403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:31.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.872+0000 7fa43b7fe700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa4240388c0 0x7fa42403ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa430006fd0 tx=0x7fa430006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:31.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:31.877+0000 7fa4397fa700 1 -- 172.21.15.57:0/1034840344 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa42c029360 con 0x7fa43c102de0 2023-10-06T22:06:32.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:32.090+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}) v1 -- 0x7fa43c10d010 con 0x7fa4240388c0 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:32.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:06:32.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:32.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:32.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:32.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:32.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:33 smithi057 ceph-mon[98492]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:33 smithi057 ceph-mon[98492]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:06:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:33 smithi057 ceph-mon[98492]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:33 smithi057 ceph-mon[98492]: Deploying cephadm binary to smithi190 2023-10-06T22:06:34.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.562+0000 7fa4397fa700 1 -- 172.21.15.57:0/1034840344 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa43c10d010 con 0x7fa4240388c0 2023-10-06T22:06:34.565 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi190' with addr '172.21.15.190' 2023-10-06T22:06:34.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa4240388c0 msgr2=0x7fa42403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:34.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa4240388c0 0x7fa42403ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa430006fd0 tx=0x7fa430006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:34.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 msgr2=0x7fa43c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:34.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1980b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa42c006010 tx=0x7fa42c004ff0 comp rx=0 tx=0).stop 2023-10-06T22:06:34.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 shutdown_connections 2023-10-06T22:06:34.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa4240388c0 0x7fa42403ad80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:34.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 --2- 172.21.15.57:0/1034840344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa43c102de0 0x7fa43c1980b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:34.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 >> 172.21.15.57:0/1034840344 conn(0x7fa43c0fe670 msgr2=0x7fa43c108940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:34.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 shutdown_connections 2023-10-06T22:06:34.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:34.565+0000 7fa442383700 1 -- 172.21.15.57:0/1034840344 wait complete. 2023-10-06T22:06:34.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:34 smithi057 ceph-mon[98492]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:35.068 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-06T22:06:35.800 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:06:35.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:35 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:35.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:35 smithi057 ceph-mon[98492]: Added host smithi190 2023-10-06T22:06:35.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:35 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:35.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:35 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:36.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:36 smithi057 ceph-mon[98492]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:36.661 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:36.661 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.311+0000 7f1dd66d2700 1 -- 172.21.15.57:0/2306296790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 msgr2=0x7f1dd0106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.311+0000 7f1dd66d2700 1 --2- 172.21.15.57:0/2306296790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd0106d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009910 tx=0x7f1dc0009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.311+0000 7f1dd66d2700 1 -- 172.21.15.57:0/2306296790 shutdown_connections 2023-10-06T22:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.311+0000 7f1dd66d2700 1 --2- 172.21.15.57:0/2306296790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd0106d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.311+0000 7f1dd66d2700 1 -- 172.21.15.57:0/2306296790 >> 172.21.15.57:0/2306296790 conn(0x7f1dd00fc4a0 msgr2=0x7f1dd00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.311+0000 7f1dd66d2700 1 -- 172.21.15.57:0/2306296790 shutdown_connections 2023-10-06T22:06:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.311+0000 7f1dd66d2700 1 -- 172.21.15.57:0/2306296790 wait complete. 2023-10-06T22:06:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.313+0000 7f1dd66d2700 1 Processor -- start 2023-10-06T22:06:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.313+0000 7f1dd66d2700 1 -- start start 2023-10-06T22:06:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.314+0000 7f1dd66d2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd00731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.314+0000 7f1dd66d2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd0075130 con 0x7f1dd0106970 2023-10-06T22:06:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.314+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd00731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.314+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd00731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33834/0 (socket says 172.21.15.57:33834) 2023-10-06T22:06:37.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.314+0000 7f1dcffff700 1 -- 172.21.15.57:0/1184743017 learned_addr learned my addr 172.21.15.57:0/1184743017 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:37.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.314+0000 7f1dcffff700 1 -- 172.21.15.57:0/1184743017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00095f0 con 0x7f1dd0106970 2023-10-06T22:06:37.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.314+0000 7f1dcffff700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd00731b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:37.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.315+0000 7f1dcd7fa700 1 -- 172.21.15.57:0/1184743017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc001d070 con 0x7f1dd0106970 2023-10-06T22:06:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.315+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd00736f0 con 0x7f1dd0106970 2023-10-06T22:06:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.315+0000 7f1dcd7fa700 1 -- 172.21.15.57:0/1184743017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc000bc10 con 0x7f1dd0106970 2023-10-06T22:06:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.315+0000 7f1dcd7fa700 1 -- 172.21.15.57:0/1184743017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc0003910 con 0x7f1dd0106970 2023-10-06T22:06:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.315+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd0073b70 con 0x7f1dd0106970 2023-10-06T22:06:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.316+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd0104c20 con 0x7f1dd0106970 2023-10-06T22:06:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.316+0000 7f1dcd7fa700 1 -- 172.21.15.57:0/1184743017 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f1dc000bd80 con 0x7f1dd0106970 2023-10-06T22:06:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.317+0000 7f1dcd7fa700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1db80388c0 0x7f1db803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.317+0000 7f1dcd7fa700 1 -- 172.21.15.57:0/1184743017 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1dc004c740 con 0x7f1dd0106970 2023-10-06T22:06:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.317+0000 7f1dcf7fe700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1db80388c0 0x7f1db803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.318+0000 7f1dcf7fe700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1db80388c0 0x7f1db803ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4006fd0 tx=0x7f1dc4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:37.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.322+0000 7f1dcd7fa700 1 -- 172.21.15.57:0/1184743017 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1dc0029360 con 0x7f1dd0106970 2023-10-06T22:06:37.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.537+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1dd0073e70 con 0x7f1db80388c0 2023-10-06T22:06:37.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.538+0000 7f1dcd7fa700 1 -- 172.21.15.57:0/1184743017 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f1dd0073e70 con 0x7f1db80388c0 2023-10-06T22:06:37.541 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:06:37.541 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2023-10-06T22:06:37.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1db80388c0 msgr2=0x7f1db803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1db80388c0 0x7f1db803ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4006fd0 tx=0x7f1dc4006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 msgr2=0x7f1dd00731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd00731b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc0004ff0 comp rx=0 tx=0).stop 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 shutdown_connections 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1db80388c0 0x7f1db803ad80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 --2- 172.21.15.57:0/1184743017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dd0106970 0x7f1dd00731b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 >> 172.21.15.57:0/1184743017 conn(0x7f1dd00fc4a0 msgr2=0x7f1dd0108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 shutdown_connections 2023-10-06T22:06:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:37.541+0000 7f1dd66d2700 1 -- 172.21.15.57:0/1184743017 wait complete. 2023-10-06T22:06:37.977 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-06T22:06:37.977 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-06T22:06:38.711 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:06:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:38 smithi057 ceph-mon[98492]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:06:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:38 smithi057 ceph-mon[98492]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:40.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.205+0000 7efdc6afb700 1 -- 172.21.15.57:0/906547353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 msgr2=0x7efdc00757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:40.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.205+0000 7efdc6afb700 1 --2- 172.21.15.57:0/906547353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc00757c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009910 tx=0x7efdb4009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:40.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.205+0000 7efdc6afb700 1 -- 172.21.15.57:0/906547353 shutdown_connections 2023-10-06T22:06:40.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.205+0000 7efdc6afb700 1 --2- 172.21.15.57:0/906547353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc00757c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:40.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.205+0000 7efdc6afb700 1 -- 172.21.15.57:0/906547353 >> 172.21.15.57:0/906547353 conn(0x7efdc00fe670 msgr2=0x7efdc0100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:40.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.205+0000 7efdc6afb700 1 -- 172.21.15.57:0/906547353 shutdown_connections 2023-10-06T22:06:40.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.205+0000 7efdc6afb700 1 -- 172.21.15.57:0/906547353 wait complete. 2023-10-06T22:06:40.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.206+0000 7efdc6afb700 1 Processor -- start 2023-10-06T22:06:40.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.206+0000 7efdc6afb700 1 -- start start 2023-10-06T22:06:40.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.207+0000 7efdc6afb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:40.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.207+0000 7efdc6afb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc019c960 con 0x7efdc00753e0 2023-10-06T22:06:40.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.207+0000 7efdc4897700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:40.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.207+0000 7efdc4897700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33858/0 (socket says 172.21.15.57:33858) 2023-10-06T22:06:40.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.207+0000 7efdc4897700 1 -- 172.21.15.57:0/3037700852 learned_addr learned my addr 172.21.15.57:0/3037700852 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:06:40.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.207+0000 7efdc4897700 1 -- 172.21.15.57:0/3037700852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40095f0 con 0x7efdc00753e0 2023-10-06T22:06:40.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.208+0000 7efdc4897700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc019c420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006010 tx=0x7efdb4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:40.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.208+0000 7efdbdffb700 1 -- 172.21.15.57:0/3037700852 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdb401d070 con 0x7efdc00753e0 2023-10-06T22:06:40.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.208+0000 7efdbdffb700 1 -- 172.21.15.57:0/3037700852 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb400bc10 con 0x7efdc00753e0 2023-10-06T22:06:40.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.208+0000 7efdbdffb700 1 -- 172.21.15.57:0/3037700852 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdb4003910 con 0x7efdc00753e0 2023-10-06T22:06:40.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.208+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc019cb60 con 0x7efdc00753e0 2023-10-06T22:06:40.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.208+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc019cf80 con 0x7efdc00753e0 2023-10-06T22:06:40.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.209+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc00624e0 con 0x7efdc00753e0 2023-10-06T22:06:40.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.210+0000 7efdbdffb700 1 -- 172.21.15.57:0/3037700852 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7efdb400bd80 con 0x7efdc00753e0 2023-10-06T22:06:40.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.210+0000 7efdbdffb700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdb0038910 0x7efdb003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:40.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.210+0000 7efdbdffb700 1 -- 172.21.15.57:0/3037700852 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efdb404c680 con 0x7efdc00753e0 2023-10-06T22:06:40.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.210+0000 7efdbffff700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdb0038910 0x7efdb003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:40.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.211+0000 7efdbffff700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdb0038910 0x7efdb003add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efdac006fd0 tx=0x7efdac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:40.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.215+0000 7efdbdffb700 1 -- 172.21.15.57:0/3037700852 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efdb4029360 con 0x7efdc00753e0 2023-10-06T22:06:40.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:40.450+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7efdc01113b0 con 0x7efdc00753e0 2023-10-06T22:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:40 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3037700852' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T22:06:41.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.428+0000 7efdbdffb700 1 -- 172.21.15.57:0/3037700852 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7efdb4026020 con 0x7efdc00753e0 2023-10-06T22:06:41.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.431+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdb0038910 msgr2=0x7efdb003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:41.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.431+0000 7efdc6afb700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdb0038910 0x7efdb003add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efdac006fd0 tx=0x7efdac006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 msgr2=0x7efdc019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc019c420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006010 tx=0x7efdb4004ff0 comp rx=0 tx=0).stop 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 shutdown_connections 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdb0038910 0x7efdb003add0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 --2- 172.21.15.57:0/3037700852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdc00753e0 0x7efdc019c420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 >> 172.21.15.57:0/3037700852 conn(0x7efdc00fe670 msgr2=0x7efdc00ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 shutdown_connections 2023-10-06T22:06:41.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:06:41.432+0000 7efdc6afb700 1 -- 172.21.15.57:0/3037700852 wait complete. 2023-10-06T22:06:41.436 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2023-10-06T22:06:41.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:41 smithi057 ceph-mon[98492]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:41.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:41 smithi057 ceph-mon[98492]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:41.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:41 smithi057 ceph-mon[98492]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:06:41.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:41 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3037700852' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T22:06:41.830 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:41 smithi057 ceph-mon[98492]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:06:41.896 INFO:tasks.cephadm:Adding mon.a on smithi057 2023-10-06T22:06:41.897 INFO:tasks.cephadm:Adding mon.b on smithi156 2023-10-06T22:06:41.897 INFO:tasks.cephadm:Adding mon.c on smithi190 2023-10-06T22:06:41.897 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c' 2023-10-06T22:06:42.488 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:42 smithi057 ceph-mon[98492]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:43.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.692+0000 7fbd04c4b700 1 -- 172.21.15.190:0/886220091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 msgr2=0x7fbd00106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:43.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.692+0000 7fbd04c4b700 1 --2- 172.21.15.190:0/886220091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00106d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009910 tx=0x7fbcf0009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:43.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.692+0000 7fbd04c4b700 1 -- 172.21.15.190:0/886220091 shutdown_connections 2023-10-06T22:06:43.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.692+0000 7fbd04c4b700 1 --2- 172.21.15.190:0/886220091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00106d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:43.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.692+0000 7fbd04c4b700 1 -- 172.21.15.190:0/886220091 >> 172.21.15.190:0/886220091 conn(0x7fbd000fc470 msgr2=0x7fbd000fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:43.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.692+0000 7fbd04c4b700 1 -- 172.21.15.190:0/886220091 shutdown_connections 2023-10-06T22:06:43.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.692+0000 7fbd04c4b700 1 -- 172.21.15.190:0/886220091 wait complete. 2023-10-06T22:06:43.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.693+0000 7fbd04c4b700 1 Processor -- start 2023-10-06T22:06:43.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.693+0000 7fbd04c4b700 1 -- start start 2023-10-06T22:06:43.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.693+0000 7fbd04c4b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:43.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.693+0000 7fbd04c4b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00196350 con 0x7fbd00106960 2023-10-06T22:06:43.697 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.694+0000 7fbcfe59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.694+0000 7fbcfe59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.190:40370/0 (socket says 172.21.15.190:40370) 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.694+0000 7fbcfe59c700 1 -- 172.21.15.190:0/1851420252 learned_addr learned my addr 172.21.15.190:0/1851420252 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.694+0000 7fbcfe59c700 1 -- 172.21.15.190:0/1851420252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00095f0 con 0x7fbd00106960 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.694+0000 7fbcfe59c700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00195e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0004f40 tx=0x7fbcf0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.695+0000 7fbcef7fe700 1 -- 172.21.15.190:0/1851420252 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcf001d070 con 0x7fbd00106960 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.695+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd001965b0 con 0x7fbd00106960 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.695+0000 7fbcef7fe700 1 -- 172.21.15.190:0/1851420252 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcf0004370 con 0x7fbd00106960 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.695+0000 7fbcef7fe700 1 -- 172.21.15.190:0/1851420252 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcf0003970 con 0x7fbd00106960 2023-10-06T22:06:43.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.695+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd00196a30 con 0x7fbd00106960 2023-10-06T22:06:43.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.695+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd000624e0 con 0x7fbd00106960 2023-10-06T22:06:43.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.696+0000 7fbcef7fe700 1 -- 172.21.15.190:0/1851420252 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fbcf0003c30 con 0x7fbd00106960 2023-10-06T22:06:43.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.696+0000 7fbcef7fe700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbce80388c0 0x7fbce803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:43.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.696+0000 7fbcef7fe700 1 -- 172.21.15.190:0/1851420252 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbcf004c600 con 0x7fbd00106960 2023-10-06T22:06:43.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.696+0000 7fbcfdd9b700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbce80388c0 0x7fbce803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:43.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.697+0000 7fbcfdd9b700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbce80388c0 0x7fbce803ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4006fd0 tx=0x7fbcf4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:43.704 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.702+0000 7fbcef7fe700 1 -- 172.21.15.190:0/1851420252 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbcf002a590 con 0x7fbd00106960 2023-10-06T22:06:43.903 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.901+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd0010aea0 con 0x7fbce80388c0 2023-10-06T22:06:43.913 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.911+0000 7fbcef7fe700 1 -- 172.21.15.190:0/1851420252 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbd0010aea0 con 0x7fbce80388c0 2023-10-06T22:06:43.913 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2023-10-06T22:06:43.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbce80388c0 msgr2=0x7fbce803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:43.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbce80388c0 0x7fbce803ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4006fd0 tx=0x7fbcf4006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:43.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 msgr2=0x7fbd00195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:43.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00195e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0004f40 tx=0x7fbcf0005020 comp rx=0 tx=0).stop 2023-10-06T22:06:43.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 shutdown_connections 2023-10-06T22:06:43.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbce80388c0 0x7fbce803ad80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:43.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 --2- 172.21.15.190:0/1851420252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbd00106960 0x7fbd00195e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:43.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 >> 172.21.15.190:0/1851420252 conn(0x7fbd000fc470 msgr2=0x7fbd00108e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:43.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 shutdown_connections 2023-10-06T22:06:43.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:43.913+0000 7fbd04c4b700 1 -- 172.21.15.190:0/1851420252 wait complete. 2023-10-06T22:06:44.345 DEBUG:teuthology.orchestra.run.smithi156:mon.b> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-06T22:06:44.348 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-06T22:06:44.350 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T22:06:44.350 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T22:06:44.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:06:44.384 INFO:journalctl@ceph.mon.b.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:06:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:06:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:45.219 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:45 smithi057 ceph-mon[98492]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:45 smithi057 ceph-mon[98492]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c;count:3 2023-10-06T22:06:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:45 smithi057 ceph-mon[98492]: Deploying daemon mon.c on smithi190 2023-10-06T22:06:46.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:46 smithi057 ceph-mon[98492]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:47.175 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.172+0000 7ff25f3af700 1 -- 172.21.15.190:0/1947313268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 msgr2=0x7ff258105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:47.175 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.172+0000 7ff25f3af700 1 --2- 172.21.15.190:0/1947313268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258105c30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff254009910 tx=0x7ff254009c20 comp rx=0 tx=0).stop 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.172+0000 7ff25f3af700 1 -- 172.21.15.190:0/1947313268 shutdown_connections 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.172+0000 7ff25f3af700 1 --2- 172.21.15.190:0/1947313268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258105c30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.172+0000 7ff25f3af700 1 -- 172.21.15.190:0/1947313268 >> 172.21.15.190:0/1947313268 conn(0x7ff2580fb360 msgr2=0x7ff2580fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.172+0000 7ff25f3af700 1 -- 172.21.15.190:0/1947313268 shutdown_connections 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.172+0000 7ff25f3af700 1 -- 172.21.15.190:0/1947313268 wait complete. 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.173+0000 7ff25f3af700 1 Processor -- start 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.173+0000 7ff25f3af700 1 -- start start 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.173+0000 7ff25f3af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.173+0000 7ff25f3af700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581971b0 con 0x7ff258105850 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.173+0000 7ff25d14b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.173+0000 7ff25d14b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.190:40376/0 (socket says 172.21.15.190:40376) 2023-10-06T22:06:47.176 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.173+0000 7ff25d14b700 1 -- 172.21.15.190:0/4230853200 learned_addr learned my addr 172.21.15.190:0/4230853200 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:06:47.178 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.174+0000 7ff25d14b700 1 -- 172.21.15.190:0/4230853200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540095f0 con 0x7ff258105850 2023-10-06T22:06:47.178 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.174+0000 7ff25d14b700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258196c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff254004f40 tx=0x7ff254005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:47.178 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.174+0000 7ff24e7fc700 1 -- 172.21.15.190:0/4230853200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff25401d070 con 0x7ff258105850 2023-10-06T22:06:47.178 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.174+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff258197410 con 0x7ff258105850 2023-10-06T22:06:47.178 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.174+0000 7ff24e7fc700 1 -- 172.21.15.190:0/4230853200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff254004370 con 0x7ff258105850 2023-10-06T22:06:47.178 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.174+0000 7ff24e7fc700 1 -- 172.21.15.190:0/4230853200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff254003970 con 0x7ff258105850 2023-10-06T22:06:47.179 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.174+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff258197890 con 0x7ff258105850 2023-10-06T22:06:47.179 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.175+0000 7ff24e7fc700 1 -- 172.21.15.190:0/4230853200 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7ff254003c30 con 0x7ff258105850 2023-10-06T22:06:47.179 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.175+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff258197ad0 con 0x7ff258105850 2023-10-06T22:06:47.179 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.175+0000 7ff24e7fc700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff2440388c0 0x7ff24403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:47.179 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.176+0000 7ff24e7fc700 1 -- 172.21.15.190:0/4230853200 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff25404c650 con 0x7ff258105850 2023-10-06T22:06:47.179 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.176+0000 7ff25c94a700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff2440388c0 0x7ff24403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:47.179 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.177+0000 7ff25c94a700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff2440388c0 0x7ff24403ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff248006fd0 tx=0x7ff248006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:47.184 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.182+0000 7ff24e7fc700 1 -- 172.21.15.190:0/4230853200 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff254029930 con 0x7ff258105850 2023-10-06T22:06:47.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:46 smithi190 systemd[1]: Starting Ceph mon.c for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:06:47.476 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.473+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff258103b00 con 0x7ff258105850 2023-10-06T22:06:47.476 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.473+0000 7ff24e7fc700 1 -- 172.21.15.190:0/4230853200 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7ff254026030 con 0x7ff258105850 2023-10-06T22:06:47.476 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:06:47.477 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","modified":"2023-10-06T22:05:00.771890Z","created":"2023-10-06T22:05:00.771890Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-06T22:06:47.478 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff2440388c0 msgr2=0x7ff24403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:47.478 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff2440388c0 0x7ff24403ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff248006fd0 tx=0x7ff248006e40 comp rx=0 tx=0).stop 2023-10-06T22:06:47.478 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 msgr2=0x7ff258196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:47.478 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258196c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff254004f40 tx=0x7ff254005020 comp rx=0 tx=0).stop 2023-10-06T22:06:47.478 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 shutdown_connections 2023-10-06T22:06:47.479 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff2440388c0 0x7ff24403ad80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:47.479 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 --2- 172.21.15.190:0/4230853200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff258105850 0x7ff258196c70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:47.479 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 >> 172.21.15.190:0/4230853200 conn(0x7ff2580fb360 msgr2=0x7ff258107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:47.479 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.475+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 shutdown_connections 2023-10-06T22:06:47.479 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:47.476+0000 7ff25f3af700 1 -- 172.21.15.190:0/4230853200 wait complete. 2023-10-06T22:06:47.479 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2023-10-06T22:06:47.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 podman[134827]: 2023-10-06T22:06:47.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:47 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/4230853200' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 7 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pidfile_write: ignore empty --pid-file 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 bash[134827]: f75ab992de01d64b9b085d2a186e790189f6668459371286e20e6ea7261f39b9 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: load: jerasure load: lrc 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: RocksDB version: 7.9.2 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Git sha 0 2023-10-06T22:06:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: DB SUMMARY 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: DB Session ID: EUDITSR5V8HS1CUCHVCT 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: CURRENT file: CURRENT 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.error_if_exists: 0 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.create_if_missing: 0 2023-10-06T22:06:48.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.paranoid_checks: 1 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.env: 0x55899049ada0 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.info_log: 0x5589918835c0 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.statistics: (nil) 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.use_fsync: 0 2023-10-06T22:06:48.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_log_file_size: 0 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.allow_fallocate: 1 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.use_direct_reads: 0 2023-10-06T22:06:48.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.db_log_dir: 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.wal_dir: 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T22:06:48.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.write_buffer_manager: 0x55899183b360 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T22:06:48.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.unordered_write: 0 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.row_cache: None 2023-10-06T22:06:48.077 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.wal_filter: None 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.two_write_queues: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.wal_compression: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.atomic_flush: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T22:06:48.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.log_readahead_size: 0 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_background_jobs: 2 2023-10-06T22:06:48.079 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_background_compactions: -1 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_subcompactions: 1 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T22:06:48.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_open_files: -1 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_background_flushes: -1 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Compression algorithms supported: 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kZSTD supported: 0 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kXpressCompression supported: 0 2023-10-06T22:06:48.081 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kZlibCompression supported: 1 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kSnappyCompression supported: 1 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kLZ4Compression supported: 1 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: kBZip2Compression supported: 0 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T22:06:48.082 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.merge_operator: 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_filter: None 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5589918836c0) 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks: 1 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T22:06:48.083 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_top_level_index_and_filter: 1 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_type: 0 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_index_type: 0 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_shortening: 1 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: checksum: 4 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: no_block_cache: 0 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache: 0x5589918d91f0 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_name: BinnedLRUCache 2023-10-06T22:06:48.084 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_options: 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: capacity : 536870912 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_shard_bits : 4 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: strict_capacity_limit : 0 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: high_pri_pool_ratio: 0.000 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_compressed: (nil) 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: persistent_cache: (nil) 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size: 4096 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size_deviation: 10 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_restart_interval: 16 2023-10-06T22:06:48.085 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_block_restart_interval: 1 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: metadata_block_size: 4096 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: partition_filters: 0 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: use_delta_encoding: 1 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: filter_policy: bloomfilter 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: whole_key_filtering: 1 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: verify_compression: 0 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: read_amp_bytes_per_bit: 0 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: format_version: 5 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_index_compression: 1 2023-10-06T22:06:48.086 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_align: 0 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout: max_auto_readahead_size: 262144 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout: prepopulate_block_cache: 0 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout: initial_auto_readahead_size: 8192 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression: NoCompression 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T22:06:48.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.num_levels: 7 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T22:06:48.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T22:06:48.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T22:06:48.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T22:06:48.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T22:06:48.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T22:06:48.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T22:06:48.093 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.table_properties_collectors: 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.inplace_update_support: 0 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.bloom_locality: 0 2023-10-06T22:06:48.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.max_successive_merges: 0 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.ttl: 2592000 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.enable_blob_files: false 2023-10-06T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.min_blob_size: 0 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T22:06:48.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e5ed29f7-4e6b-4d34-afc6-80b65e5b938a 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696630007741048, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696630007741449, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696630007, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e5ed29f7-4e6b-4d34-afc6-80b65e5b938a", "db_session_id": "EUDITSR5V8HS1CUCHVCT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696630007741515, "job": 1, "event": "recovery_finished"} 2023-10-06T22:06:48.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558991976000 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: DB pointer 0x558991962000 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** DB Stats ** 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T22:06:48.098 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T22:06:48.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:48.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T22:06:48.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache BinnedLRUCache@0x5589918d91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T22:06:48.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: using public_addr v2:172.21.15.190:0/0 -> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: starting mon.c rank -1 at public addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] at bind addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(???) e0 preinit fsid 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).mds e1 new map 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).mds e1 print_map 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout: e1 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T22:06:48.103 INFO:journalctl@ceph.mon.c.smithi190.stdout: legacy client fscid: -1 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout: No filesystems configured 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:06:48.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:06:48.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:06:48.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Deploying cephadm binary to smithi190 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Added host smithi190 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3037700852' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3037700852' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c;count:3 2023-10-06T22:06:48.110 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: Deploying daemon mon.c on smithi190 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/4230853200' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expand map: {default=false} 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta from 'false' to 'false' 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expanded map: {default=false} 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expand map: {default=info} 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta from 'info' to 'info' 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expanded map: {default=info} 2023-10-06T22:06:48.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expand map: {default=daemon} 2023-10-06T22:06:48.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-06T22:06:48.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expanded map: {default=daemon} 2023-10-06T22:06:48.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expand map: {default=debug} 2023-10-06T22:06:48.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta from 'debug' to 'debug' 2023-10-06T22:06:48.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 ceph-mon[134891]: expand_channel_meta expanded map: {default=debug} 2023-10-06T22:06:48.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:47 smithi190 systemd[1]: Started Ceph mon.c for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:06:49.128 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T22:06:49.128 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T22:06:49.698 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:06:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:06:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: mon.a calling monitor election 2023-10-06T22:06:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: mon.c calling monitor election 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:06:53.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: fsmap 2023-10-06T22:06:53.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:06:53.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: mgrmap e13: x(active, since 47s) 2023-10-06T22:06:53.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: overall HEALTH_OK 2023-10-06T22:06:53.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:53.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:53.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:06:53.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: mon.a calling monitor election 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: mon.c calling monitor election 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T22:06:53.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:06:53.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: fsmap 2023-10-06T22:06:53.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:06:53.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: mgrmap e13: x(active, since 47s) 2023-10-06T22:06:53.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: overall HEALTH_OK 2023-10-06T22:06:53.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:53.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:53.894 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.891+0000 7f73747f7700 1 -- 172.21.15.190:0/1872062378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73580035c0 msgr2=0x7f7358005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:53.894 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.891+0000 7f73747f7700 1 --2- 172.21.15.190:0/1872062378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73580035c0 0x7f7358005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f736c077150 tx=0x7f735c00feb0 comp rx=0 tx=0).stop 2023-10-06T22:06:53.894 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.891+0000 7f73747f7700 1 -- 172.21.15.190:0/1872062378 shutdown_connections 2023-10-06T22:06:53.894 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.891+0000 7f73747f7700 1 --2- 172.21.15.190:0/1872062378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73580035c0 0x7f7358005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:53.894 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.891+0000 7f73747f7700 1 -- 172.21.15.190:0/1872062378 >> 172.21.15.190:0/1872062378 conn(0x7f736c0fb050 msgr2=0x7f736c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:53.894 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.892+0000 7f73747f7700 1 -- 172.21.15.190:0/1872062378 shutdown_connections 2023-10-06T22:06:53.894 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.892+0000 7f73747f7700 1 -- 172.21.15.190:0/1872062378 wait complete. 2023-10-06T22:06:53.896 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.892+0000 7f73747f7700 1 Processor -- start 2023-10-06T22:06:53.896 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.892+0000 7f73747f7700 1 -- start start 2023-10-06T22:06:53.896 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f73747f7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 0x7f736c196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f73747f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f736c1965d0 0x7f736c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f73747f7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c196b60 con 0x7f736c1965d0 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f73747f7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c196cd0 con 0x7f73580035c0 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7372593700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 0x7f736c196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7372593700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 0x7f736c196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46074/0 (socket says 172.21.15.190:46074) 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7372593700 1 -- 172.21.15.190:0/3848321550 learned_addr learned my addr 172.21.15.190:0/3848321550 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7372593700 1 -- 172.21.15.190:0/3848321550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 msgr2=0x7f736c196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7372593700 1 -- 172.21.15.190:0/3848321550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 msgr2=0x7f736c196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7372593700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 0x7f736c196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7372593700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 0x7f736c196090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:06:53.897 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7371d92700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f736c1965d0 0x7f736c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7371d92700 1 -- 172.21.15.190:0/3848321550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 msgr2=0x7f736c196090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7371d92700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 0x7f736c196090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.893+0000 7f7371d92700 1 -- 172.21.15.190:0/3848321550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c00f900 con 0x7f736c1965d0 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.894+0000 7f7371d92700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f736c1965d0 0x7f736c19aa40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f736800b4d0 tx=0x7f736800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.894+0000 7f73637fe700 1 -- 172.21.15.190:0/3848321550 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7368011630 con 0x7f736c1965d0 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.894+0000 7f73637fe700 1 -- 172.21.15.190:0/3848321550 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7368011c50 con 0x7f736c1965d0 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.894+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c19afe0 con 0x7f736c1965d0 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.894+0000 7f73637fe700 1 -- 172.21.15.190:0/3848321550 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7368010d90 con 0x7f736c1965d0 2023-10-06T22:06:53.898 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.894+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c19b530 con 0x7f736c1965d0 2023-10-06T22:06:53.899 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.895+0000 7f73637fe700 1 -- 172.21.15.190:0/3848321550 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f7368010410 con 0x7f736c1965d0 2023-10-06T22:06:53.899 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.895+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c0624e0 con 0x7f736c1965d0 2023-10-06T22:06:53.899 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.896+0000 7f73637fe700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7350038700 0x7f735003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:06:53.899 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.896+0000 7f73637fe700 1 -- 172.21.15.190:0/3848321550 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f736804afd0 con 0x7f736c1965d0 2023-10-06T22:06:53.900 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.896+0000 7f7372593700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7350038700 0x7f735003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:06:53.900 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.896+0000 7f7372593700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7350038700 0x7f735003abc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f735c010010 tx=0x7f735c001e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:06:53.903 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:53.901+0000 7f73637fe700 1 -- 172.21.15.190:0/3848321550 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7368011790 con 0x7f736c1965d0 2023-10-06T22:06:54.168 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.166+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f736c1088a0 con 0x7f736c1965d0 2023-10-06T22:06:54.169 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.167+0000 7f73637fe700 1 -- 172.21.15.190:0/3848321550 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f7368017730 con 0x7f736c1965d0 2023-10-06T22:06:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:06:54.169 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","modified":"2023-10-06T22:06:47.775880Z","created":"2023-10-06T22:05:00.771890Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.168+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7350038700 msgr2=0x7f735003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.168+0000 7f73747f7700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7350038700 0x7f735003abc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f735c010010 tx=0x7f735c001e50 comp rx=0 tx=0).stop 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.168+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f736c1965d0 msgr2=0x7f736c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.168+0000 7f73747f7700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f736c1965d0 0x7f736c19aa40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f736800b4d0 tx=0x7f736800b890 comp rx=0 tx=0).stop 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.168+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 shutdown_connections 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.168+0000 7f73747f7700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7350038700 0x7f735003abc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.169+0000 7f73747f7700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73580035c0 0x7f736c196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.169+0000 7f73747f7700 1 --2- 172.21.15.190:0/3848321550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f736c1965d0 0x7f736c19aa40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:06:54.171 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.169+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 >> 172.21.15.190:0/3848321550 conn(0x7f736c0fb050 msgr2=0x7f736c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:06:54.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.169+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 shutdown_connections 2023-10-06T22:06:54.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:06:54.169+0000 7f73747f7700 1 -- 172.21.15.190:0/3848321550 wait complete. 2023-10-06T22:06:54.172 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2023-10-06T22:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:53 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:53 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:53 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:53 smithi057 ceph-mon[98492]: Deploying daemon mon.b on smithi156 2023-10-06T22:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:53 smithi057 ceph-mon[98492]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:53 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:54.195 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:53 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:54.195 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:53 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:06:54.195 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:53 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:54.195 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:53 smithi190 ceph-mon[134891]: Deploying daemon mon.b on smithi156 2023-10-06T22:06:54.195 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:53 smithi190 ceph-mon[134891]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:54.195 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:53 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:55.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:06:54 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/3848321550' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:06:55.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:06:54 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/3848321550' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:06:55.561 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T22:06:55.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T22:06:55.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 systemd[1]: Starting Ceph mon.b for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:06:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 podman[134318]: 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 7 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pidfile_write: ignore empty --pid-file 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 bash[134318]: f12a53d717ac5038d2777449dea9675c2fec5d340861967b9d8d271afedba297 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: load: jerasure load: lrc 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: RocksDB version: 7.9.2 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Git sha 0 2023-10-06T22:06:56.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: DB SUMMARY 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: DB Session ID: M8EOHNXX3CGCY09C0BQK 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: CURRENT file: CURRENT 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.error_if_exists: 0 2023-10-06T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.create_if_missing: 0 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.paranoid_checks: 1 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.env: 0x55b1917ddda0 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.info_log: 0x55b192ca75c0 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.statistics: (nil) 2023-10-06T22:06:56.007 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.use_fsync: 0 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_log_file_size: 0 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.allow_fallocate: 1 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.use_direct_reads: 0 2023-10-06T22:06:56.008 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.db_log_dir: 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.wal_dir: 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T22:06:56.009 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.write_buffer_manager: 0x55b192c5f360 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T22:06:56.010 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.unordered_write: 0 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.row_cache: None 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.wal_filter: None 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T22:06:56.011 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.two_write_queues: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.wal_compression: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.atomic_flush: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.log_readahead_size: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T22:06:56.012 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_background_jobs: 2 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_background_compactions: -1 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_subcompactions: 1 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T22:06:56.013 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_open_files: -1 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T22:06:56.014 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_background_flushes: -1 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Compression algorithms supported: 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kZSTD supported: 0 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kXpressCompression supported: 0 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kZlibCompression supported: 1 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kSnappyCompression supported: 1 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kLZ4Compression supported: 1 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: kBZip2Compression supported: 0 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T22:06:56.015 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.merge_operator: 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_filter: None 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b192ca76c0) 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout: cache_index_and_filter_blocks: 1 2023-10-06T22:06:56.016 INFO:journalctl@ceph.mon.b.smithi156.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: pin_top_level_index_and_filter: 1 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: index_type: 0 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: data_block_index_type: 0 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: index_shortening: 1 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: checksum: 4 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: no_block_cache: 0 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache: 0x55b192cfd1f0 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache_name: BinnedLRUCache 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache_options: 2023-10-06T22:06:56.017 INFO:journalctl@ceph.mon.b.smithi156.stdout: capacity : 536870912 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: num_shard_bits : 4 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: strict_capacity_limit : 0 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: high_pri_pool_ratio: 0.000 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_cache_compressed: (nil) 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: persistent_cache: (nil) 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_size: 4096 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_size_deviation: 10 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_restart_interval: 16 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: index_block_restart_interval: 1 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: metadata_block_size: 4096 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: partition_filters: 0 2023-10-06T22:06:56.018 INFO:journalctl@ceph.mon.b.smithi156.stdout: use_delta_encoding: 1 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: filter_policy: bloomfilter 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: whole_key_filtering: 1 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: verify_compression: 0 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: read_amp_bytes_per_bit: 0 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: format_version: 5 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: enable_index_compression: 1 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: block_align: 0 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: max_auto_readahead_size: 262144 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: prepopulate_block_cache: 0 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: initial_auto_readahead_size: 8192 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T22:06:56.019 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression: NoCompression 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.num_levels: 7 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T22:06:56.020 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T22:06:56.021 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T22:06:56.022 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T22:06:56.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T22:06:56.024 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T22:06:56.024 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T22:06:56.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T22:06:56.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T22:06:56.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T22:06:56.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T22:06:56.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T22:06:56.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T22:06:56.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.table_properties_collectors: 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.inplace_update_support: 0 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.bloom_locality: 0 2023-10-06T22:06:56.026 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.max_successive_merges: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.ttl: 2592000 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.enable_blob_files: false 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.min_blob_size: 0 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T22:06:56.027 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: edd6dbfa-5cf6-4bf2-b687-44f7f0cbdbcd 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696630015878326, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-06T22:06:56.028 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696630015878730, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696630015, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "edd6dbfa-5cf6-4bf2-b687-44f7f0cbdbcd", "db_session_id": "M8EOHNXX3CGCY09C0BQK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696630015878797, "job": 1, "event": "recovery_finished"} 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b192d9a000 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: DB pointer 0x55b192d86000 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** DB Stats ** 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T22:06:56.029 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** Compaction Stats [default] ** 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** Compaction Stats [default] ** 2023-10-06T22:06:56.030 INFO:journalctl@ceph.mon.b.smithi156.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T22:06:56.031 INFO:journalctl@ceph.mon.b.smithi156.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout: Block cache BinnedLRUCache@0x55b192cfd1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.9e-05 secs_since: 0 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: using public_addr v2:172.21.15.156:0/0 -> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: starting mon.b rank -1 at public addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] at bind addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(???) e0 preinit fsid 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).mds e1 new map 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-06T22:06:56.032 INFO:journalctl@ceph.mon.b.smithi156.stdout: e1 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout: legacy client fscid: -1 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout: 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout: No filesystems configured 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T22:06:56.033 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.034 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:56.035 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Deploying cephadm binary to smithi190 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Added host smithi190 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3037700852' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T22:06:56.037 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3037700852' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:06:56.038 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c;count:3 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Deploying daemon mon.c on smithi190 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/4230853200' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.a calling monitor election 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:56.039 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.c calling monitor election 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: fsmap 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mgrmap e13: x(active, since 47s) 2023-10-06T22:06:56.040 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: overall HEALTH_OK 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: Deploying daemon mon.b on smithi156 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/3848321550' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 systemd[1]: Started Ceph mon.b for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:06:56.041 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expand map: {default=false} 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta from 'false' to 'false' 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expanded map: {default=false} 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expand map: {default=info} 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta from 'info' to 'info' 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expanded map: {default=info} 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expand map: {default=daemon} 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expanded map: {default=daemon} 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expand map: {default=debug} 2023-10-06T22:06:56.042 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta from 'debug' to 'debug' 2023-10-06T22:06:56.043 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:06:55 smithi156 ceph-mon[134358]: expand_channel_meta expanded map: {default=debug} 2023-10-06T22:06:56.132 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:07:01.044 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.041+0000 7fe836968700 1 -- 172.21.15.190:0/2988857718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8301059b0 msgr2=0x7fe81c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:01.044 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.041+0000 7fe836968700 1 --2- 172.21.15.190:0/2988857718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8301059b0 0x7fe81c005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe820005fa0 tx=0x7fe82000f4a0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.044 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.041+0000 7fe836968700 1 -- 172.21.15.190:0/2988857718 shutdown_connections 2023-10-06T22:07:01.044 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.041+0000 7fe836968700 1 --2- 172.21.15.190:0/2988857718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8301059b0 0x7fe81c005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.044 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.041+0000 7fe836968700 1 -- 172.21.15.190:0/2988857718 >> 172.21.15.190:0/2988857718 conn(0x7fe830076520 msgr2=0x7fe830076930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:01.044 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.041+0000 7fe836968700 1 -- 172.21.15.190:0/2988857718 shutdown_connections 2023-10-06T22:07:01.044 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.041+0000 7fe836968700 1 -- 172.21.15.190:0/2988857718 wait complete. 2023-10-06T22:07:01.046 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.042+0000 7fe836968700 1 Processor -- start 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe836968700 1 -- start start 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe836968700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 0x7fe830194e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe836968700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 0x7fe830199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe836968700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe830199d70 0x7fe83019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe836968700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830195ab0 con 0x7fe830199d70 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe836968700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8301a1ae0 con 0x7fe8301059b0 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe836968700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8301a1c20 con 0x7fe8301953c0 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe827fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 0x7fe830199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe827fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 0x7fe830199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46098/0 (socket says 172.21.15.190:46098) 2023-10-06T22:07:01.047 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe827fff700 1 -- 172.21.15.190:0/1536018842 learned_addr learned my addr 172.21.15.190:0/1536018842 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe834f05700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe830199d70 0x7fe83019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe827fff700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 msgr2=0x7fe830199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe827fff700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 msgr2=0x7fe830199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe827fff700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 0x7fe830199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe82ffff700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 0x7fe830194e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe827fff700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 0x7fe830199830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.043+0000 7fe82ffff700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 msgr2=0x7fe830194e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe82ffff700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 msgr2=0x7fe830194e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe82ffff700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 0x7fe830194e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe82ffff700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 0x7fe830194e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe834f05700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 msgr2=0x7fe830199830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe834f05700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 0x7fe830199830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe834f05700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 msgr2=0x7fe830194e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe834f05700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 0x7fe830194e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe834f05700 1 -- 172.21.15.190:0/1536018842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200149d0 con 0x7fe830199d70 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe834f05700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe830199d70 0x7fe83019a220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe82800bd50 tx=0x7fe82800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe82dffb700 1 -- 172.21.15.190:0/1536018842 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800cd10 con 0x7fe830199d70 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe82dffb700 1 -- 172.21.15.190:0/1536018842 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe82800ce70 con 0x7fe830199d70 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8300686a0 con 0x7fe830199d70 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe82dffb700 1 -- 172.21.15.190:0/1536018842 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828012840 con 0x7fe830199d70 2023-10-06T22:07:01.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.044+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe830068ba0 con 0x7fe830199d70 2023-10-06T22:07:01.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.045+0000 7fe82dffb700 1 -- 172.21.15.190:0/1536018842 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fe8280129c0 con 0x7fe830199d70 2023-10-06T22:07:01.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.046+0000 7fe82dffb700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe810038d90 0x7fe81003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:01.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.046+0000 7fe82dffb700 1 -- 172.21.15.190:0/1536018842 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe82804ae20 con 0x7fe830199d70 2023-10-06T22:07:01.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.046+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8300624e0 con 0x7fe830199d70 2023-10-06T22:07:01.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.046+0000 7fe82ffff700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe810038d90 0x7fe81003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:01.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.046+0000 7fe82ffff700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe810038d90 0x7fe81003b250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe820000c00 tx=0x7fe820016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:01.055 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.053+0000 7fe82dffb700 1 -- 172.21.15.190:0/1536018842 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe82801d3b0 con 0x7fe830199d70 2023-10-06T22:07:01.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:07:01.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: mon.a calling monitor election 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: mon.c calling monitor election 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T22:07:01.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: fsmap 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: mgrmap e13: x(active, since 55s) 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:01.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:01.336 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.333+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe8301961e0 con 0x7fe830199d70 2023-10-06T22:07:01.336 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.334+0000 7fe82dffb700 1 -- 172.21.15.190:0/1536018842 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7fe8280112a0 con 0x7fe830199d70 2023-10-06T22:07:01.337 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-06T22:07:01.337 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","modified":"2023-10-06T22:06:55.902149Z","created":"2023-10-06T22:05:00.771890Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-06T22:07:01.338 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe810038d90 msgr2=0x7fe81003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe810038d90 0x7fe81003b250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe820000c00 tx=0x7fe820016000 comp rx=0 tx=0).stop 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe830199d70 msgr2=0x7fe83019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe830199d70 0x7fe83019a220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe82800bd50 tx=0x7fe82800be30 comp rx=0 tx=0).stop 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 shutdown_connections 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe810038d90 0x7fe81003b250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe830199d70 0x7fe83019a220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8301059b0 0x7fe830194e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 --2- 172.21.15.190:0/1536018842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8301953c0 0x7fe830199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:01.339 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 >> 172.21.15.190:0/1536018842 conn(0x7fe830076520 msgr2=0x7fe83018dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:01.340 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.336+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 shutdown_connections 2023-10-06T22:07:01.340 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:01.337+0000 7fe836968700 1 -- 172.21.15.190:0/1536018842 wait complete. 2023-10-06T22:07:01.340 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2023-10-06T22:07:01.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:07:01.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: mon.a calling monitor election 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: mon.c calling monitor election 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:07:01.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: fsmap 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: mgrmap e13: x(active, since 55s) 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:01.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:01.695 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-06T22:07:01.695 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-06T22:07:02.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:02.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:02.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:01 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/1536018842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:07:02.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:01 smithi057 ceph-mon[98492]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:02.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:02.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:02.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:02.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:01 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/1536018842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:07:02.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:01 smithi190 ceph-mon[134891]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:02.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:02.440 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: mon.a calling monitor election 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: mon.c calling monitor election 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: fsmap 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: mgrmap e13: x(active, since 55s) 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-06T22:07:03.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/1536018842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:03.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.062+0000 7fb056d10700 1 -- 172.21.15.57:0/343862424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050106970 msgr2=0x7fb050106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.062+0000 7fb056d10700 1 --2- 172.21.15.57:0/343862424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050106970 0x7fb050106d50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb044009910 tx=0x7fb044009c20 comp rx=0 tx=0).stop 2023-10-06T22:07:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.062+0000 7fb056d10700 1 -- 172.21.15.57:0/343862424 shutdown_connections 2023-10-06T22:07:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.062+0000 7fb056d10700 1 --2- 172.21.15.57:0/343862424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050106970 0x7fb050106d50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.062+0000 7fb056d10700 1 -- 172.21.15.57:0/343862424 >> 172.21.15.57:0/343862424 conn(0x7fb0500fc4a0 msgr2=0x7fb0500fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.062+0000 7fb056d10700 1 -- 172.21.15.57:0/343862424 shutdown_connections 2023-10-06T22:07:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.062+0000 7fb056d10700 1 -- 172.21.15.57:0/343862424 wait complete. 2023-10-06T22:07:04.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.063+0000 7fb056d10700 1 Processor -- start 2023-10-06T22:07:04.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.063+0000 7fb056d10700 1 -- start start 2023-10-06T22:07:04.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb056d10700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb050106970 0x7fb0501981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:04.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb056d10700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050198700 0x7fb05019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:04.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb056d10700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05019d0b0 0x7fb05019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb056d10700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb050198df0 con 0x7fb050198700 2023-10-06T22:07:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb056d10700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501a4e20 con 0x7fb05019d0b0 2023-10-06T22:07:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb056d10700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501a4f60 con 0x7fb050106970 2023-10-06T22:07:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb04ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050198700 0x7fb05019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb04ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050198700 0x7fb05019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58864/0 (socket says 172.21.15.57:58864) 2023-10-06T22:07:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb0552ad700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05019d0b0 0x7fb05019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb04ffff700 1 -- 172.21.15.57:0/2359677842 learned_addr learned my addr 172.21.15.57:0/2359677842 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:07:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.064+0000 7fb054aac700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb050106970 0x7fb0501981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:04.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb0552ad700 1 -- 172.21.15.57:0/2359677842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb050106970 msgr2=0x7fb0501981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:04.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb0552ad700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb050106970 0x7fb0501981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb0552ad700 1 -- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050198700 msgr2=0x7fb05019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:04.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb0552ad700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050198700 0x7fb05019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb0552ad700 1 -- 172.21.15.57:0/2359677842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440095f0 con 0x7fb05019d0b0 2023-10-06T22:07:04.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb054aac700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb050106970 0x7fb0501981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:07:04.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb04ffff700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050198700 0x7fb05019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb0552ad700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05019d0b0 0x7fb05019d560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb04800bd50 tx=0x7fb04800bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb04dffb700 1 -- 172.21.15.57:0/2359677842 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04800cbb0 con 0x7fb05019d0b0 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb04dffb700 1 -- 172.21.15.57:0/2359677842 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04800cd10 con 0x7fb05019d0b0 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0501a5240 con 0x7fb05019d0b0 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb04dffb700 1 -- 172.21.15.57:0/2359677842 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048012690 con 0x7fb05019d0b0 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.065+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0501a5740 con 0x7fb05019d0b0 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.066+0000 7fb04dffb700 1 -- 172.21.15.57:0/2359677842 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fb048014440 con 0x7fb05019d0b0 2023-10-06T22:07:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.066+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0500624e0 con 0x7fb05019d0b0 2023-10-06T22:07:04.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: mon.b calling monitor election 2023-10-06T22:07:04.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: mon.b calling monitor election 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: mon.c calling monitor election 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: mon.a calling monitor election 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:07:04.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: fsmap 2023-10-06T22:07:04.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:07:04.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: mgrmap e13: x(active, since 57s) 2023-10-06T22:07:04.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-06T22:07:04.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Cluster is now healthy 2023-10-06T22:07:04.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: overall HEALTH_OK 2023-10-06T22:07:04.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.075 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:03 smithi190 ceph-mon[134891]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:04.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.067+0000 7fb04dffb700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb040038d90 0x7fb04003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:04.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.067+0000 7fb04dffb700 1 -- 172.21.15.57:0/2359677842 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb04804adb0 con 0x7fb05019d0b0 2023-10-06T22:07:04.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.067+0000 7fb054aac700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb040038d90 0x7fb04003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:04.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.067+0000 7fb054aac700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb040038d90 0x7fb04003b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb044009fd0 tx=0x7fb0440058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:04.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.073+0000 7fb04dffb700 1 -- 172.21.15.57:0/2359677842 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb04801b8d0 con 0x7fb05019d0b0 2023-10-06T22:07:04.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: mon.b calling monitor election 2023-10-06T22:07:04.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-06T22:07:04.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: mon.b calling monitor election 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: mon.c calling monitor election 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: mon.a calling monitor election 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: fsmap 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: mgrmap e13: x(active, since 57s) 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-06T22:07:04.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Cluster is now healthy 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: overall HEALTH_OK 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:03 smithi057 ceph-mon[98492]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:04.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: mon.b calling monitor election 2023-10-06T22:07:04.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-06T22:07:04.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:07:04.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:07:04.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: mon.b calling monitor election 2023-10-06T22:07:04.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: mon.c calling monitor election 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: mon.a calling monitor election 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: fsmap 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: mgrmap e13: x(active, since 57s) 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Cluster is now healthy 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: overall HEALTH_OK 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:07:04.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:04.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:03 smithi156 ceph-mon[134358]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:04.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.300+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb05010aec0 con 0x7fb05019d0b0 2023-10-06T22:07:04.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.300+0000 7fb04dffb700 1 -- 172.21.15.57:0/2359677842 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fb04801b8d0 con 0x7fb05019d0b0 2023-10-06T22:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout:# minimal ceph.conf for 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2023-10-06T22:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: mon_host = [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb040038d90 msgr2=0x7fb04003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb040038d90 0x7fb04003b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb044009fd0 tx=0x7fb0440058e0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05019d0b0 msgr2=0x7fb05019d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05019d0b0 0x7fb05019d560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb04800bd50 tx=0x7fb04800bd80 comp rx=0 tx=0).stop 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 shutdown_connections 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb040038d90 0x7fb04003b250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb05019d0b0 0x7fb05019d560 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb050106970 0x7fb0501981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 --2- 172.21.15.57:0/2359677842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb050198700 0x7fb05019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:04.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 >> 172.21.15.57:0/2359677842 conn(0x7fb0500fc4a0 msgr2=0x7fb050073370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:04.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 shutdown_connections 2023-10-06T22:07:04.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:04.303+0000 7fb056d10700 1 -- 172.21.15.57:0/2359677842 wait complete. 2023-10-06T22:07:04.830 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-06T22:07:04.830 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:07:04.830 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T22:07:04.913 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:07:04.913 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:07:05.035 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:07:05.035 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T22:07:05.069 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:05.069 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.069 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: Reconfiguring daemon mon.a on smithi057 2023-10-06T22:07:05.070 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:04 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2359677842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:05.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:05.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T22:07:05.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:05.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:05.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:05.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: Reconfiguring daemon mon.a on smithi057 2023-10-06T22:07:05.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:04 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2359677842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:05.075 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:07:05.075 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:07:05.107 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:07:05.107 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T22:07:05.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:07:04.902+0000 7fbd1c6ef700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T22:07:05.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:05.117 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:05.117 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:05.117 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: Reconfiguring daemon mon.a on smithi057 2023-10-06T22:07:05.117 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:04 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2359677842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:05.148 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:07:05.148 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:07:05.224 INFO:tasks.cephadm:Adding mgr.x on smithi057 2023-10-06T22:07:05.224 INFO:tasks.cephadm:Adding mgr.y on smithi156 2023-10-06T22:07:05.225 INFO:tasks.cephadm:Adding mgr.z on smithi190 2023-10-06T22:07:05.225 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi057=x;smithi156=y;smithi190=z' 2023-10-06T22:07:05.824 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:07:06.651 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:06 smithi156 ceph-mon[134358]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:06.651 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.651 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.651 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.651 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:06.652 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:06.652 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:06.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:06 smithi190 ceph-mon[134891]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:06.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.707 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:06.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:06.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:06 smithi057 ceph-mon[98492]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:07.106 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.104+0000 7f61d6d97700 1 -- 172.21.15.190:0/3340053129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 msgr2=0x7f61d0103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:07.107 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.104+0000 7f61d6d97700 1 --2- 172.21.15.190:0/3340053129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 0x7f61d0103980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009910 tx=0x7f61c4009c20 comp rx=0 tx=0).stop 2023-10-06T22:07:07.107 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.104+0000 7f61d6d97700 1 -- 172.21.15.190:0/3340053129 shutdown_connections 2023-10-06T22:07:07.107 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.104+0000 7f61d6d97700 1 --2- 172.21.15.190:0/3340053129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 0x7f61d0103980 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.107 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.104+0000 7f61d6d97700 1 -- 172.21.15.190:0/3340053129 >> 172.21.15.190:0/3340053129 conn(0x7f61d00fafc0 msgr2=0x7f61d00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:07.107 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.104+0000 7f61d6d97700 1 -- 172.21.15.190:0/3340053129 shutdown_connections 2023-10-06T22:07:07.107 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.104+0000 7f61d6d97700 1 -- 172.21.15.190:0/3340053129 wait complete. 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.105+0000 7f61d6d97700 1 Processor -- start 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.105+0000 7f61d6d97700 1 -- start start 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.105+0000 7f61d6d97700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 0x7f61d0195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d6d97700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61d0196250 0x7f61d019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d6d97700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d01966d0 0x7f61d019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d6d97700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d019e000 con 0x7f61d0101590 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d6d97700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d019e170 con 0x7f61d0196250 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d6d97700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d019e330 con 0x7f61d01966d0 2023-10-06T22:07:07.109 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d01966d0 0x7f61d019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d01966d0 0x7f61d019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46110/0 (socket says 172.21.15.190:46110) 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 -- 172.21.15.190:0/1045313566 learned_addr learned my addr 172.21.15.190:0/1045313566 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61cffff700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61d0196250 0x7f61d019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d4b33700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 0x7f61d0195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 -- 172.21.15.190:0/1045313566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61d0196250 msgr2=0x7f61d019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61d0196250 0x7f61d019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 -- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 msgr2=0x7f61d0195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 0x7f61d0195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 -- 172.21.15.190:0/1045313566 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40095f0 con 0x7f61d01966d0 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d5334700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d01966d0 0x7f61d019d830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61c800ba50 tx=0x7f61c800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:07.110 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61cdffb700 1 -- 172.21.15.190:0/1045313566 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800c9b0 con 0x7f61d01966d0 2023-10-06T22:07:07.111 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61cdffb700 1 -- 172.21.15.190:0/1045313566 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61c8014440 con 0x7f61d01966d0 2023-10-06T22:07:07.111 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d019e610 con 0x7f61d01966d0 2023-10-06T22:07:07.111 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.106+0000 7f61cdffb700 1 -- 172.21.15.190:0/1045313566 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8007590 con 0x7f61d01966d0 2023-10-06T22:07:07.111 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.107+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d019eb40 con 0x7f61d01966d0 2023-10-06T22:07:07.112 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.107+0000 7f61cdffb700 1 -- 172.21.15.190:0/1045313566 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f61c80077b0 con 0x7f61d01966d0 2023-10-06T22:07:07.112 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.108+0000 7f61cdffb700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f61c0038d90 0x7f61c003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:07.112 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.108+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d018f610 con 0x7f61d01966d0 2023-10-06T22:07:07.112 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.108+0000 7f61cdffb700 1 -- 172.21.15.190:0/1045313566 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f61c804aba0 con 0x7f61d01966d0 2023-10-06T22:07:07.112 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.108+0000 7f61d4b33700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f61c0038d90 0x7f61c003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:07.112 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.109+0000 7f61d4b33700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f61c0038d90 0x7f61c003b250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009fd0 tx=0x7f61c40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:07.116 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.114+0000 7f61cdffb700 1 -- 172.21.15.190:0/1045313566 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f61c801b8b0 con 0x7f61d01966d0 2023-10-06T22:07:07.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.330+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi156=y;smithi190=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f61d002d2a0 con 0x7f61c0038d90 2023-10-06T22:07:07.341 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.339+0000 7f61cdffb700 1 -- 172.21.15.190:0/1045313566 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f61d002d2a0 con 0x7f61c0038d90 2023-10-06T22:07:07.341 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2023-10-06T22:07:07.343 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f61c0038d90 msgr2=0x7f61c003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:07.343 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f61c0038d90 0x7f61c003b250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009fd0 tx=0x7f61c40058e0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.344 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d01966d0 msgr2=0x7f61d019d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:07.344 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d01966d0 0x7f61d019d830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61c800ba50 tx=0x7f61c800bd60 comp rx=0 tx=0).stop 2023-10-06T22:07:07.344 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 shutdown_connections 2023-10-06T22:07:07.344 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f61c0038d90 0x7f61c003b250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.344 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d01966d0 0x7f61d019d830 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.344 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61d0101590 0x7f61d0195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.345 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 --2- 172.21.15.190:0/1045313566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61d0196250 0x7f61d019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:07.345 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 >> 172.21.15.190:0/1045313566 conn(0x7f61d00fafc0 msgr2=0x7f61d00fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:07.345 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 shutdown_connections 2023-10-06T22:07:07.345 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:07:07.341+0000 7f61d6d97700 1 -- 172.21.15.190:0/1045313566 wait complete. 2023-10-06T22:07:07.781 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: Reconfiguring mon.b (monmap changed)... 2023-10-06T22:07:07.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: Reconfiguring daemon mon.b on smithi156 2023-10-06T22:07:07.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:07.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:07.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:07.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:07.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:07.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: Reconfiguring mon.b (monmap changed)... 2023-10-06T22:07:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: Reconfiguring daemon mon.b on smithi156 2023-10-06T22:07:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:07.969 DEBUG:teuthology.orchestra.run.smithi156:mgr.y> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-06T22:07:07.972 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.z.service 2023-10-06T22:07:07.974 INFO:tasks.cephadm:Deploying OSDs... 2023-10-06T22:07:07.974 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:07:07.974 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T22:07:08.000 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: Reconfiguring mon.b (monmap changed)... 2023-10-06T22:07:08.000 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: Reconfiguring daemon mon.b on smithi156 2023-10-06T22:07:08.000 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:08.001 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:08.001 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T22:07:08.001 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T22:07:08.001 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:08.001 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:08.002 INFO:journalctl@ceph.mgr.z.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:07:08.006 INFO:journalctl@ceph.mgr.y.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:07:08.022 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T22:07:08.023 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-06 22:06:18.160605309 +0000 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-06 22:00:26.799274115 +0000 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-06 22:00:26.799274115 +0000 2023-10-06T22:07:08.109 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2023-10-06 22:00:26.770274531 +0000 2023-10-06T22:07:08.110 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T22:07:08.218 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-06T22:07:08.218 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-06T22:07:08.218 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000333834 s, 1.5 MB/s 2023-10-06T22:07:08.221 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T22:07:08.266 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2023-10-06T22:07:08.351 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T22:07:08.351 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:08.351 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-06T22:07:08.351 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:08.351 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:08.352 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-06 22:06:18.243604113 +0000 2023-10-06T22:07:08.352 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-06 22:00:27.647261955 +0000 2023-10-06T22:07:08.352 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-06 22:00:27.647261955 +0000 2023-10-06T22:07:08.352 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2023-10-06 22:00:27.617262385 +0000 2023-10-06T22:07:08.352 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T22:07:08.458 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-06T22:07:08.458 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-06T22:07:08.458 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000339887 s, 1.5 MB/s 2023-10-06T22:07:08.460 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T22:07:08.505 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2023-10-06T22:07:08.589 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T22:07:08.589 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:08.589 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-06T22:07:08.589 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:08.589 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:08.589 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-06 22:06:18.331602845 +0000 2023-10-06T22:07:08.590 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-06 22:00:28.518249466 +0000 2023-10-06T22:07:08.590 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-06 22:00:28.518249466 +0000 2023-10-06T22:07:08.590 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2023-10-06 22:00:28.487249911 +0000 2023-10-06T22:07:08.590 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T22:07:08.695 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-06T22:07:08.696 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-06T22:07:08.696 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.00038314 s, 1.3 MB/s 2023-10-06T22:07:08.698 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T22:07:08.743 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2023-10-06T22:07:08.828 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T22:07:08.828 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:08.828 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-06T22:07:08.828 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:08.829 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:08.829 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-06 22:06:18.425601491 +0000 2023-10-06T22:07:08.829 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-06 22:00:29.345237608 +0000 2023-10-06T22:07:08.829 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-06 22:00:29.345237608 +0000 2023-10-06T22:07:08.829 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2023-10-06 22:00:29.317238009 +0000 2023-10-06T22:07:08.829 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: Reconfiguring mon.c (monmap changed)... 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: Reconfiguring daemon mon.c on smithi190 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi156=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: Saving service mgr spec with placement smithi057=x;smithi156=y;smithi190=z;count:3 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:08.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:08.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:08.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:08.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:08.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T22:07:08.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:08.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:08.934 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-06T22:07:08.934 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-06T22:07:08.934 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.00051437 s, 995 kB/s 2023-10-06T22:07:08.936 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T22:07:08.981 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-10-06T22:07:08.981 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T22:07:09.002 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T22:07:09.002 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: Reconfiguring mon.c (monmap changed)... 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: Reconfiguring daemon mon.c on smithi190 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi156=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: Saving service mgr spec with placement smithi057=x;smithi156=y;smithi190=z;count:3 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:09.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:09.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T22:07:09.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:09.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:09.024 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T22:07:09.024 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:09.024 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 395649 Links: 1 2023-10-06T22:07:09.024 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:09.025 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:09.025 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-10-06 22:06:55.364756498 +0000 2023-10-06T22:07:09.025 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-10-06 21:56:43.028490309 +0000 2023-10-06T22:07:09.025 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-10-06 21:56:43.028490309 +0000 2023-10-06T22:07:09.025 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-10-06T22:07:09.025 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T22:07:09.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: Reconfiguring mon.c (monmap changed)... 2023-10-06T22:07:09.035 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: Reconfiguring daemon mon.c on smithi190 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi156=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: Saving service mgr spec with placement smithi057=x;smithi156=y;smithi190=z;count:3 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T22:07:09.036 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:09.037 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:09.093 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-10-06T22:07:09.093 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-10-06T22:07:09.093 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000150089 s, 3.4 MB/s 2023-10-06T22:07:09.094 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T22:07:09.155 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-10-06T22:07:09.213 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T22:07:09.214 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:09.214 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 394769 Links: 1 2023-10-06T22:07:09.214 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:09.214 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:09.214 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-10-06 22:06:55.364756498 +0000 2023-10-06T22:07:09.214 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-10-06 21:56:43.520482679 +0000 2023-10-06T22:07:09.214 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-10-06 21:56:43.520482679 +0000 2023-10-06T22:07:09.215 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-10-06T22:07:09.215 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T22:07:09.281 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-10-06T22:07:09.281 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-10-06T22:07:09.281 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000123661 s, 4.1 MB/s 2023-10-06T22:07:09.282 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T22:07:09.342 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 392974 Links: 1 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-10-06 22:06:55.364756498 +0000 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-10-06 21:56:44.033474723 +0000 2023-10-06T22:07:09.400 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-10-06 21:56:44.033474723 +0000 2023-10-06T22:07:09.401 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-10-06T22:07:09.401 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T22:07:09.466 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-10-06T22:07:09.466 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-10-06T22:07:09.466 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 8.9598e-05 s, 5.7 MB/s 2023-10-06T22:07:09.467 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T22:07:09.525 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-10-06T22:07:09.582 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T22:07:09.582 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:09.582 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 391493 Links: 1 2023-10-06T22:07:09.582 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:09.583 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:09.583 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-10-06 22:06:55.364756498 +0000 2023-10-06T22:07:09.583 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-10-06 21:56:44.557466596 +0000 2023-10-06T22:07:09.583 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-10-06 21:56:44.557466596 +0000 2023-10-06T22:07:09.583 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-10-06T22:07:09.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T22:07:09.648 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-10-06T22:07:09.649 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-10-06T22:07:09.649 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000114901 s, 4.5 MB/s 2023-10-06T22:07:09.650 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T22:07:09.707 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-06T22:07:09.708 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T22:07:09.725 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T22:07:09.725 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2023-10-06T22:07:09.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:09 smithi190 ceph-mon[134891]: Deploying daemon mgr.z on smithi190 2023-10-06T22:07:09.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:09 smithi156 ceph-mon[134358]: Deploying daemon mgr.z on smithi190 2023-10-06T22:07:09.759 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T22:07:09.759 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:09.759 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 386867 Links: 1 2023-10-06T22:07:09.759 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:09.759 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:09.759 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-06 22:06:46.944145279 +0000 2023-10-06T22:07:09.759 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-06 21:56:42.204726631 +0000 2023-10-06T22:07:09.760 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-06 21:56:42.204726631 +0000 2023-10-06T22:07:09.760 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-06T22:07:09.760 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T22:07:09.847 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-06T22:07:09.847 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-06T22:07:09.847 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00022839 s, 2.2 MB/s 2023-10-06T22:07:09.848 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T22:07:09.866 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2023-10-06T22:07:09.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:09 smithi057 ceph-mon[98492]: Deploying daemon mgr.z on smithi190 2023-10-06T22:07:09.924 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 393850 Links: 1 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-06 22:07:09.793490647 +0000 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-06 21:56:42.703712544 +0000 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-06 21:56:42.703712544 +0000 2023-10-06T22:07:09.925 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-06T22:07:09.926 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T22:07:09.993 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-06T22:07:09.994 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-06T22:07:09.994 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000257102 s, 2.0 MB/s 2023-10-06T22:07:09.995 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:09 smithi190 systemd[1]: Starting Ceph mgr.z for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:07:09.997 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T22:07:10.057 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2023-10-06T22:07:10.117 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T22:07:10.117 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:10.117 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 393883 Links: 1 2023-10-06T22:07:10.117 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:10.117 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:10.117 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-06 22:07:09.930486722 +0000 2023-10-06T22:07:10.118 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-06 21:56:43.218698005 +0000 2023-10-06T22:07:10.118 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-06 21:56:43.218698005 +0000 2023-10-06T22:07:10.118 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-06T22:07:10.118 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T22:07:10.185 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-06T22:07:10.185 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-06T22:07:10.185 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000224859 s, 2.3 MB/s 2023-10-06T22:07:10.186 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T22:07:10.246 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 386966 Links: 1 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-06 22:07:09.930486722 +0000 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-06 21:56:43.752682930 +0000 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-06 21:56:43.752682930 +0000 2023-10-06T22:07:10.304 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-06T22:07:10.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T22:07:10.374 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-06T22:07:10.374 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-06T22:07:10.374 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000223415 s, 2.3 MB/s 2023-10-06T22:07:10.375 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T22:07:10.434 INFO:tasks.cephadm:Deploying osd.0 on smithi057 with /dev/vg_nvme/lv_4... 2023-10-06T22:07:10.435 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T22:07:10.534 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 podman[136894]: 2023-10-06T22:07:10.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:10 smithi190 ceph-mon[134891]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:10.819 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 ceph-mgr[137039]: -- 172.21.15.190:0/2842666640 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55996efb31e0 con 0x55996ef88c00 2023-10-06T22:07:10.820 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 bash[136894]: f94a1257c0c3fc6b648ca0ba18c5cdb343d5a7f8ade2f13b2485c7e43a4b9c1c 2023-10-06T22:07:10.820 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 systemd[1]: Started Ceph mgr.z for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:07:10.820 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:10.655+0000 7f51e594f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:07:10.820 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:10.734+0000 7f51e594f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:07:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:10 smithi057 ceph-mon[98492]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:11.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:10 smithi156 ceph-mon[134358]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:11.084 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:10.822+0000 7f51e594f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:07:11.085 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:10.914+0000 7f51e594f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:07:11.184 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:07:11.445 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:11.082+0000 7f51e594f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:07:11.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:11.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T22:07:11.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:11.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:11.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:11 smithi057 ceph-mon[98492]: Deploying daemon mgr.y on smithi156 2023-10-06T22:07:11.762 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:11.443+0000 7f51e594f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:07:11.762 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:11.577+0000 7f51e594f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:07:11.762 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:11.665+0000 7f51e594f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:07:11.762 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:11.759+0000 7f51e594f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:07:11.762 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.762 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.762 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.763 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:11.763 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T22:07:11.763 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:11.763 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:11.763 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:11 smithi190 ceph-mon[134891]: Deploying daemon mgr.y on smithi156 2023-10-06T22:07:11.804 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.805 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.805 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:11.805 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:11.805 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T22:07:11.805 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:11.805 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:11.805 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:11 smithi156 ceph-mon[134358]: Deploying daemon mgr.y on smithi156 2023-10-06T22:07:12.254 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 systemd[1]: Starting Ceph mgr.y for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:07:12.569 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:12.178+0000 7f51e594f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:07:12.616 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 podman[135801]: 2023-10-06T22:07:12.616 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 bash[135801]: 5b13adeaa1a396bddfb1138c1f13565657380c16da77f21f873cc57792f3cae7 2023-10-06T22:07:12.616 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 ceph-mgr[135842]: -- 172.21.15.156:0/3395067814 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557837a051e0 con 0x5578379da800 2023-10-06T22:07:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:12 smithi057 ceph-mon[98492]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:12.930 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:12 smithi190 ceph-mon[134891]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:12.931 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:12.842+0000 7f51e594f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:07:12.953 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:12 smithi156 ceph-mon[134358]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:12.953 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 systemd[1]: Started Ceph mgr.y for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:07:12.953 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:12.670+0000 7f2c5608c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:07:12.953 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:12.752+0000 7f2c5608c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:07:12.954 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:12.845+0000 7f2c5608c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:07:12.954 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:12.951+0000 7f2c5608c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:07:13.181 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T22:07:13.181 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T22:07:13.181 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-06T22:07:13.181 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-06T22:07:13.181 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0313383 s, 335 MB/s 2023-10-06T22:07:13.181 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-06T22:07:13.253 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:13.132+0000 7f2c5608c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:07:13.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:12.928+0000 7f51e594f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:07:13.670 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_4 2023-10-06T22:07:13.858 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.858 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.859 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.859 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.859 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:13.859 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:13.517+0000 7f2c5608c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:07:13.859 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:13.659+0000 7f2c5608c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:07:13.859 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:13.755+0000 7f2c5608c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:07:13.859 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:13.856+0000 7f2c5608c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:07:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:14.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:14.025 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:13.741+0000 7f51e594f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:07:14.025 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:13.838+0000 7f51e594f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:07:14.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:14.021+0000 7f51e594f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:07:14.420 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:07:14.698 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:14.318+0000 7f2c5608c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:07:14.699 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:14.324+0000 7f51e594f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:07:14.699 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:14.556+0000 7f51e594f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:07:14.970 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:14.707+0000 7f51e594f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:14.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:14.972 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:14.972 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:15.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:15.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:15.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:15.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:15.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:15.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:15.062 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:15.062 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T22:07:15.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:15.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:14.968+0000 7f51e594f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:07:15.503 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:15.016+0000 7f2c5608c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:07:15.504 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:15.098+0000 7f2c5608c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:07:15.968 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:15.875+0000 7f2c5608c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:07:15.968 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:15 smithi156 ceph-mon[134358]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T22:07:15.969 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:15 smithi156 ceph-mon[134358]: Reconfiguring daemon mgr.x on smithi057 2023-10-06T22:07:15.969 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:15 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:16.015 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:15.691+0000 7f51e594f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:07:16.016 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:15.821+0000 7f51e594f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:07:16.016 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:15 smithi190 ceph-mon[134891]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T22:07:16.016 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:15 smithi190 ceph-mon[134891]: Reconfiguring daemon mgr.x on smithi057 2023-10-06T22:07:16.016 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:15 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:16.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- 172.21.15.57:0/368799771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102b10 msgr2=0x7f30d01093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:16.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 --2- 172.21.15.57:0/368799771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102b10 0x7f30d01093c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f30c800b0d0 tx=0x7f30c800b3e0 comp rx=0 tx=0).stop 2023-10-06T22:07:16.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- 172.21.15.57:0/368799771 shutdown_connections 2023-10-06T22:07:16.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 --2- 172.21.15.57:0/368799771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102b10 0x7f30d01093c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:16.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 --2- 172.21.15.57:0/368799771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 0x7f30d0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:16.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 --2- 172.21.15.57:0/368799771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d0102170 0x7f30d01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:16.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- 172.21.15.57:0/368799771 >> 172.21.15.57:0/368799771 conn(0x7f30d00fc4d0 msgr2=0x7f30d00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:16.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- 172.21.15.57:0/368799771 shutdown_connections 2023-10-06T22:07:16.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- 172.21.15.57:0/368799771 wait complete. 2023-10-06T22:07:16.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:15 smithi057 ceph-mon[98492]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T22:07:16.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:15 smithi057 ceph-mon[98492]: Reconfiguring daemon mgr.x on smithi057 2023-10-06T22:07:16.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:15 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:16.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 Processor -- start 2023-10-06T22:07:16.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- start start 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 0x7f30d019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102170 0x7f30d019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d0102b10 0x7f30d01a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01a4ee0 con 0x7f30d0102170 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01a5020 con 0x7f30d0100f70 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01a51b0 con 0x7f30d0102b10 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 0x7f30d019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 0x7f30d019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55542/0 (socket says 172.21.15.57:55542) 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 -- 172.21.15.57:0/2080998176 learned_addr learned my addr 172.21.15.57:0/2080998176 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30cffff700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102170 0x7f30d019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:16.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d50c2700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d0102b10 0x7f30d01a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:16.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 -- 172.21.15.57:0/2080998176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d0102b10 msgr2=0x7f30d01a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d0102b10 0x7f30d01a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 -- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102170 msgr2=0x7f30d019d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102170 0x7f30d019d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 -- 172.21.15.57:0/2080998176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8009d00 con 0x7f30d0100f70 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d50c2700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d0102b10 0x7f30d01a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d48c1700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 0x7f30d019d350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30c400e970 tx=0x7f30c400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4009790 con 0x7f30d0100f70 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30c400eec0 con 0x7f30d0100f70 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d01a5490 con 0x7f30d0100f70 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4003ce0 con 0x7f30d0100f70 2023-10-06T22:07:16.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.084+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d01a5990 con 0x7f30d0100f70 2023-10-06T22:07:16.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.089+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f30c4018430 con 0x7f30d0100f70 2023-10-06T22:07:16.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.089+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d010c7d0 con 0x7f30d0100f70 2023-10-06T22:07:16.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.089+0000 7f30cdffb700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30c0038af0 0x7f30c003afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:16.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.089+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f30c4014070 con 0x7f30d0100f70 2023-10-06T22:07:16.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.089+0000 7f30cffff700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30c0038af0 0x7f30c003afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:16.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.089+0000 7f30cffff700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30c0038af0 0x7f30c003afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009c60 tx=0x7f30bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:16.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.091+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f30c400ccd0 con 0x7f30d0100f70 2023-10-06T22:07:16.253 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:15.966+0000 7f2c5608c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:07:16.253 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:16.139+0000 7f2c5608c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:07:16.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:16.013+0000 7f51e594f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:07:16.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:16.126+0000 7f51e594f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:07:16.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:16.321+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d0108a30 con 0x7f30c0038af0 2023-10-06T22:07:16.705 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:16.418+0000 7f2c5608c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:07:16.705 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:16.651+0000 7f2c5608c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:07:16.705 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:16.342+0000 7f51e594f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:16.801+0000 7f2c5608c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:17.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:17.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:17.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:07:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:07:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:17.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:17.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:17.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:07:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:07:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:17.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:17.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:17.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:17.503 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:17.065+0000 7f2c5608c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:07:17.569 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:07:17.213+0000 7f51e594f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:07:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:17.713+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 14) v1 ==== 90791+0+0 (secure 0 0 0) 0x7f30c40186e0 con 0x7f30d0100f70 2023-10-06T22:07:17.996 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:17 smithi057 ceph-mon[98492]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:17.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:17 smithi057 ceph-mon[98492]: Standby manager daemon z started 2023-10-06T22:07:17.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:17 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T22:07:17.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:17 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:07:17.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:17 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T22:07:17.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:17 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:07:17.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:17 smithi057 ceph-mon[98492]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:18.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:17.770+0000 7f2c5608c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:07:18.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:17.901+0000 7f2c5608c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:07:18.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-mon[134358]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:18.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-mon[134358]: Standby manager daemon z started 2023-10-06T22:07:18.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T22:07:18.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:07:18.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T22:07:18.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:07:18.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:17 smithi156 ceph-mon[134358]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:18.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-mon[134891]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-mon[134891]: Standby manager daemon z started 2023-10-06T22:07:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T22:07:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:07:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T22:07:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/2407404925' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:07:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:17 smithi190 ceph-mon[134891]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:18.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:18.095+0000 7f2c5608c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:07:18.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:18.208+0000 7f2c5608c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:07:18.715 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:18.425+0000 7f2c5608c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:07:19.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:18 smithi156 ceph-mon[134358]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-06T22:07:19.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T22:07:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:18 smithi190 ceph-mon[134891]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-06T22:07:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T22:07:19.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:18 smithi057 ceph-mon[98492]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-06T22:07:19.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T22:07:19.720 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:07:19.295+0000 7f2c5608c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:07:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:19.735+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mon.2 v2:172.21.15.156:3300/0 8 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f30c4014230 con 0x7f30d0100f70 2023-10-06T22:07:20.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: Standby manager daemon y started 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2118506200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]: dispatch 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]: dispatch 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]': finished 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:20.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:19 smithi156 ceph-mon[134358]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:20.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: Standby manager daemon y started 2023-10-06T22:07:20.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T22:07:20.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2118506200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]: dispatch 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]: dispatch 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]': finished 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:19 smithi190 ceph-mon[134891]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: Standby manager daemon y started 2023-10-06T22:07:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T22:07:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:07:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T22:07:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/4209597305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:07:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2118506200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]: dispatch 2023-10-06T22:07:20.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]: dispatch 2023-10-06T22:07:20.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25492bd6-3367-475f-b073-59bd613eab8a"}]': finished 2023-10-06T22:07:20.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T22:07:20.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:20.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:19 smithi057 ceph-mon[98492]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:21.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:21 smithi190 ceph-mon[134891]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-06T22:07:21.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T22:07:21.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:21 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2047541451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:07:21.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:21 smithi057 ceph-mon[98492]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-06T22:07:21.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T22:07:21.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:21 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2047541451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:07:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:21 smithi156 ceph-mon[134358]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-06T22:07:21.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T22:07:21.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:21 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2047541451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:07:22.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:22 smithi190 ceph-mon[134891]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:22.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:22 smithi057 ceph-mon[98492]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:22.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:22 smithi156 ceph-mon[134358]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:24 smithi057 ceph-mon[98492]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:25.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:24 smithi156 ceph-mon[134358]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:25.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:24 smithi190 ceph-mon[134891]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:26.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:26 smithi057 ceph-mon[98492]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:27.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:26 smithi156 ceph-mon[134358]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:26 smithi190 ceph-mon[134891]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:28 smithi057 ceph-mon[98492]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T22:07:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:29.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:28 smithi156 ceph-mon[134358]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T22:07:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:29.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:28 smithi190 ceph-mon[134891]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:29.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T22:07:29.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:29.744 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:29 smithi057 ceph-mon[98492]: Deploying daemon osd.0 on smithi057 2023-10-06T22:07:30.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:29 smithi156 ceph-mon[134358]: Deploying daemon osd.0 on smithi057 2023-10-06T22:07:30.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:29 smithi190 ceph-mon[134891]: Deploying daemon osd.0 on smithi057 2023-10-06T22:07:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:30 smithi057 ceph-mon[98492]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:30 smithi156 ceph-mon[134358]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:30 smithi190 ceph-mon[134891]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:32 smithi057 ceph-mon[98492]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:33.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:32 smithi156 ceph-mon[134358]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:32 smithi190 ceph-mon[134891]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:34 smithi057 ceph-mon[98492]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:35.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:34 smithi156 ceph-mon[134358]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:35.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:35.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:35.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:34 smithi190 ceph-mon[134891]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:36.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:36 smithi057 ceph-mon[98492]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:37.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:36 smithi156 ceph-mon[134358]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:37.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:36 smithi190 ceph-mon[134891]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:37.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.735+0000 7f30cdffb700 1 -- 172.21.15.57:0/2080998176 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f30d0108a30 con 0x7f30c0038af0 2023-10-06T22:07:37.738 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 0 on host 'smithi057' 2023-10-06T22:07:37.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30c0038af0 msgr2=0x7f30c003afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:37.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30c0038af0 0x7f30c003afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009c60 tx=0x7f30bc009260 comp rx=0 tx=0).stop 2023-10-06T22:07:37.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 msgr2=0x7f30d019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:37.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 0x7f30d019d350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30c400e970 tx=0x7f30c400ec80 comp rx=0 tx=0).stop 2023-10-06T22:07:37.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 shutdown_connections 2023-10-06T22:07:37.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30c0038af0 0x7f30c003afb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:37.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d0102b10 0x7f30d01a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:37.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0100f70 0x7f30d019d350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:37.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 --2- 172.21.15.57:0/2080998176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30d0102170 0x7f30d019d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:37.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 >> 172.21.15.57:0/2080998176 conn(0x7f30d00fc4d0 msgr2=0x7f30d00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:37.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 shutdown_connections 2023-10-06T22:07:37.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:07:37.738+0000 7f30d6b25700 1 -- 172.21.15.57:0/2080998176 wait complete. 2023-10-06T22:07:38.314 DEBUG:teuthology.orchestra.run.smithi057:osd.0> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-06T22:07:38.321 INFO:tasks.cephadm:Deploying osd.1 on smithi156 with /dev/vg_nvme/lv_4... 2023-10-06T22:07:38.321 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T22:07:38.399 INFO:journalctl@ceph.osd.0.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:07:38.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:38.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:38.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:38.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:38.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:38.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:38.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:38.887 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:07:39.362 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:39 smithi156 ceph-mon[134358]: from='osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T22:07:39.363 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:39 smithi156 ceph-mon[134358]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T22:07:39.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:39 smithi057 ceph-mon[98492]: from='osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T22:07:39.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:39 smithi057 ceph-mon[98492]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T22:07:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:39 smithi190 ceph-mon[134891]: from='osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T22:07:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:39 smithi190 ceph-mon[134891]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T22:07:40.266 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T22:07:40.266 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T22:07:40.266 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-10-06T22:07:40.266 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-10-06T22:07:40.267 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00820759 s, 1.3 GB/s 2023-10-06T22:07:40.267 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-10-06T22:07:40.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:40 smithi190 ceph-mon[134891]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T22:07:40.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:40 smithi190 ceph-mon[134891]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T22:07:40.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:40 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:40.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:40 smithi190 ceph-mon[134891]: from='osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:07:40.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:40 smithi190 ceph-mon[134891]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:07:40.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:40 smithi190 ceph-mon[134891]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:40.628 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:40 smithi156 ceph-mon[134358]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T22:07:40.628 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:40 smithi156 ceph-mon[134358]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T22:07:40.628 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:40 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:40.628 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:40 smithi156 ceph-mon[134358]: from='osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:07:40.628 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:40 smithi156 ceph-mon[134358]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:07:40.628 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:40 smithi156 ceph-mon[134358]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:40.630 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2023-10-06T22:07:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-mon[98492]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T22:07:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-mon[98492]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T22:07:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-mon[98492]: from='osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:07:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-mon[98492]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:07:40.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-mon[98492]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:40.674 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-0[116815]: 2023-10-06T22:07:40.270+0000 7f1844bfb700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T22:07:40.675 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:07:40 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-0[116815]: 2023-10-06T22:07:40.274+0000 7f183f9ee700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:07:41.191 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:07:41.549 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:41 smithi057 ceph-mon[98492]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:07:41.550 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:41 smithi057 ceph-mon[98492]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T22:07:41.550 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:41 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:41.550 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:41 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:41 smithi190 ceph-mon[134891]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:07:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:41 smithi190 ceph-mon[134891]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T22:07:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:41 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:41 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:41.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:41 smithi156 ceph-mon[134358]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:07:41.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:41 smithi156 ceph-mon[134358]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T22:07:41.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:41 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:41.605 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:41 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 -- 172.21.15.156:0/2953136795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 msgr2=0x7f85581090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 --2- 172.21.15.156:0/2953136795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f85581090d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f855000b0a0 tx=0x7f855000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 -- 172.21.15.156:0/2953136795 shutdown_connections 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 --2- 172.21.15.156:0/2953136795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f85581090d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 --2- 172.21.15.156:0/2953136795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558101a60 0x7f8558103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 --2- 172.21.15.156:0/2953136795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8558104390 0x7f8558106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 -- 172.21.15.156:0/2953136795 >> 172.21.15.156:0/2953136795 conn(0x7f85580fb3f0 msgr2=0x7f85580fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 -- 172.21.15.156:0/2953136795 shutdown_connections 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.393+0000 7f855d9fa700 1 -- 172.21.15.156:0/2953136795 wait complete. 2023-10-06T22:07:42.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 Processor -- start 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 -- start start 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8558101a60 0x7f8558196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558104390 0x7f8558197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f855819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855819e780 con 0x7f8558101a60 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855819e8c0 con 0x7f8558106cc0 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f855d9fa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855819ea80 con 0x7f8558104390 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f85577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f855819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f85577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f855819e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37464/0 (socket says 172.21.15.156:37464) 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.394+0000 7f85577fe700 1 -- 172.21.15.156:0/3016610922 learned_addr learned my addr 172.21.15.156:0/3016610922 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-10-06T22:07:42.398 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f8556ffd700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8558101a60 0x7f8558196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:42.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85567fc700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558104390 0x7f8558197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:42.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85577fe700 1 -- 172.21.15.156:0/3016610922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558104390 msgr2=0x7f8558197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:42.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85577fe700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558104390 0x7f8558197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:42.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85577fe700 1 -- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8558101a60 msgr2=0x7f8558196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:42.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85577fe700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8558101a60 0x7f8558196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:42.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85577fe700 1 -- 172.21.15.156:0/3016610922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8550009d00 con 0x7f8558106cc0 2023-10-06T22:07:42.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85567fc700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558104390 0x7f8558197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:07:42.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f85577fe700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f855819e1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f855000b830 tx=0x7f8550004b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:42.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f855c9f8700 1 -- 172.21.15.156:0/3016610922 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855000e040 con 0x7f8558106cc0 2023-10-06T22:07:42.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f855c9f8700 1 -- 172.21.15.156:0/3016610922 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855001db40 con 0x7f8558106cc0 2023-10-06T22:07:42.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855819ed00 con 0x7f8558106cc0 2023-10-06T22:07:42.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f855c9f8700 1 -- 172.21.15.156:0/3016610922 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855001bd50 con 0x7f8558106cc0 2023-10-06T22:07:42.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.395+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855819f170 con 0x7f8558106cc0 2023-10-06T22:07:42.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.396+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85580fcf70 con 0x7f8558106cc0 2023-10-06T22:07:42.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.397+0000 7f855c9f8700 1 -- 172.21.15.156:0/3016610922 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f855001dcb0 con 0x7f8558106cc0 2023-10-06T22:07:42.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.398+0000 7f855c9f8700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f85400a13f0 0x7f85400a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:07:42.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.398+0000 7f855c9f8700 1 -- 172.21.15.156:0/3016610922 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f85500cd460 con 0x7f8558106cc0 2023-10-06T22:07:42.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.398+0000 7f8556ffd700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f85400a13f0 0x7f85400a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:07:42.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.399+0000 7f8556ffd700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f85400a13f0 0x7f85400a38b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f854c005ea0 tx=0x7f854c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:07:42.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.402+0000 7f855c9f8700 1 -- 172.21.15.156:0/3016610922 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8550098520 con 0x7f8558106cc0 2023-10-06T22:07:42.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] boot 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:42.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:42.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] boot 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:42.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:42.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:42.599+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f855802d290 con 0x7f85400a13f0 2023-10-06T22:07:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:07:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:07:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: osd.0 [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] boot 2023-10-06T22:07:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T22:07:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:42.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:43 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi057 2023-10-06T22:07:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:43 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi057 to 17272M 2023-10-06T22:07:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:43 smithi190 ceph-mon[134891]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T22:07:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:43 smithi190 ceph-mon[134891]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:43.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:07:43.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:07:43.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:43.608 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:43 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi057 2023-10-06T22:07:43.608 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:43 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi057 to 17272M 2023-10-06T22:07:43.608 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:43 smithi156 ceph-mon[134358]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T22:07:43.608 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:43 smithi156 ceph-mon[134358]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:43.608 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:07:43.608 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:07:43.608 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:43 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi057 2023-10-06T22:07:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:43 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi057 to 17272M 2023-10-06T22:07:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:43 smithi057 ceph-mon[98492]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T22:07:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:43 smithi057 ceph-mon[98492]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:07:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:07:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:07:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:44.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:44 smithi156 ceph-mon[134358]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:44.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:44 smithi190 ceph-mon[134891]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:44.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:44 smithi057 ceph-mon[98492]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:45.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:45 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/1229476190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]: dispatch 2023-10-06T22:07:45.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:45 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]: dispatch 2023-10-06T22:07:45.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:45 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]': finished 2023-10-06T22:07:45.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:45 smithi190 ceph-mon[134891]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T22:07:45.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:45 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:07:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:45 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/1229476190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]: dispatch 2023-10-06T22:07:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:45 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]: dispatch 2023-10-06T22:07:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:45 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]': finished 2023-10-06T22:07:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:45 smithi057 ceph-mon[98492]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T22:07:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:45 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:07:45.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:45 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/1229476190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]: dispatch 2023-10-06T22:07:45.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:45 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]: dispatch 2023-10-06T22:07:45.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:45 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7f3e993-648c-4873-9ecc-fee568a3a076"}]': finished 2023-10-06T22:07:45.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:45 smithi156 ceph-mon[134358]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T22:07:45.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:45 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:07:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:46 smithi190 ceph-mon[134891]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:46 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/815306752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:07:46.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:46 smithi057 ceph-mon[98492]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:46.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:46 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/815306752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:07:47.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:46 smithi156 ceph-mon[134358]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:46 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/815306752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:07:48.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:48 smithi190 ceph-mon[134891]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:48.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:48 smithi057 ceph-mon[98492]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:48.938 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:48 smithi156 ceph-mon[134358]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:50.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:50 smithi057 ceph-mon[98492]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:50.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:50 smithi156 ceph-mon[134358]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:50 smithi190 ceph-mon[134891]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:51.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:51 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T22:07:51.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:51 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:51.924 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:51 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T22:07:51.925 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:51 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:51 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T22:07:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:51 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:52.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:52 smithi057 ceph-mon[98492]: Deploying daemon osd.1 on smithi156 2023-10-06T22:07:52.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:52 smithi057 ceph-mon[98492]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:53.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:52 smithi156 ceph-mon[134358]: Deploying daemon osd.1 on smithi156 2023-10-06T22:07:53.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:52 smithi156 ceph-mon[134358]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:52 smithi190 ceph-mon[134891]: Deploying daemon osd.1 on smithi156 2023-10-06T22:07:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:52 smithi190 ceph-mon[134891]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:54.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:54 smithi057 ceph-mon[98492]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:55.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:54 smithi156 ceph-mon[134358]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:54 smithi190 ceph-mon[134891]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:56.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:56.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:56.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:56.664 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:56 smithi156 ceph-mon[134358]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:56.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:56.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:56.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:56.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:56 smithi190 ceph-mon[134891]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:56 smithi057 ceph-mon[98492]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:58.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.308+0000 7f855c9f8700 1 -- 172.21.15.156:0/3016610922 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f855802d290 con 0x7f85400a13f0 2023-10-06T22:07:58.311 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 1 on host 'smithi156' 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.310+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f85400a13f0 msgr2=0x7f85400a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.310+0000 7f855d9fa700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f85400a13f0 0x7f85400a38b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f854c005ea0 tx=0x7f854c005e30 comp rx=0 tx=0).stop 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 msgr2=0x7f855819e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f855819e1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f855000b830 tx=0x7f8550004b00 comp rx=0 tx=0).stop 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 shutdown_connections 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f85400a13f0 0x7f85400a38b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558106cc0 0x7f855819e1b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8558101a60 0x7f8558196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:58.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 --2- 172.21.15.156:0/3016610922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558104390 0x7f8558197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:07:58.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 >> 172.21.15.156:0/3016610922 conn(0x7f85580fb3f0 msgr2=0x7f85581026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:07:58.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 shutdown_connections 2023-10-06T22:07:58.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:07:58.311+0000 7f855d9fa700 1 -- 172.21.15.156:0/3016610922 wait complete. 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:58.797 DEBUG:teuthology.orchestra.run.smithi156:osd.1> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-06T22:07:58.799 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/vg_nvme/lv_4... 2023-10-06T22:07:58.799 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T22:07:58.829 INFO:journalctl@ceph.osd.1.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:07:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:07:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:07:59.366 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:07:59.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:59 smithi156 ceph-mon[134358]: from='osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T22:07:59.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:07:59 smithi156 ceph-mon[134358]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T22:07:59.754 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:07:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-1[139452]: 2023-10-06T22:07:59.475+0000 7fb30a38d640 -1 osd.1 0 log_to_monitors true 2023-10-06T22:07:59.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:59 smithi190 ceph-mon[134891]: from='osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T22:07:59.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:07:59 smithi190 ceph-mon[134891]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T22:07:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:59 smithi057 ceph-mon[98492]: from='osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T22:07:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:07:59 smithi057 ceph-mon[98492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T22:08:00.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:00 smithi190 ceph-mon[134891]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:08:00.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:00 smithi190 ceph-mon[134891]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T22:08:00.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:00 smithi190 ceph-mon[134891]: from='osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:08:00.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:00 smithi190 ceph-mon[134891]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T22:08:00.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:00.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:00 smithi190 ceph-mon[134891]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:08:00.891 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T22:08:00.891 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T22:08:00.891 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-10-06T22:08:00.891 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-10-06T22:08:00.891 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110615 s, 948 MB/s 2023-10-06T22:08:00.891 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-10-06T22:08:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:00 smithi057 ceph-mon[98492]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:08:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:00 smithi057 ceph-mon[98492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T22:08:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:00 smithi057 ceph-mon[98492]: from='osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:08:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:00 smithi057 ceph-mon[98492]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T22:08:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:00 smithi057 ceph-mon[98492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:08:00.951 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-1[139452]: 2023-10-06T22:08:00.634+0000 7fb2ff203700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T22:08:00.951 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-1[139452]: 2023-10-06T22:08:00.638+0000 7fb2f97f5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:08:00.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-mon[134358]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:08:00.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-mon[134358]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T22:08:00.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-mon[134358]: from='osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:08:00.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-mon[134358]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T22:08:00.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:00.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:00 smithi156 ceph-mon[134358]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:08:01.310 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2023-10-06T22:08:01.868 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:08:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:08:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T22:08:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:01.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:02.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:02.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:02.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:02.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:08:02.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:08:02.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi156 2023-10-06T22:08:02.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi156 to 17168M 2023-10-06T22:08:02.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:08:02.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] boot 2023-10-06T22:08:02.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T22:08:02.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:08:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:08:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi156 2023-10-06T22:08:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi156 to 17168M 2023-10-06T22:08:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:08:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] boot 2023-10-06T22:08:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T22:08:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi156 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi156 to 17168M 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: osd.1 [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] boot 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T22:08:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 -- 172.21.15.190:0/49641807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4100fd0 msgr2=0x7f7db4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 --2- 172.21.15.190:0/49641807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4100fd0 0x7f7db4101450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009990 tx=0x7f7da4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 -- 172.21.15.190:0/49641807 shutdown_connections 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 --2- 172.21.15.190:0/49641807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7db4101990 0x7f7db41082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 --2- 172.21.15.190:0/49641807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7db40ffe70 0x7f7db4100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 --2- 172.21.15.190:0/49641807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4100fd0 0x7f7db4101450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 -- 172.21.15.190:0/49641807 >> 172.21.15.190:0/49641807 conn(0x7f7db40fb3f0 msgr2=0x7f7db40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:08:03.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 -- 172.21.15.190:0/49641807 shutdown_connections 2023-10-06T22:08:03.212 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.208+0000 7f7dba24a700 1 -- 172.21.15.190:0/49641807 wait complete. 2023-10-06T22:08:03.212 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 Processor -- start 2023-10-06T22:08:03.212 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 -- start start 2023-10-06T22:08:03.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7db40ffe70 0x7f7db4196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:03.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7db4100fd0 0x7f7db4197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:03.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4101990 0x7f7db419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:03.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419e8c0 con 0x7f7db4100fd0 2023-10-06T22:08:03.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419ea00 con 0x7f7db40ffe70 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7dba24a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419eb90 con 0x7f7db4101990 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7db3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4101990 0x7f7db419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7db3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4101990 0x7f7db419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46140/0 (socket says 172.21.15.190:46140) 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7db3fff700 1 -- 172.21.15.190:0/4282306049 learned_addr learned my addr 172.21.15.190:0/4282306049 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.209+0000 7f7db37fe700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7db40ffe70 0x7f7db4196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db2ffd700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7db4100fd0 0x7f7db4197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db3fff700 1 -- 172.21.15.190:0/4282306049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7db40ffe70 msgr2=0x7f7db4196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db3fff700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7db40ffe70 0x7f7db4196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db3fff700 1 -- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7db4100fd0 msgr2=0x7f7db4197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db3fff700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7db4100fd0 0x7f7db4197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:03.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db3fff700 1 -- 172.21.15.190:0/4282306049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40095f0 con 0x7f7db4101990 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db3fff700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4101990 0x7f7db419e2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00add0 tx=0x7f7dac00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db0ff9700 1 -- 172.21.15.190:0/4282306049 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00f070 con 0x7f7db4101990 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db0ff9700 1 -- 172.21.15.190:0/4282306049 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dac007b30 con 0x7f7db4101990 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db419ee70 con 0x7f7db4101990 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7db0ff9700 1 -- 172.21.15.190:0/4282306049 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac005310 con 0x7f7db4101990 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.210+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db419f2e0 con 0x7f7db4101990 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.211+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db41074b0 con 0x7f7db4101990 2023-10-06T22:08:03.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.212+0000 7f7db0ff9700 1 -- 172.21.15.190:0/4282306049 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f7dac00c940 con 0x7f7db4101990 2023-10-06T22:08:03.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.213+0000 7f7db0ff9700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7d9c0a1470 0x7f7d9c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:03.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.213+0000 7f7db0ff9700 1 -- 172.21.15.190:0/4282306049 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f7dac0caa20 con 0x7f7db4101990 2023-10-06T22:08:03.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.213+0000 7f7db37fe700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7d9c0a1470 0x7f7d9c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:03.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.214+0000 7f7db37fe700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7d9c0a1470 0x7f7d9c0a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009a10 tx=0x7f7da8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:08:03.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.217+0000 7f7db0ff9700 1 -- 172.21.15.190:0/4282306049 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7dac095960 con 0x7f7db4101990 2023-10-06T22:08:03.417 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:03.415+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7db402d2b0 con 0x7f7d9c0a1470 2023-10-06T22:08:04.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:04 smithi190 ceph-mon[134891]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T22:08:04.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:04 smithi190 ceph-mon[134891]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:04.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:04.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:04.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:04 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:04.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:04 smithi190 ceph-mon[134891]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:04.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:04 smithi057 ceph-mon[98492]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T22:08:04.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:04 smithi057 ceph-mon[98492]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:04.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:04.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:04.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:04 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:04.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:04 smithi057 ceph-mon[98492]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:04.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:04 smithi156 ceph-mon[134358]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T22:08:04.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:04 smithi156 ceph-mon[134358]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:04.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:04.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:04.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:04 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:04.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:04 smithi156 ceph-mon[134358]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:06.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:06 smithi057 ceph-mon[98492]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:06.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:06 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/375958853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]: dispatch 2023-10-06T22:08:06.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:06 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]: dispatch 2023-10-06T22:08:06.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:06 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]': finished 2023-10-06T22:08:06.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:06 smithi057 ceph-mon[98492]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T22:08:06.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:06.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:06 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/1869596751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:06 smithi156 ceph-mon[134358]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:06 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/375958853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]: dispatch 2023-10-06T22:08:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:06 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]: dispatch 2023-10-06T22:08:07.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:06 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]': finished 2023-10-06T22:08:07.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:06 smithi156 ceph-mon[134358]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T22:08:07.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:07.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:06 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/1869596751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:06 smithi190 ceph-mon[134891]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:06 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/375958853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]: dispatch 2023-10-06T22:08:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:06 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]: dispatch 2023-10-06T22:08:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:06 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ce3125d-3f65-4413-9fcb-486f5b565ad5"}]': finished 2023-10-06T22:08:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:06 smithi190 ceph-mon[134891]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T22:08:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:06 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/1869596751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:08 smithi057 ceph-mon[98492]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:09.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:08 smithi156 ceph-mon[134358]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:08 smithi190 ceph-mon[134891]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:10 smithi057 ceph-mon[98492]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:11.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:10 smithi156 ceph-mon[134358]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:10 smithi190 ceph-mon[134891]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:12 smithi057 ceph-mon[98492]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:13.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:12 smithi156 ceph-mon[134358]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:12 smithi190 ceph-mon[134891]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:13.916 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T22:08:13.916 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T22:08:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:14.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T22:08:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:14.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:14 smithi057 ceph-mon[98492]: Deploying daemon osd.2 on smithi190 2023-10-06T22:08:14.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:14 smithi057 ceph-mon[98492]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:15.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:14 smithi156 ceph-mon[134358]: Deploying daemon osd.2 on smithi190 2023-10-06T22:08:15.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:14 smithi156 ceph-mon[134358]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:15.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:14 smithi190 ceph-mon[134891]: Deploying daemon osd.2 on smithi190 2023-10-06T22:08:15.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:14 smithi190 ceph-mon[134891]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:16 smithi057 ceph-mon[98492]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:17.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:16 smithi156 ceph-mon[134358]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:17.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:16 smithi190 ceph-mon[134891]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:18.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:18.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:18.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:18.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:18 smithi190 ceph-mon[134891]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:18.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:18.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:18.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:18.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:18 smithi057 ceph-mon[98492]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:18.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:18.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:18.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:18.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:18 smithi156 ceph-mon[134358]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:20.284 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.282+0000 7f7db0ff9700 1 -- 172.21.15.190:0/4282306049 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7db402d2b0 con 0x7f7d9c0a1470 2023-10-06T22:08:20.284 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7d9c0a1470 msgr2=0x7f7d9c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7d9c0a1470 0x7f7d9c0a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009a10 tx=0x7f7da8009190 comp rx=0 tx=0).stop 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4101990 msgr2=0x7f7db419e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4101990 0x7f7db419e2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00add0 tx=0x7f7dac00c330 comp rx=0 tx=0).stop 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 shutdown_connections 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7d9c0a1470 0x7f7d9c0a3930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db4101990 0x7f7db419e2f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7db40ffe70 0x7f7db4196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 --2- 172.21.15.190:0/4282306049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7db4100fd0 0x7f7db4197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:20.287 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 >> 172.21.15.190:0/4282306049 conn(0x7f7db40fb3f0 msgr2=0x7f7db41045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:08:20.288 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 shutdown_connections 2023-10-06T22:08:20.288 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:08:20.284+0000 7f7dba24a700 1 -- 172.21.15.190:0/4282306049 wait complete. 2023-10-06T22:08:20.749 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:20 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:20.751 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-06T22:08:20.753 INFO:tasks.cephadm:Deploying osd.3 on smithi057 with /dev/vg_nvme/lv_3... 2023-10-06T22:08:20.754 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T22:08:20.786 INFO:journalctl@ceph.osd.2.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:20.823 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:20 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:21.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:21.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:20 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:21.497 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:08:21.498 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:21 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-2[140652]: 2023-10-06T22:08:21.391+0000 7f5873245640 -1 osd.2 0 log_to_monitors true 2023-10-06T22:08:21.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:21 smithi190 ceph-mon[134891]: from='osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T22:08:21.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:21 smithi190 ceph-mon[134891]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T22:08:21.901 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:21 smithi057 ceph-mon[98492]: from='osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T22:08:21.901 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:21 smithi057 ceph-mon[98492]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T22:08:22.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:21 smithi156 ceph-mon[134358]: from='osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T22:08:22.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:21 smithi156 ceph-mon[134358]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T22:08:22.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:22 smithi057 ceph-mon[98492]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:22.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:22 smithi057 ceph-mon[98492]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T22:08:22.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:22 smithi057 ceph-mon[98492]: from='osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:08:22.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:22 smithi057 ceph-mon[98492]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T22:08:22.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:22 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:22.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:22 smithi057 ceph-mon[98492]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:08:22.986 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-2[140652]: 2023-10-06T22:08:22.589+0000 7f58680bb700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T22:08:22.986 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-2[140652]: 2023-10-06T22:08:22.594+0000 7f5862eae700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:08:22.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-mon[134891]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:22.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-mon[134891]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T22:08:22.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-mon[134891]: from='osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:08:22.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-mon[134891]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T22:08:22.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:22.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:22 smithi190 ceph-mon[134891]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:08:23.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:22 smithi156 ceph-mon[134358]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:23.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:22 smithi156 ceph-mon[134358]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T22:08:23.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:22 smithi156 ceph-mon[134358]: from='osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:08:23.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:22 smithi156 ceph-mon[134358]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T22:08:23.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:22 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:23.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:22 smithi156 ceph-mon[134358]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:08:23.461 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T22:08:23.461 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T22:08:23.461 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-06T22:08:23.461 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-06T22:08:23.461 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0310091 s, 338 MB/s 2023-10-06T22:08:23.462 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 2023-10-06T22:08:23.462 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-06T22:08:23.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:23.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:08:23.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:23.972 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_3 2023-10-06T22:08:24.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:08:24.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T22:08:24.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:24.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:24.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:24.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:24.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:24.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:24.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:24.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:24.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:24.711 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:08:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:08:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:08:24.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi190 2023-10-06T22:08:24.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi190 to 17168M 2023-10-06T22:08:24.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:24.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:24.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] boot 2023-10-06T22:08:24.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T22:08:24.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:24.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:24 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:08:25.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi190 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi190 to 17168M 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] boot 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:24 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:08:25.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:08:25.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:08:25.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi190 2023-10-06T22:08:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi190 to 17168M 2023-10-06T22:08:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T22:08:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: osd.2 [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] boot 2023-10-06T22:08:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T22:08:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:08:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:24 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:08:25.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:25 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T22:08:25.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:25 smithi057 ceph-mon[98492]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T22:08:25.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:25 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:08:26.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:25 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T22:08:26.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:25 smithi156 ceph-mon[134358]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T22:08:26.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:25 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:08:26.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:25 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T22:08:26.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:25 smithi190 ceph-mon[134891]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T22:08:26.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:25 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 -- 172.21.15.57:0/855723186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c108e60 msgr2=0x7f3b0c10b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 --2- 172.21.15.57:0/855723186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c108e60 0x7f3b0c10b2f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400b0a0 tx=0x7f3b0400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 -- 172.21.15.57:0/855723186 shutdown_connections 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 --2- 172.21.15.57:0/855723186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c108e60 0x7f3b0c10b2f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 --2- 172.21.15.57:0/855723186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c074ee0 0x7f3b0c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 --2- 172.21.15.57:0/855723186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 0x7f3b0c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 -- 172.21.15.57:0/855723186 >> 172.21.15.57:0/855723186 conn(0x7f3b0c0fc4f0 msgr2=0x7f3b0c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:08:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 -- 172.21.15.57:0/855723186 shutdown_connections 2023-10-06T22:08:26.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.357+0000 7f3b13c71700 1 -- 172.21.15.57:0/855723186 wait complete. 2023-10-06T22:08:26.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.358+0000 7f3b13c71700 1 Processor -- start 2023-10-06T22:08:26.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b13c71700 1 -- start start 2023-10-06T22:08:26.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b13c71700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 0x7f3b0c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b13c71700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c074ee0 0x7f3b0c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b13c71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c108e60 0x7f3b0c1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b13c71700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c1a0b30 con 0x7f3b0c073910 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b13c71700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c1a0c70 con 0x7f3b0c108e60 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b13c71700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c1a0e00 con 0x7f3b0c074ee0 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b11a0d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 0x7f3b0c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b11a0d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 0x7f3b0c198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54958/0 (socket says 172.21.15.57:54958) 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b11a0d700 1 -- 172.21.15.57:0/1813380630 learned_addr learned my addr 172.21.15.57:0/1813380630 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.359+0000 7f3b1220e700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c108e60 0x7f3b0c1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b1120c700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c074ee0 0x7f3b0c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b11a0d700 1 -- 172.21.15.57:0/1813380630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c074ee0 msgr2=0x7f3b0c1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:26.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b11a0d700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c074ee0 0x7f3b0c1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:26.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b11a0d700 1 -- 172.21.15.57:0/1813380630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c108e60 msgr2=0x7f3b0c1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:26.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b11a0d700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c108e60 0x7f3b0c1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:26.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b11a0d700 1 -- 172.21.15.57:0/1813380630 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04009d00 con 0x7f3b0c073910 2023-10-06T22:08:26.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b11a0d700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 0x7f3b0c198fa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800e9a0 tx=0x7f3b0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:08:26.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08009790 con 0x7f3b0c073910 2023-10-06T22:08:26.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c1a10e0 con 0x7f3b0c073910 2023-10-06T22:08:26.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b0800c810 con 0x7f3b0c073910 2023-10-06T22:08:26.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.360+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080054c0 con 0x7f3b0c073910 2023-10-06T22:08:26.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.361+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c1a1630 con 0x7f3b0c073910 2023-10-06T22:08:26.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.362+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c107510 con 0x7f3b0c073910 2023-10-06T22:08:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.364+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f3b0800cc80 con 0x7f3b0c073910 2023-10-06T22:08:26.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.366+0000 7f3b02ffd700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3af80a13f0 0x7f3af80a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:26.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.366+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f3b08014070 con 0x7f3b0c073910 2023-10-06T22:08:26.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.366+0000 7f3b1120c700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3af80a13f0 0x7f3af80a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:26.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.366+0000 7f3b1120c700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3af80a13f0 0x7f3af80a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c074c10 tx=0x7f3afc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:08:26.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.369+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3b08096740 con 0x7f3b0c073910 2023-10-06T22:08:26.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:26.605+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b0c02d170 con 0x7f3af80a13f0 2023-10-06T22:08:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:26 smithi057 ceph-mon[98492]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:26 smithi057 ceph-mon[98492]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T22:08:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T22:08:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:26 smithi057 ceph-mon[98492]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T22:08:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:27.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:26 smithi156 ceph-mon[134358]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:26 smithi156 ceph-mon[134358]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T22:08:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T22:08:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:26 smithi156 ceph-mon[134358]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T22:08:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:26 smithi190 ceph-mon[134891]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:26 smithi190 ceph-mon[134891]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T22:08:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T22:08:27.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:26 smithi190 ceph-mon[134891]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T22:08:27.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:27.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:26 smithi057 sudo[122923]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T22:08:27.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:26 smithi057 sudo[122923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:27.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:26 smithi057 sudo[122923]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:27.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:27 smithi057 sudo[122923]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:27.849 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:27 smithi057 sudo[122969]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T22:08:27.849 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:27 smithi057 sudo[122969]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:27.850 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:27 smithi057 sudo[122969]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:27.850 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:08:27 smithi057 sudo[122969]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:27.850 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:27 smithi057 ceph-mon[98492]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:27.850 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:27 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:27.850 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:27 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:27.850 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:27 smithi057 ceph-mon[98492]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T22:08:27.850 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:27 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:28.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:27 smithi156 ceph-mon[134358]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:28.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:27 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:28.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:27 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:28.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:27 smithi156 ceph-mon[134358]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T22:08:28.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:27 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:28.005 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:27 smithi156 sudo[144790]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T22:08:28.005 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:27 smithi156 sudo[144790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:28.005 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:27 smithi156 sudo[144790]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:28.005 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:27 smithi156 sudo[144790]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:28.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:27 smithi190 ceph-mon[134891]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:28.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:27 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:28.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:27 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:28.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:27 smithi190 ceph-mon[134891]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T22:08:28.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:27 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:28.503 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:28 smithi156 sudo[144793]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T22:08:28.503 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:28 smithi156 sudo[144793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:28.504 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:28 smithi156 sudo[144793]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:28.504 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:08:28 smithi156 sudo[144793]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:28.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:28.595+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3b0800c980 con 0x7f3b0c073910 2023-10-06T22:08:28.600 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145332]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T22:08:28.600 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145332]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:28.600 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145332]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:28.600 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145332]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:29.069 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145335]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T22:08:29.069 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145335]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:29.069 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145335]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:29.070 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:08:28 smithi190 sudo[145335]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:29.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123081]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T22:08:29.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:29.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123081]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:29.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123081]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:29.976 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 ceph-mon[134891]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:29.976 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 ceph-mon[134891]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 ceph-mon[134891]: Cluster is now healthy 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 ceph-mon[134891]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 ceph-mon[134891]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 sudo[145338]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 sudo[145338]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:29.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 sudo[145338]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:30.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:29 smithi156 ceph-mon[134358]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:30.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:29 smithi156 ceph-mon[134358]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T22:08:30.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:29 smithi156 ceph-mon[134358]: Cluster is now healthy 2023-10-06T22:08:30.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:29 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:30.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:29 smithi156 ceph-mon[134358]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T22:08:30.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:29 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:30.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:29 smithi156 ceph-mon[134358]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123115]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T22:08:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123115]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 sudo[123115]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 ceph-mon[98492]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:30.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 ceph-mon[98492]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T22:08:30.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 ceph-mon[98492]: Cluster is now healthy 2023-10-06T22:08:30.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:30.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 ceph-mon[98492]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T22:08:30.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:30.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:29 smithi057 ceph-mon[98492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:30.226 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:29 smithi190 sudo[145338]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:30.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 sudo[145341]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T22:08:30.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 sudo[145341]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:30.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 sudo[145341]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:30.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 sudo[145341]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144796]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T22:08:30.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144796]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:30.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144796]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:30.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144796]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144799]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144799]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144799]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 sudo[144799]: pam_unix(sudo:session): session closed for user root 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1694886270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]: dispatch 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]': finished 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: osdmap e22: 4 total, 3 up, 4 in 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:30 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3449122556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:31.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1694886270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]: dispatch 2023-10-06T22:08:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]: dispatch 2023-10-06T22:08:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]': finished 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: osdmap e22: 4 total, 3 up, 4 in 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:30 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3449122556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:31.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1694886270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]: dispatch 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]: dispatch 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8df05b17-2d03-43d9-a57d-5abed0d51e65"}]': finished 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: osdmap e22: 4 total, 3 up, 4 in 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:31.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:31.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:08:31.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:08:31.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:08:31.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:30 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3449122556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:32.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:31 smithi156 ceph-mon[134358]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:31 smithi156 ceph-mon[134358]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:31 smithi156 ceph-mon[134358]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:32.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:31 smithi190 ceph-mon[134891]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:32.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:31 smithi190 ceph-mon[134891]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:32.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:31 smithi190 ceph-mon[134891]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:31 smithi057 ceph-mon[98492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T22:08:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:31 smithi057 ceph-mon[98492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T22:08:32.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:31 smithi057 ceph-mon[98492]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:34 smithi057 ceph-mon[98492]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-10-06T22:08:35.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:34 smithi156 ceph-mon[134358]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-10-06T22:08:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:34 smithi190 ceph-mon[134891]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-10-06T22:08:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:36 smithi057 ceph-mon[98492]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-10-06T22:08:37.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:36 smithi156 ceph-mon[134358]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-10-06T22:08:37.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:36 smithi190 ceph-mon[134891]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-10-06T22:08:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:38 smithi057 ceph-mon[98492]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-10-06T22:08:39.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:38 smithi156 ceph-mon[134358]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-10-06T22:08:39.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:38 smithi190 ceph-mon[134891]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-10-06T22:08:39.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:39 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T22:08:39.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:39 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:40.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:39 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T22:08:40.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:39 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:39 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T22:08:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:39 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:40.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:40 smithi057 ceph-mon[98492]: Deploying daemon osd.3 on smithi057 2023-10-06T22:08:40.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:40 smithi057 ceph-mon[98492]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-06T22:08:41.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:40 smithi156 ceph-mon[134358]: Deploying daemon osd.3 on smithi057 2023-10-06T22:08:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:40 smithi156 ceph-mon[134358]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-06T22:08:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:40 smithi190 ceph-mon[134891]: Deploying daemon osd.3 on smithi057 2023-10-06T22:08:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:40 smithi190 ceph-mon[134891]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-06T22:08:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:42 smithi057 ceph-mon[98492]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 110 KiB/s wr, 7 op/s 2023-10-06T22:08:43.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:42 smithi156 ceph-mon[134358]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 110 KiB/s wr, 7 op/s 2023-10-06T22:08:43.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:42 smithi190 ceph-mon[134891]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 110 KiB/s wr, 7 op/s 2023-10-06T22:08:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:44 smithi057 ceph-mon[98492]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-06T22:08:45.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:44 smithi156 ceph-mon[134358]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-06T22:08:45.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:44 smithi190 ceph-mon[134891]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-06T22:08:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:46 smithi057 ceph-mon[98492]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:46.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:46 smithi156 ceph-mon[134358]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:46 smithi190 ceph-mon[134891]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:48.679 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 3 on host 'smithi057' 2023-10-06T22:08:48.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.676+0000 7f3b02ffd700 1 -- 172.21.15.57:0/1813380630 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3b0c02d170 con 0x7f3af80a13f0 2023-10-06T22:08:48.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.679+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3af80a13f0 msgr2=0x7f3af80a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:48.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.679+0000 7f3b13c71700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3af80a13f0 0x7f3af80a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c074c10 tx=0x7f3afc009260 comp rx=0 tx=0).stop 2023-10-06T22:08:48.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 msgr2=0x7f3b0c198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:48.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 0x7f3b0c198fa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800e9a0 tx=0x7f3b0800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:08:48.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 shutdown_connections 2023-10-06T22:08:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3af80a13f0 0x7f3af80a38b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b0c108e60 0x7f3b0c1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b0c073910 0x7f3b0c198fa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 --2- 172.21.15.57:0/1813380630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b0c074ee0 0x7f3b0c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 >> 172.21.15.57:0/1813380630 conn(0x7f3b0c0fc4f0 msgr2=0x7f3b0c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:08:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 shutdown_connections 2023-10-06T22:08:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:08:48.680+0000 7f3b13c71700 1 -- 172.21.15.57:0/1813380630 wait complete. 2023-10-06T22:08:48.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:48 smithi057 ceph-mon[98492]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:48.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:48.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:48.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:48.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:48.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:48 smithi156 ceph-mon[134358]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:49.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:49.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:48 smithi190 ceph-mon[134891]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.145 DEBUG:teuthology.orchestra.run.smithi057:osd.3> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-06T22:08:49.150 INFO:tasks.cephadm:Deploying osd.4 on smithi156 with /dev/vg_nvme/lv_3... 2023-10-06T22:08:49.150 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T22:08:49.226 INFO:journalctl@ceph.osd.3.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:08:49.673 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:08:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-3[125351]: 2023-10-06T22:08:49.382+0000 7f835b51a640 -1 osd.3 0 log_to_monitors true 2023-10-06T22:08:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:49.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:49 smithi057 ceph-mon[98492]: from='osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T22:08:49.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:49 smithi057 ceph-mon[98492]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T22:08:49.718 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:08:49.925 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.925 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:49.925 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:49.926 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:49 smithi156 ceph-mon[134358]: from='osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T22:08:49.926 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:49 smithi156 ceph-mon[134358]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T22:08:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:08:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:49 smithi190 ceph-mon[134891]: from='osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T22:08:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:49 smithi190 ceph-mon[134891]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T22:08:50.787 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-3[125351]: 2023-10-06T22:08:50.651+0000 7f8350390700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T22:08:50.787 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-3[125351]: 2023-10-06T22:08:50.659+0000 7f834a181700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:08:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-mon[98492]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-mon[98492]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T22:08:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-mon[98492]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T22:08:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-mon[98492]: from='osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:08:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:50 smithi057 ceph-mon[98492]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:08:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:50 smithi156 ceph-mon[134358]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:50 smithi156 ceph-mon[134358]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T22:08:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:50 smithi156 ceph-mon[134358]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T22:08:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:50 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:50 smithi156 ceph-mon[134358]: from='osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:08:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:50 smithi156 ceph-mon[134358]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:08:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:50 smithi190 ceph-mon[134891]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:50 smithi190 ceph-mon[134891]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T22:08:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:50 smithi190 ceph-mon[134891]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T22:08:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:50 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:50 smithi190 ceph-mon[134891]: from='osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:08:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:50 smithi190 ceph-mon[134891]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:08:51.175 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T22:08:51.175 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T22:08:51.175 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-10-06T22:08:51.175 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-10-06T22:08:51.176 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00831579 s, 1.3 GB/s 2023-10-06T22:08:51.176 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-10-06T22:08:51.558 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2023-10-06T22:08:51.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:51 smithi057 ceph-mon[98492]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:08:51.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:51 smithi057 ceph-mon[98492]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T22:08:51.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:51 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:51.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:51 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:52.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:51 smithi156 ceph-mon[134358]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:08:52.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:51 smithi156 ceph-mon[134358]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T22:08:52.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:51 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:52.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:51 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:51 smithi190 ceph-mon[134891]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:08:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:51 smithi190 ceph-mon[134891]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T22:08:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:51 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:52.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:51 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:52.115 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:08:52.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:08:52.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:08:52.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] boot 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: osdmap e25: 4 total, 4 up, 4 in 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:52.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:52.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:52.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:08:53.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:08:53.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:53.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] boot 2023-10-06T22:08:53.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: osdmap e25: 4 total, 4 up, 4 in 2023-10-06T22:08:53.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:53.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:53.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:53.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:53.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:53.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:08:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:08:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T22:08:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: osd.3 [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] boot 2023-10-06T22:08:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: osdmap e25: 4 total, 4 up, 4 in 2023-10-06T22:08:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:08:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:08:53.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:53.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:08:53.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 -- 172.21.15.156:0/3708616698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 msgr2=0x7f63bc101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 --2- 172.21.15.156:0/3708616698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc101880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009990 tx=0x7f63ac009ca0 comp rx=0 tx=0).stop 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 -- 172.21.15.156:0/3708616698 shutdown_connections 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 --2- 172.21.15.156:0/3708616698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc100b40 0x7f63bc108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 --2- 172.21.15.156:0/3708616698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc101880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 --2- 172.21.15.156:0/3708616698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63bc1001a0 0x7f63bc100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 -- 172.21.15.156:0/3708616698 >> 172.21.15.156:0/3708616698 conn(0x7f63bc0fabd0 msgr2=0x7f63bc0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:08:53.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 -- 172.21.15.156:0/3708616698 shutdown_connections 2023-10-06T22:08:53.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.295+0000 7f63c2644700 1 -- 172.21.15.156:0/3708616698 wait complete. 2023-10-06T22:08:53.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 Processor -- start 2023-10-06T22:08:53.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 -- start start 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1001a0 0x7f63bc196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63bc100b40 0x7f63bc197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19e870 con 0x7f63bc100b40 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19e9b0 con 0x7f63bc101460 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.296+0000 7f63c2644700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19eb70 con 0x7f63bc1001a0 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc19e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37538/0 (socket says 172.21.15.156:37538) 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 -- 172.21.15.156:0/3700875129 learned_addr learned my addr 172.21.15.156:0/3700875129 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-10-06T22:08:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63bbfff700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1001a0 0x7f63bc196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63bb7fe700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63bc100b40 0x7f63bc197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 -- 172.21.15.156:0/3700875129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1001a0 msgr2=0x7f63bc196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1001a0 0x7f63bc196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 -- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63bc100b40 msgr2=0x7f63bc197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63bc100b40 0x7f63bc197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 -- 172.21.15.156:0/3700875129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac0095f0 con 0x7f63bc101460 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c0be1700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc19e2a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63b400ef30 tx=0x7f63b400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63b97fa700 1 -- 172.21.15.156:0/3700875129 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4018070 con 0x7f63bc101460 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc19edf0 con 0x7f63bc101460 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63b97fa700 1 -- 172.21.15.156:0/3700875129 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b400f040 con 0x7f63bc101460 2023-10-06T22:08:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63b97fa700 1 -- 172.21.15.156:0/3700875129 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4003dc0 con 0x7f63bc101460 2023-10-06T22:08:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.297+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc19f280 con 0x7f63bc101460 2023-10-06T22:08:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.298+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc1075d0 con 0x7f63bc101460 2023-10-06T22:08:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.299+0000 7f63b97fa700 1 -- 172.21.15.156:0/3700875129 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f63b400caa0 con 0x7f63bc101460 2023-10-06T22:08:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.300+0000 7f63b97fa700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f63a80a1480 0x7f63a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:08:53.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.300+0000 7f63b97fa700 1 -- 172.21.15.156:0/3700875129 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f63b40cb870 con 0x7f63bc101460 2023-10-06T22:08:53.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.300+0000 7f63bbfff700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f63a80a1480 0x7f63a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:08:53.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.300+0000 7f63bbfff700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f63a80a1480 0x7f63a80a3940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:08:53.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.304+0000 7f63b97fa700 1 -- 172.21.15.156:0/3700875129 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f63b40960c0 con 0x7f63bc101460 2023-10-06T22:08:53.501 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:08:53.499+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f63bc02d350 con 0x7f63a80a1480 2023-10-06T22:08:53.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:53 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi057 2023-10-06T22:08:53.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:53 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi057 to 8636M 2023-10-06T22:08:53.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:53 smithi057 ceph-mon[98492]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T22:08:53.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:53 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:53 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:53 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:53 smithi057 ceph-mon[98492]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T22:08:54.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:53 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi057 2023-10-06T22:08:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:53 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi057 to 8636M 2023-10-06T22:08:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:53 smithi156 ceph-mon[134358]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T22:08:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:53 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:53 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:53 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:53 smithi156 ceph-mon[134358]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T22:08:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:53 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi057 2023-10-06T22:08:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:53 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi057 to 8636M 2023-10-06T22:08:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:53 smithi190 ceph-mon[134891]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T22:08:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:53 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:08:54.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:53 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:08:54.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:53 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:08:54.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:53 smithi190 ceph-mon[134891]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T22:08:54.990 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:54 smithi156 ceph-mon[134358]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:54.991 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:54 smithi156 ceph-mon[134358]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:54 smithi190 ceph-mon[134891]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:54 smithi190 ceph-mon[134891]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:55.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:54 smithi057 ceph-mon[98492]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:08:55.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:54 smithi057 ceph-mon[98492]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:56.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:55 smithi156 ceph-mon[134358]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T22:08:56.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:55 smithi190 ceph-mon[134891]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T22:08:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:55 smithi057 ceph-mon[98492]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T22:08:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T22:08:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/1238410139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]: dispatch 2023-10-06T22:08:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]: dispatch 2023-10-06T22:08:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]': finished 2023-10-06T22:08:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: osdmap e30: 5 total, 4 up, 5 in 2023-10-06T22:08:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:08:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:56 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/125344062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T22:08:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/1238410139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]: dispatch 2023-10-06T22:08:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]: dispatch 2023-10-06T22:08:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]': finished 2023-10-06T22:08:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: osdmap e30: 5 total, 4 up, 5 in 2023-10-06T22:08:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:08:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:56 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/125344062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T22:08:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/1238410139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]: dispatch 2023-10-06T22:08:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]: dispatch 2023-10-06T22:08:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48270831-4102-4536-b87e-6f3373e1125c"}]': finished 2023-10-06T22:08:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: osdmap e30: 5 total, 4 up, 5 in 2023-10-06T22:08:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:08:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:56 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/125344062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:08:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:08:58 smithi156 ceph-mon[134358]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:08:58 smithi190 ceph-mon[134891]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:08:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:08:58 smithi057 ceph-mon[98492]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:09:00.933 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:00 smithi156 ceph-mon[134358]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-10-06T22:09:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:00 smithi190 ceph-mon[134891]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-10-06T22:09:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:00 smithi057 ceph-mon[98492]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-10-06T22:09:03.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:02 smithi156 ceph-mon[134358]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-10-06T22:09:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T22:09:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:02 smithi190 ceph-mon[134891]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-10-06T22:09:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T22:09:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:02 smithi057 ceph-mon[98492]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-10-06T22:09:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T22:09:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:03.962 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:03 smithi156 ceph-mon[134358]: Deploying daemon osd.4 on smithi156 2023-10-06T22:09:03.962 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:03 smithi156 ceph-mon[134358]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:09:04.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:03 smithi190 ceph-mon[134891]: Deploying daemon osd.4 on smithi156 2023-10-06T22:09:04.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:03 smithi190 ceph-mon[134891]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:09:04.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:03 smithi057 ceph-mon[98492]: Deploying daemon osd.4 on smithi156 2023-10-06T22:09:04.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:03 smithi057 ceph-mon[98492]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:09:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:06 smithi057 ceph-mon[98492]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:09:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:07.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:06 smithi156 ceph-mon[134358]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:09:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:06 smithi190 ceph-mon[134891]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:09:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:08 smithi057 ceph-mon[98492]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-06T22:09:08.958 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:08 smithi156 ceph-mon[134358]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-06T22:09:09.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.012+0000 7f63b97fa700 1 -- 172.21.15.156:0/3700875129 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f63bc02d350 con 0x7f63a80a1480 2023-10-06T22:09:09.014 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 4 on host 'smithi156' 2023-10-06T22:09:09.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f63a80a1480 msgr2=0x7f63a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:09.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f63a80a1480 0x7f63a80a3940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac005fb0 comp rx=0 tx=0).stop 2023-10-06T22:09:09.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 msgr2=0x7f63bc19e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:09.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc19e2a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63b400ef30 tx=0x7f63b400c450 comp rx=0 tx=0).stop 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 shutdown_connections 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f63a80a1480 0x7f63a80a3940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63bc101460 0x7f63bc19e2a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1001a0 0x7f63bc196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 --2- 172.21.15.156:0/3700875129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63bc100b40 0x7f63bc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 >> 172.21.15.156:0/3700875129 conn(0x7f63bc0fabd0 msgr2=0x7f63bc104710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 shutdown_connections 2023-10-06T22:09:09.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:09:09.014+0000 7f63c2644700 1 -- 172.21.15.156:0/3700875129 wait complete. 2023-10-06T22:09:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:08 smithi190 ceph-mon[134891]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-06T22:09:09.415 DEBUG:teuthology.orchestra.run.smithi156:osd.4> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-06T22:09:09.417 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2023-10-06T22:09:09.417 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T22:09:09.449 INFO:journalctl@ceph.osd.4.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:09:09.986 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:09:10.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:10.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:10.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:10.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:09 smithi156 ceph-mon[134358]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T22:09:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:09 smithi190 ceph-mon[134891]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:10.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:09 smithi057 ceph-mon[98492]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T22:09:10.349 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:09:10 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:09:10.109+0000 7ff3a1e78640 -1 osd.4 0 log_to_monitors true 2023-10-06T22:09:11.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:11 smithi190 ceph-mon[134891]: from='osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T22:09:11.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:11 smithi190 ceph-mon[134891]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T22:09:11.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:11 smithi057 ceph-mon[98492]: from='osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T22:09:11.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:11 smithi057 ceph-mon[98492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T22:09:11.498 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:11 smithi156 ceph-mon[134358]: from='osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T22:09:11.499 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:11 smithi156 ceph-mon[134358]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T22:09:11.624 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T22:09:11.625 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T22:09:11.625 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-10-06T22:09:11.625 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-10-06T22:09:11.625 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109531 s, 957 MB/s 2023-10-06T22:09:11.625 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-10-06T22:09:12.047 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2023-10-06T22:09:12.253 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:09:12.032+0000 7ff396cee700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T22:09:12.254 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:09:12.039+0000 7ff392ae3700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:09:12.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T22:09:12.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T22:09:12.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:12.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:09:12.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:09:12.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:12.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T22:09:12.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:12.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:12.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T22:09:12.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T22:09:12.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:12.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:09:12.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:09:12.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:12.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:12.606 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:09:13.284 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:13 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi156 2023-10-06T22:09:13.285 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:13 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi156 to 8584M 2023-10-06T22:09:13.285 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:13 smithi190 ceph-mon[134891]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:09:13.285 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:13 smithi190 ceph-mon[134891]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T22:09:13.285 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:13.285 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:13.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:13 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi156 2023-10-06T22:09:13.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:13 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi156 to 8584M 2023-10-06T22:09:13.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:13 smithi057 ceph-mon[98492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:09:13.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:13 smithi057 ceph-mon[98492]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T22:09:13.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:13.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:13 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi156 2023-10-06T22:09:13.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:13 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi156 to 8584M 2023-10-06T22:09:13.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:13 smithi156 ceph-mon[134358]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:09:13.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:13 smithi156 ceph-mon[134358]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T22:09:13.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:13.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.006+0000 7f86c3da0700 1 -- 172.21.15.190:0/3647383811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc101990 msgr2=0x7f86bc1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.006+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3647383811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc101990 0x7f86bc1082b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f86b400b0a0 tx=0x7f86b400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- 172.21.15.190:0/3647383811 shutdown_connections 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3647383811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc101990 0x7f86bc1082b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3647383811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc0ffe70 0x7f86bc100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3647383811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86bc100fd0 0x7f86bc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- 172.21.15.190:0/3647383811 >> 172.21.15.190:0/3647383811 conn(0x7f86bc0fb3f0 msgr2=0x7f86bc0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- 172.21.15.190:0/3647383811 shutdown_connections 2023-10-06T22:09:14.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- 172.21.15.190:0/3647383811 wait complete. 2023-10-06T22:09:14.013 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 Processor -- start 2023-10-06T22:09:14.013 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- start start 2023-10-06T22:09:14.013 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc0ffe70 0x7f86bc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:14.013 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86bc100fd0 0x7f86bc19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:14.013 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101990 0x7f86bc1a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:14.013 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1a0810 con 0x7f86bc100fd0 2023-10-06T22:09:14.013 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1a0980 con 0x7f86bc101990 2023-10-06T22:09:14.014 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c3da0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1a0b40 con 0x7f86bc0ffe70 2023-10-06T22:09:14.014 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.007+0000 7f86c1b3c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc0ffe70 0x7f86bc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:14.014 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc0ffe70 0x7f86bc069500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46218/0 (socket says 172.21.15.190:46218) 2023-10-06T22:09:14.014 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 -- 172.21.15.190:0/3414322965 learned_addr learned my addr 172.21.15.190:0/3414322965 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:09:14.014 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c133b700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86bc100fd0 0x7f86bc19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:14.014 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c233d700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101990 0x7f86bc1a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:14.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 -- 172.21.15.190:0/3414322965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101990 msgr2=0x7f86bc1a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:14.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101990 0x7f86bc1a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:14.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 -- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86bc100fd0 msgr2=0x7f86bc19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:14.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86bc100fd0 0x7f86bc19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:14.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 -- 172.21.15.190:0/3414322965 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4009d00 con 0x7f86bc0ffe70 2023-10-06T22:09:14.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c233d700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101990 0x7f86bc1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:09:14.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c1b3c700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc0ffe70 0x7f86bc069500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c920 tx=0x7f86ac00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:09:14.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86b2ffd700 1 -- 172.21.15.190:0/3414322965 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac007b00 con 0x7f86bc0ffe70 2023-10-06T22:09:14.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc1a0e20 con 0x7f86bc0ffe70 2023-10-06T22:09:14.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86b2ffd700 1 -- 172.21.15.190:0/3414322965 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86ac004cf0 con 0x7f86bc0ffe70 2023-10-06T22:09:14.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86b2ffd700 1 -- 172.21.15.190:0/3414322965 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac005610 con 0x7f86bc0ffe70 2023-10-06T22:09:14.017 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.008+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc1a1290 con 0x7f86bc0ffe70 2023-10-06T22:09:14.018 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.009+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc1073e0 con 0x7f86bc0ffe70 2023-10-06T22:09:14.018 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.011+0000 7f86b2ffd700 1 -- 172.21.15.190:0/3414322965 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f86ac004750 con 0x7f86bc0ffe70 2023-10-06T22:09:14.018 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.011+0000 7f86b2ffd700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f86a80a1480 0x7f86a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:14.018 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.012+0000 7f86b2ffd700 1 -- 172.21.15.190:0/3414322965 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f86ac0cbcb0 con 0x7f86bc0ffe70 2023-10-06T22:09:14.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.012+0000 7f86c133b700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f86a80a1480 0x7f86a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:14.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.012+0000 7f86c133b700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f86a80a1480 0x7f86a80a3940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86bc069a80 tx=0x7f86b800b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:09:14.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.015+0000 7f86b2ffd700 1 -- 172.21.15.190:0/3414322965 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f86ac096380 con 0x7f86bc0ffe70 2023-10-06T22:09:14.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:14.212+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f86bc02d2d0 con 0x7f86a80a1480 2023-10-06T22:09:14.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:14 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:09:14.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:14 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:09:14.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:14 smithi190 ceph-mon[134891]: osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] boot 2023-10-06T22:09:14.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:14 smithi190 ceph-mon[134891]: osdmap e33: 5 total, 5 up, 5 in 2023-10-06T22:09:14.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:14 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:14.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:14 smithi190 ceph-mon[134891]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:14.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:14 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:09:14.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:14 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:09:14.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:14 smithi057 ceph-mon[98492]: osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] boot 2023-10-06T22:09:14.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:14 smithi057 ceph-mon[98492]: osdmap e33: 5 total, 5 up, 5 in 2023-10-06T22:09:14.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:14 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:14.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:14 smithi057 ceph-mon[98492]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:14.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:14 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:09:14.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:14 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:09:14.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:14 smithi156 ceph-mon[134358]: osd.4 [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] boot 2023-10-06T22:09:14.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:14 smithi156 ceph-mon[134358]: osdmap e33: 5 total, 5 up, 5 in 2023-10-06T22:09:14.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:14 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:09:14.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:14 smithi156 ceph-mon[134358]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:15.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:15 smithi190 ceph-mon[134891]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T22:09:15.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:15 smithi190 ceph-mon[134891]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:09:15.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:15 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:09:15.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:15 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:09:15.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:15 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:15 smithi057 ceph-mon[98492]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T22:09:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:15 smithi057 ceph-mon[98492]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:09:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:15 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:09:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:15 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:09:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:15 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:15.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:15 smithi156 ceph-mon[134358]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T22:09:15.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:15 smithi156 ceph-mon[134358]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:09:15.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:15 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:09:15.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:15 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:09:15.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:15 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:16 smithi057 ceph-mon[98492]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:17.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:16 smithi156 ceph-mon[134358]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:16 smithi190 ceph-mon[134891]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:17.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:17 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/386945552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]: dispatch 2023-10-06T22:09:17.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:17 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]: dispatch 2023-10-06T22:09:17.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:17 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]': finished 2023-10-06T22:09:17.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:17 smithi057 ceph-mon[98492]: osdmap e35: 6 total, 5 up, 6 in 2023-10-06T22:09:17.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:17 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:17.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:17 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/2825069261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:09:18.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:17 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/386945552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]: dispatch 2023-10-06T22:09:18.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:17 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]: dispatch 2023-10-06T22:09:18.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:17 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]': finished 2023-10-06T22:09:18.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:17 smithi156 ceph-mon[134358]: osdmap e35: 6 total, 5 up, 6 in 2023-10-06T22:09:18.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:17 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:18.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:17 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/2825069261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:09:18.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:17 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/386945552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]: dispatch 2023-10-06T22:09:18.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:17 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]: dispatch 2023-10-06T22:09:18.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:17 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6"}]': finished 2023-10-06T22:09:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:17 smithi190 ceph-mon[134891]: osdmap e35: 6 total, 5 up, 6 in 2023-10-06T22:09:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:17 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:18.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:17 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/2825069261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:09:19.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:18 smithi156 ceph-mon[134358]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:18 smithi190 ceph-mon[134891]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:19.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:18 smithi057 ceph-mon[98492]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:20.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:19 smithi190 ceph-mon[134891]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:19 smithi057 ceph-mon[98492]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:20.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:19 smithi156 ceph-mon[134358]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:22 smithi057 ceph-mon[98492]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:23.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:22 smithi156 ceph-mon[134358]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:23.061 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:22 smithi190 ceph-mon[134891]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:23.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T22:09:23.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T22:09:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:24.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T22:09:24.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:24.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:24 smithi190 ceph-mon[134891]: Deploying daemon osd.5 on smithi190 2023-10-06T22:09:24.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:24 smithi190 ceph-mon[134891]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:24 smithi057 ceph-mon[98492]: Deploying daemon osd.5 on smithi190 2023-10-06T22:09:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:24 smithi057 ceph-mon[98492]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:25.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:24 smithi156 ceph-mon[134358]: Deploying daemon osd.5 on smithi190 2023-10-06T22:09:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:24 smithi156 ceph-mon[134358]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:26.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:26 smithi190 ceph-mon[134891]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:26 smithi057 ceph-mon[98492]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:27.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:26 smithi156 ceph-mon[134358]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:28.895 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:28 smithi190 ceph-mon[134891]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:28.895 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:28.896 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:28.896 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:28 smithi057 ceph-mon[98492]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:28 smithi156 ceph-mon[134358]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:30.781 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2023-10-06T22:09:30.782 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.779+0000 7f86b2ffd700 1 -- 172.21.15.190:0/3414322965 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f86bc02d2d0 con 0x7f86a80a1480 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.781+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f86a80a1480 msgr2=0x7f86a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.781+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f86a80a1480 0x7f86a80a3940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86bc069a80 tx=0x7f86b800b510 comp rx=0 tx=0).stop 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.781+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc0ffe70 msgr2=0x7f86bc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.781+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc0ffe70 0x7f86bc069500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c920 tx=0x7f86ac00cce0 comp rx=0 tx=0).stop 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 shutdown_connections 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f86a80a1480 0x7f86a80a3940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc101990 0x7f86bc1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86bc0ffe70 0x7f86bc069500 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 --2- 172.21.15.190:0/3414322965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86bc100fd0 0x7f86bc19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 >> 172.21.15.190:0/3414322965 conn(0x7f86bc0fb3f0 msgr2=0x7f86bc104520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:09:30.784 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 shutdown_connections 2023-10-06T22:09:30.785 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:09:30.782+0000 7f86c3da0700 1 -- 172.21.15.190:0/3414322965 wait complete. 2023-10-06T22:09:30.908 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:30 smithi190 ceph-mon[134891]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:30.909 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:30.909 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:30.909 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:30.909 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:30.909 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:30 smithi057 ceph-mon[98492]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:30 smithi156 ceph-mon[134358]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:31.187 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-06T22:09:31.189 INFO:tasks.cephadm:Deploying osd.6 on smithi057 with /dev/vg_nvme/lv_2... 2023-10-06T22:09:31.190 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T22:09:31.230 INFO:journalctl@ceph.osd.5.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:09:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:09:32.057 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:09:31 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:09:31.743+0000 7fd740d15640 -1 osd.5 0 log_to_monitors true 2023-10-06T22:09:32.057 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:32.057 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:32.057 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:32.057 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:31 smithi190 ceph-mon[134891]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:32.058 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:31 smithi190 ceph-mon[134891]: from='osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T22:09:32.058 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:31 smithi190 ceph-mon[134891]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T22:09:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:31 smithi057 ceph-mon[98492]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:31 smithi057 ceph-mon[98492]: from='osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T22:09:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:31 smithi057 ceph-mon[98492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T22:09:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:31 smithi156 ceph-mon[134358]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:31 smithi156 ceph-mon[134358]: from='osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T22:09:32.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:31 smithi156 ceph-mon[134358]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T22:09:33.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:32 smithi190 ceph-mon[134891]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T22:09:33.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:32 smithi190 ceph-mon[134891]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T22:09:33.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:32 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:33.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:32 smithi190 ceph-mon[134891]: from='osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:09:33.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:32 smithi190 ceph-mon[134891]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:09:33.071 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:09:32 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:09:32.772+0000 7fd735b8b700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T22:09:33.071 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:09:32 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:09:32.781+0000 7fd72f97c700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:09:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:32 smithi057 ceph-mon[98492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T22:09:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:32 smithi057 ceph-mon[98492]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T22:09:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:32 smithi057 ceph-mon[98492]: from='osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:09:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:32 smithi057 ceph-mon[98492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:09:33.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:32 smithi156 ceph-mon[134358]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T22:09:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:32 smithi156 ceph-mon[134358]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T22:09:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:32 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:32 smithi156 ceph-mon[134358]: from='osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:09:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:32 smithi156 ceph-mon[134358]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:09:33.954 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T22:09:33.955 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T22:09:33.955 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-06T22:09:33.955 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-06T22:09:33.955 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0322937 s, 325 MB/s 2023-10-06T22:09:33.955 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-06T22:09:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:33 smithi190 ceph-mon[134891]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:09:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:33 smithi190 ceph-mon[134891]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T22:09:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:34.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:34.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:33 smithi190 ceph-mon[134891]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:34.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:33 smithi057 ceph-mon[98492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:09:34.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:33 smithi057 ceph-mon[98492]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T22:09:34.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:34.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:34.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:33 smithi057 ceph-mon[98492]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:34.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:33 smithi156 ceph-mon[134358]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:09:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:33 smithi156 ceph-mon[134358]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T22:09:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:33 smithi156 ceph-mon[134358]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T22:09:34.535 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_2 2023-10-06T22:09:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:09:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] boot 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: osdmap e38: 6 total, 6 up, 6 in 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi190 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:35.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi190 to 8584M 2023-10-06T22:09:35.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:35.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:35.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:09:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:09:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] boot 2023-10-06T22:09:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: osdmap e38: 6 total, 6 up, 6 in 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi190 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi190 to 8584M 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:35.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: osd.5 [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] boot 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: osdmap e38: 6 total, 6 up, 6 in 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi190 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:35.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:09:35.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi190 to 8584M 2023-10-06T22:09:35.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:35.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:35.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:35.277 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:09:36.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:35 smithi190 ceph-mon[134891]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T22:09:36.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:35 smithi190 ceph-mon[134891]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:36.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:35 smithi057 ceph-mon[98492]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T22:09:36.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:35 smithi057 ceph-mon[98492]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:36.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:35 smithi156 ceph-mon[134358]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T22:09:36.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:35 smithi156 ceph-mon[134358]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.777+0000 7f7ea094a700 1 -- 172.21.15.57:0/1312011664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102b10 msgr2=0x7f7e9c1093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.777+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1312011664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102b10 0x7f7e9c1093c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400b0d0 tx=0x7f7e9400b3e0 comp rx=0 tx=0).stop 2023-10-06T22:09:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.777+0000 7f7ea094a700 1 -- 172.21.15.57:0/1312011664 shutdown_connections 2023-10-06T22:09:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.777+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1312011664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102b10 0x7f7e9c1093c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:36.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.777+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1312011664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9c100f70 0x7f7e9c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:36.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.777+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1312011664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c102170 0x7f7e9c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:36.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.777+0000 7f7ea094a700 1 -- 172.21.15.57:0/1312011664 >> 172.21.15.57:0/1312011664 conn(0x7f7e9c0fc4d0 msgr2=0x7f7e9c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:09:36.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.778+0000 7f7ea094a700 1 -- 172.21.15.57:0/1312011664 shutdown_connections 2023-10-06T22:09:36.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.778+0000 7f7ea094a700 1 -- 172.21.15.57:0/1312011664 wait complete. 2023-10-06T22:09:36.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 Processor -- start 2023-10-06T22:09:36.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 -- start start 2023-10-06T22:09:36.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c100f70 0x7f7e9c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:36.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102170 0x7f7e9c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:36.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9c102b10 0x7f7e9c1a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1a0ab0 con 0x7f7e9c102170 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1a0bf0 con 0x7f7e9c102b10 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7ea094a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1a0d80 con 0x7f7e9c100f70 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7e99d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102170 0x7f7e9c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7e99d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102170 0x7f7e9c1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43588/0 (socket says 172.21.15.57:43588) 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7e9a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c100f70 0x7f7e9c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.779+0000 7f7e99d9b700 1 -- 172.21.15.57:0/1265192210 learned_addr learned my addr 172.21.15.57:0/1265192210 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e9ad9d700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9c102b10 0x7f7e9c1a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e99d9b700 1 -- 172.21.15.57:0/1265192210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c100f70 msgr2=0x7f7e9c198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e99d9b700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c100f70 0x7f7e9c198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e99d9b700 1 -- 172.21.15.57:0/1265192210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9c102b10 msgr2=0x7f7e9c1a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:36.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e99d9b700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9c102b10 0x7f7e9c1a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:36.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e99d9b700 1 -- 172.21.15.57:0/1265192210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e94009d00 con 0x7f7e9c102170 2023-10-06T22:09:36.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e99d9b700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102170 0x7f7e9c1994b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000d9b0 tx=0x7f7e9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:09:36.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e8b7fe700 1 -- 172.21.15.57:0/1265192210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90004220 con 0x7f7e9c102170 2023-10-06T22:09:36.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e8b7fe700 1 -- 172.21.15.57:0/1265192210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e90007640 con 0x7f7e9c102170 2023-10-06T22:09:36.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c1a1060 con 0x7f7e9c102170 2023-10-06T22:09:36.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.780+0000 7f7e8b7fe700 1 -- 172.21.15.57:0/1265192210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90005590 con 0x7f7e9c102170 2023-10-06T22:09:36.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.781+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c1a15b0 con 0x7f7e9c102170 2023-10-06T22:09:36.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.782+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c1935c0 con 0x7f7e9c102170 2023-10-06T22:09:36.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.784+0000 7f7e8b7fe700 1 -- 172.21.15.57:0/1265192210 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7e90004750 con 0x7f7e9c102170 2023-10-06T22:09:36.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.785+0000 7f7e8b7fe700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e840a1500 0x7f7e840a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:09:36.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.785+0000 7f7e8b7fe700 1 -- 172.21.15.57:0/1265192210 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f7e900cc120 con 0x7f7e9c102170 2023-10-06T22:09:36.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.786+0000 7f7e9a59c700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e840a1500 0x7f7e840a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:09:36.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.786+0000 7f7e9a59c700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e840a1500 0x7f7e840a39c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c101fd0 tx=0x7f7e8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:09:36.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:36.788+0000 7f7e8b7fe700 1 -- 172.21.15.57:0/1265192210 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e90096580 con 0x7f7e9c102170 2023-10-06T22:09:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:37.009+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e9c02d1c0 con 0x7f7e840a1500 2023-10-06T22:09:37.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:36 smithi190 ceph-mon[134891]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T22:09:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:36 smithi057 ceph-mon[98492]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T22:09:37.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:36 smithi156 ceph-mon[134358]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T22:09:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:37 smithi190 ceph-mon[134891]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T22:09:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:37 smithi190 ceph-mon[134891]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:09:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:37 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:09:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:37 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:09:38.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:37 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:38.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:37 smithi190 ceph-mon[134891]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:38.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:37 smithi190 ceph-mon[134891]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T22:09:38.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:37 smithi057 ceph-mon[98492]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T22:09:38.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:37 smithi057 ceph-mon[98492]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:09:38.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:37 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:09:38.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:37 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:09:38.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:37 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:38.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:37 smithi057 ceph-mon[98492]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:38.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:37 smithi057 ceph-mon[98492]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T22:09:38.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:37 smithi156 ceph-mon[134358]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T22:09:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:37 smithi156 ceph-mon[134358]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:09:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:37 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:09:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:37 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:09:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:37 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:37 smithi156 ceph-mon[134358]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:37 smithi156 ceph-mon[134358]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T22:09:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:40 smithi057 ceph-mon[98492]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2023-10-06T22:09:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:40 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3621987364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]: dispatch 2023-10-06T22:09:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:40 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]: dispatch 2023-10-06T22:09:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:40 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]': finished 2023-10-06T22:09:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:40 smithi057 ceph-mon[98492]: osdmap e43: 7 total, 6 up, 7 in 2023-10-06T22:09:40.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:09:41.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:40 smithi156 ceph-mon[134358]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2023-10-06T22:09:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:40 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3621987364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]: dispatch 2023-10-06T22:09:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:40 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]: dispatch 2023-10-06T22:09:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:40 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]': finished 2023-10-06T22:09:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:40 smithi156 ceph-mon[134358]: osdmap e43: 7 total, 6 up, 7 in 2023-10-06T22:09:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:40 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:09:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:40 smithi190 ceph-mon[134891]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2023-10-06T22:09:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:40 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3621987364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]: dispatch 2023-10-06T22:09:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:40 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]: dispatch 2023-10-06T22:09:41.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:40 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a987a0ba-b9f4-43e5-be83-274279f3b00c"}]': finished 2023-10-06T22:09:41.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:40 smithi190 ceph-mon[134891]: osdmap e43: 7 total, 6 up, 7 in 2023-10-06T22:09:41.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:40 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:09:42.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:42 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3216412917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:09:42.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:42 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3216412917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:09:42.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:42 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3216412917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:09:43.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:43 smithi190 ceph-mon[134891]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T22:09:43.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:43 smithi057 ceph-mon[98492]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T22:09:43.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:43 smithi156 ceph-mon[134358]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T22:09:44.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:44 smithi190 ceph-mon[134891]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T22:09:44.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:44 smithi057 ceph-mon[98492]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T22:09:44.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:44 smithi156 ceph-mon[134358]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T22:09:46.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:46 smithi057 ceph-mon[98492]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:09:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:46 smithi156 ceph-mon[134358]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:09:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:46 smithi190 ceph-mon[134891]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:09:48.904 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:48 smithi057 ceph-mon[98492]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:48 smithi156 ceph-mon[134358]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:48 smithi190 ceph-mon[134891]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:49.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T22:09:49.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:50.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T22:09:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T22:09:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:50.906 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:50 smithi057 ceph-mon[98492]: Deploying daemon osd.6 on smithi057 2023-10-06T22:09:50.906 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:50 smithi057 ceph-mon[98492]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:51.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:50 smithi156 ceph-mon[134358]: Deploying daemon osd.6 on smithi057 2023-10-06T22:09:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:50 smithi156 ceph-mon[134358]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:50 smithi190 ceph-mon[134891]: Deploying daemon osd.6 on smithi057 2023-10-06T22:09:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:50 smithi190 ceph-mon[134891]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:52 smithi057 ceph-mon[98492]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:53.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:52 smithi156 ceph-mon[134358]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:52 smithi190 ceph-mon[134891]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:54.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:54 smithi057 ceph-mon[98492]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:55.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:54 smithi156 ceph-mon[134358]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:54 smithi190 ceph-mon[134891]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:56.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:56.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:56.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:56.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:56 smithi057 ceph-mon[98492]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:56.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:56.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:56.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:56.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:56 smithi156 ceph-mon[134358]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:56.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:56.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:56.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:56.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:56 smithi190 ceph-mon[134891]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:58.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.581+0000 7f7e8b7fe700 1 -- 172.21.15.57:0/1265192210 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e9c02d1c0 con 0x7f7e840a1500 2023-10-06T22:09:58.584 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 6 on host 'smithi057' 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e840a1500 msgr2=0x7f7e840a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e840a1500 0x7f7e840a39c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c101fd0 tx=0x7f7e8c009260 comp rx=0 tx=0).stop 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102170 msgr2=0x7f7e9c1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102170 0x7f7e9c1994b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000d9b0 tx=0x7f7e9000dcc0 comp rx=0 tx=0).stop 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 shutdown_connections 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e840a1500 0x7f7e840a39c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e9c102b10 0x7f7e9c1a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:58.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c100f70 0x7f7e9c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:58.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 --2- 172.21.15.57:0/1265192210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9c102170 0x7f7e9c1994b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:09:58.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.584+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 >> 172.21.15.57:0/1265192210 conn(0x7f7e9c0fc4d0 msgr2=0x7f7e9c107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:09:58.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.585+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 shutdown_connections 2023-10-06T22:09:58.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:09:58.585+0000 7f7ea094a700 1 -- 172.21.15.57:0/1265192210 wait complete. 2023-10-06T22:09:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:58.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:58 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:59.001 DEBUG:teuthology.orchestra.run.smithi057:osd.6> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-06T22:09:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:59.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:59.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:58 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:59.006 INFO:tasks.cephadm:Deploying osd.7 on smithi156 with /dev/vg_nvme/lv_2... 2023-10-06T22:09:59.006 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T22:09:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:09:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:09:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:09:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:09:59.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:58 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:09:59.080 INFO:journalctl@ceph.osd.6.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:09:59.423 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:09:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:09:59.194+0000 7fac2992f640 -1 osd.6 0 log_to_monitors true 2023-10-06T22:09:59.587 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:09:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:59 smithi057 ceph-mon[98492]: from='osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T22:09:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:09:59 smithi057 ceph-mon[98492]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T22:09:59.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:59 smithi156 ceph-mon[134358]: from='osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T22:09:59.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:09:59 smithi156 ceph-mon[134358]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T22:10:00.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:59 smithi190 ceph-mon[134891]: from='osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T22:10:00.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:09:59 smithi190 ceph-mon[134891]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T22:10:00.923 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:10:00.646+0000 7fac1e7a5700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T22:10:00.923 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:10:00.659+0000 7fac18596700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:10:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-mon[98492]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:10:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-mon[98492]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T22:10:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-mon[98492]: from='osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:10:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-mon[98492]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T22:10:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-mon[98492]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:10:00.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:00 smithi057 ceph-mon[98492]: overall HEALTH_OK 2023-10-06T22:10:01.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:00 smithi156 ceph-mon[134358]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:10:01.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:00 smithi156 ceph-mon[134358]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T22:10:01.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:00 smithi156 ceph-mon[134358]: from='osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:10:01.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:00 smithi156 ceph-mon[134358]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T22:10:01.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:00 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:01.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:00 smithi156 ceph-mon[134358]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:10:01.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:00 smithi156 ceph-mon[134358]: overall HEALTH_OK 2023-10-06T22:10:01.009 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T22:10:01.009 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T22:10:01.009 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-10-06T22:10:01.009 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-10-06T22:10:01.010 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00829004 s, 1.3 GB/s 2023-10-06T22:10:01.010 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-10-06T22:10:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:00 smithi190 ceph-mon[134891]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:10:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:00 smithi190 ceph-mon[134891]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T22:10:01.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:00 smithi190 ceph-mon[134891]: from='osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:10:01.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:00 smithi190 ceph-mon[134891]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T22:10:01.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:00 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:01.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:00 smithi190 ceph-mon[134891]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:10:01.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:00 smithi190 ceph-mon[134891]: overall HEALTH_OK 2023-10-06T22:10:01.435 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2023-10-06T22:10:01.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:01 smithi156 ceph-mon[134358]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:10:01.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:01 smithi156 ceph-mon[134358]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T22:10:01.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:01.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:01 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:01.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:01 smithi057 ceph-mon[98492]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:10:01.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:01 smithi057 ceph-mon[98492]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T22:10:01.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:01.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:01 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:01.993 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:10:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:01 smithi190 ceph-mon[134891]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:10:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:01 smithi190 ceph-mon[134891]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T22:10:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:02.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:01 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:10:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:10:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:10:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] boot 2023-10-06T22:10:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: osdmap e46: 7 total, 7 up, 7 in 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:02.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:02 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:02.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:10:02.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:10:02.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:10:02.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] boot 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: osdmap e46: 7 total, 7 up, 7 in 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:02.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:02.989 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:02.989 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:02 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:10:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: osd.6 [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] boot 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: osdmap e46: 7 total, 7 up, 7 in 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:03.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:03.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:03.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:03.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:03.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:02 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:03.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 -- 172.21.15.156:0/197686725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20102190 msgr2=0x7f7e201025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:03.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 --2- 172.21.15.156:0/197686725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20102190 0x7f7e201025f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7e100099c0 tx=0x7f7e10009cd0 comp rx=0 tx=0).stop 2023-10-06T22:10:03.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 -- 172.21.15.156:0/197686725 shutdown_connections 2023-10-06T22:10:03.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 --2- 172.21.15.156:0/197686725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e20102b30 0x7f7e201093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:03.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 --2- 172.21.15.156:0/197686725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e20100f90 0x7f7e201013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:03.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 --2- 172.21.15.156:0/197686725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20102190 0x7f7e201025f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:03.200 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 -- 172.21.15.156:0/197686725 >> 172.21.15.156:0/197686725 conn(0x7f7e200fc4f0 msgr2=0x7f7e200fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 -- 172.21.15.156:0/197686725 shutdown_connections 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.195+0000 7f7e25be1700 1 -- 172.21.15.156:0/197686725 wait complete. 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 Processor -- start 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 -- start start 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20100f90 0x7f7e20074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e20102190 0x7f7e200732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e20102b30 0x7f7e200737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20073f60 con 0x7f7e20102190 2023-10-06T22:10:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200740d0 con 0x7f7e20100f90 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e25be1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201a0d10 con 0x7f7e20102b30 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20100f90 0x7f7e20074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20100f90 0x7f7e20074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37648/0 (socket says 172.21.15.156:37648) 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e1f7fe700 1 -- 172.21.15.156:0/3826172923 learned_addr learned my addr 172.21.15.156:0/3826172923 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.196+0000 7f7e1ffff700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e20102b30 0x7f7e200737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1effd700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e20102190 0x7f7e200732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1f7fe700 1 -- 172.21.15.156:0/3826172923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e20102b30 msgr2=0x7f7e200737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1f7fe700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e20102b30 0x7f7e200737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1f7fe700 1 -- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e20102190 msgr2=0x7f7e200732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:03.202 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1f7fe700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e20102190 0x7f7e200732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1f7fe700 1 -- 172.21.15.156:0/3826172923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100095f0 con 0x7f7e20100f90 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1effd700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e20102190 0x7f7e200732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1f7fe700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20100f90 0x7f7e20074c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400c970 tx=0x7f7e1400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1cff9700 1 -- 172.21.15.156:0/3826172923 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14004220 con 0x7f7e20100f90 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1cff9700 1 -- 172.21.15.156:0/3826172923 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e14007d60 con 0x7f7e20100f90 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e201a0ff0 con 0x7f7e20100f90 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e1cff9700 1 -- 172.21.15.156:0/3826172923 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14003c90 con 0x7f7e20100f90 2023-10-06T22:10:03.203 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.197+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e201a1410 con 0x7f7e20100f90 2023-10-06T22:10:03.204 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.198+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2010c810 con 0x7f7e20100f90 2023-10-06T22:10:03.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.199+0000 7f7e1cff9700 1 -- 172.21.15.156:0/3826172923 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7e14004380 con 0x7f7e20100f90 2023-10-06T22:10:03.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.200+0000 7f7e1cff9700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e080a1500 0x7f7e080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:03.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.200+0000 7f7e1cff9700 1 -- 172.21.15.156:0/3826172923 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f7e140cbeb0 con 0x7f7e20100f90 2023-10-06T22:10:03.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.200+0000 7f7e1effd700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e080a1500 0x7f7e080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:03.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.200+0000 7f7e1effd700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e080a1500 0x7f7e080a39c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006010 tx=0x7f7e10000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:10:03.205 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.203+0000 7f7e1cff9700 1 -- 172.21.15.156:0/3826172923 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e140962a0 con 0x7f7e20100f90 2023-10-06T22:10:03.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:03.402+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e200612f0 con 0x7f7e080a1500 2023-10-06T22:10:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:03 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi057 2023-10-06T22:10:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:03 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi057 to 5757M 2023-10-06T22:10:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:03 smithi057 ceph-mon[98492]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T22:10:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:03 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:03 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:03 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:04.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:03 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi057 2023-10-06T22:10:04.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:03 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi057 to 5757M 2023-10-06T22:10:04.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:03 smithi156 ceph-mon[134358]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T22:10:04.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:03 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:04.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:03 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:04.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:03 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:04.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:03 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi057 2023-10-06T22:10:04.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:03 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi057 to 5757M 2023-10-06T22:10:04.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:03 smithi190 ceph-mon[134891]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T22:10:04.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:03 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:04.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:03 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:04.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:03 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:04.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:04 smithi156 ceph-mon[134358]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:04.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:04 smithi156 ceph-mon[134358]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:04.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:04 smithi057 ceph-mon[98492]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:04.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:04 smithi057 ceph-mon[98492]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:04 smithi190 ceph-mon[134891]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:04 smithi190 ceph-mon[134891]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:06 smithi057 ceph-mon[98492]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:06 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/110635885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]: dispatch 2023-10-06T22:10:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:06 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]: dispatch 2023-10-06T22:10:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:06 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]': finished 2023-10-06T22:10:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:06 smithi057 ceph-mon[98492]: osdmap e48: 8 total, 7 up, 8 in 2023-10-06T22:10:06.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:06.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:06 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/3720924405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:06 smithi156 ceph-mon[134358]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:06 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/110635885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]: dispatch 2023-10-06T22:10:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:06 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]: dispatch 2023-10-06T22:10:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:06 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]': finished 2023-10-06T22:10:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:06 smithi156 ceph-mon[134358]: osdmap e48: 8 total, 7 up, 8 in 2023-10-06T22:10:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:07.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:06 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/3720924405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:06 smithi190 ceph-mon[134891]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:06 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/110635885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]: dispatch 2023-10-06T22:10:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:06 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]: dispatch 2023-10-06T22:10:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:06 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4b99ae3-8f83-4e52-a9c0-9536579dc262"}]': finished 2023-10-06T22:10:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:06 smithi190 ceph-mon[134891]: osdmap e48: 8 total, 7 up, 8 in 2023-10-06T22:10:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:06 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/3720924405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:08 smithi057 ceph-mon[98492]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:09.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:08 smithi156 ceph-mon[134358]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:08 smithi190 ceph-mon[134891]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:10.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:10 smithi156 ceph-mon[134358]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:10 smithi057 ceph-mon[98492]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:10 smithi190 ceph-mon[134891]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:12.879 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:12 smithi156 ceph-mon[134358]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:12.879 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T22:10:12.879 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:12 smithi057 ceph-mon[98492]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T22:10:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:12 smithi190 ceph-mon[134891]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T22:10:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:13 smithi057 ceph-mon[98492]: Deploying daemon osd.7 on smithi156 2023-10-06T22:10:13.986 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:13 smithi156 ceph-mon[134358]: Deploying daemon osd.7 on smithi156 2023-10-06T22:10:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:13 smithi190 ceph-mon[134891]: Deploying daemon osd.7 on smithi156 2023-10-06T22:10:14.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:14 smithi057 ceph-mon[98492]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:15.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:14 smithi156 ceph-mon[134358]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:14 smithi190 ceph-mon[134891]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:16 smithi057 ceph-mon[98492]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:16.994 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:16 smithi156 ceph-mon[134358]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:16.994 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:16.994 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:16.994 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:16 smithi190 ceph-mon[134891]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:18.667 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:18 smithi156 ceph-mon[134358]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:18.668 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:18.668 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:18.668 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:18.668 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:18.668 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:18 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:18.817 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.815+0000 7f7e1cff9700 1 -- 172.21.15.156:0/3826172923 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e200612f0 con 0x7f7e080a1500 2023-10-06T22:10:18.817 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 7 on host 'smithi156' 2023-10-06T22:10:18.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e080a1500 msgr2=0x7f7e080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:18.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e080a1500 0x7f7e080a39c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006010 tx=0x7f7e10000bc0 comp rx=0 tx=0).stop 2023-10-06T22:10:18.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20100f90 msgr2=0x7f7e20074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:18.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20100f90 0x7f7e20074c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400c970 tx=0x7f7e1400cc80 comp rx=0 tx=0).stop 2023-10-06T22:10:18.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 shutdown_connections 2023-10-06T22:10:18.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7e080a1500 0x7f7e080a39c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:18.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e20102b30 0x7f7e200737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:18.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e20100f90 0x7f7e20074c50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:18.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 --2- 172.21.15.156:0/3826172923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e20102190 0x7f7e200732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:18.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 >> 172.21.15.156:0/3826172923 conn(0x7f7e200fc4f0 msgr2=0x7f7e200fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:10:18.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 shutdown_connections 2023-10-06T22:10:18.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:10:18.817+0000 7f7e25be1700 1 -- 172.21.15.156:0/3826172923 wait complete. 2023-10-06T22:10:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:18 smithi057 ceph-mon[98492]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:18 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:18 smithi190 ceph-mon[134891]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:19.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:19.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:19.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:19.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:18 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:19.172 DEBUG:teuthology.orchestra.run.smithi156:osd.7> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-06T22:10:19.174 INFO:tasks.cephadm:Deploying osd.8 on smithi190 with /dev/vg_nvme/lv_2... 2023-10-06T22:10:19.174 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T22:10:19.203 INFO:journalctl@ceph.osd.7.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:10:19.743 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:10:19.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:19.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:19.978 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:19.979 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:19 smithi156 ceph-mon[134358]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:20.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:20.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:20.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:20.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:19 smithi190 ceph-mon[134891]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:20.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:19 smithi057 ceph-mon[98492]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:20.254 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:10:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:10:20.175+0000 7fdaef610640 -1 osd.7 0 log_to_monitors true 2023-10-06T22:10:21.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:20 smithi190 ceph-mon[134891]: from='osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T22:10:21.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:20 smithi190 ceph-mon[134891]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T22:10:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:20 smithi057 ceph-mon[98492]: from='osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T22:10:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:20 smithi057 ceph-mon[98492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T22:10:21.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:20 smithi156 ceph-mon[134358]: from='osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T22:10:21.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:20 smithi156 ceph-mon[134358]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T22:10:21.264 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T22:10:21.264 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T22:10:21.264 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-10-06T22:10:21.264 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-10-06T22:10:21.264 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116453 s, 900 MB/s 2023-10-06T22:10:21.264 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-10-06T22:10:21.703 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2023-10-06T22:10:22.004 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:10:21.786+0000 7fdae4486700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T22:10:22.004 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:10:21.802+0000 7fdade277700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:10:22.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T22:10:22.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:10:22.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T22:10:22.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:22.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi156 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi156 to 5722M 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:22.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:22.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:10:22.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T22:10:22.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:22.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi156 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi156 to 5722M 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T22:10:22.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T22:10:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:10:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T22:10:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:10:22.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi156 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi156 to 5722M 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:22.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:10:22.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T22:10:22.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:22.262 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:10:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:22 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:22 smithi190 ceph-mon[134891]: osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] boot 2023-10-06T22:10:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:22 smithi190 ceph-mon[134891]: osdmap e51: 8 total, 8 up, 8 in 2023-10-06T22:10:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:22 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:23.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:22 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:23.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:22 smithi057 ceph-mon[98492]: osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] boot 2023-10-06T22:10:23.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:22 smithi057 ceph-mon[98492]: osdmap e51: 8 total, 8 up, 8 in 2023-10-06T22:10:23.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:22 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:23.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:22 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:23.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:22 smithi156 ceph-mon[134358]: osd.7 [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] boot 2023-10-06T22:10:23.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:22 smithi156 ceph-mon[134358]: osdmap e51: 8 total, 8 up, 8 in 2023-10-06T22:10:23.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:22 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:10:23.690 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 -- 172.21.15.190:0/2669466662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e60101990 msgr2=0x7f4e601082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2669466662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e60101990 0x7f4e601082b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b0a0 tx=0x7f4e5800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 -- 172.21.15.190:0/2669466662 shutdown_connections 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2669466662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e60101990 0x7f4e601082b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2669466662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e600ffe70 0x7f4e60100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2669466662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e60100fd0 0x7f4e60101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 -- 172.21.15.190:0/2669466662 >> 172.21.15.190:0/2669466662 conn(0x7f4e600fb3f0 msgr2=0x7f4e600fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 -- 172.21.15.190:0/2669466662 shutdown_connections 2023-10-06T22:10:23.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.688+0000 7f4e672e0700 1 -- 172.21.15.190:0/2669466662 wait complete. 2023-10-06T22:10:23.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 Processor -- start 2023-10-06T22:10:23.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 -- start start 2023-10-06T22:10:23.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e600ffe70 0x7f4e60069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:23.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e60100fd0 0x7f4e6019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e60101990 0x7f4e6019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6019e790 con 0x7f4e60100fd0 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6019e900 con 0x7f4e60101990 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.689+0000 7f4e672e0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6019eac0 con 0x7f4e600ffe70 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e600ffe70 0x7f4e60069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e600ffe70 0x7f4e60069520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46334/0 (socket says 172.21.15.190:46334) 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 -- 172.21.15.190:0/2019005313 learned_addr learned my addr 172.21.15.190:0/2019005313 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6587d700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e60101990 0x7f4e6019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 -- 172.21.15.190:0/2019005313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e60101990 msgr2=0x7f4e6019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e60101990 0x7f4e6019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:23.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 -- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e60100fd0 msgr2=0x7f4e6019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e60100fd0 0x7f4e6019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 -- 172.21.15.190:0/2019005313 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58009d00 con 0x7f4e600ffe70 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6587d700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e60101990 0x7f4e6019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e6507c700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e600ffe70 0x7f4e60069520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400b530 tx=0x7f4e5400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e527fc700 1 -- 172.21.15.190:0/2019005313 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54011830 con 0x7f4e600ffe70 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e60195890 con 0x7f4e600ffe70 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e527fc700 1 -- 172.21.15.190:0/2019005313 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e54011e50 con 0x7f4e600ffe70 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e527fc700 1 -- 172.21.15.190:0/2019005313 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400f2e0 con 0x7f4e600ffe70 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.690+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e60195e60 con 0x7f4e600ffe70 2023-10-06T22:10:23.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.691+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6010b6c0 con 0x7f4e600ffe70 2023-10-06T22:10:23.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.692+0000 7f4e527fc700 1 -- 172.21.15.190:0/2019005313 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4e54010580 con 0x7f4e600ffe70 2023-10-06T22:10:23.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.693+0000 7f4e527fc700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4e4c0a1480 0x7f4e4c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:23.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.693+0000 7f4e527fc700 1 -- 172.21.15.190:0/2019005313 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f4e540cbcf0 con 0x7f4e600ffe70 2023-10-06T22:10:23.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.693+0000 7f4e6487b700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4e4c0a1480 0x7f4e4c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:23.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.694+0000 7f4e6487b700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4e4c0a1480 0x7f4e4c0a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e60069a80 tx=0x7f4e5c006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:10:23.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.697+0000 7f4e527fc700 1 -- 172.21.15.190:0/2019005313 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e540d0070 con 0x7f4e600ffe70 2023-10-06T22:10:23.902 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:23.900+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e6002d2e0 con 0x7f4e4c0a1480 2023-10-06T22:10:24.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:23 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:10:24.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:23 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:10:24.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:23 smithi190 ceph-mon[134891]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:24.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:23 smithi190 ceph-mon[134891]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T22:10:24.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:23 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:10:24.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:23 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:10:24.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:23 smithi057 ceph-mon[98492]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:24.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:23 smithi057 ceph-mon[98492]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T22:10:24.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:23 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:10:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:23 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:10:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:23 smithi156 ceph-mon[134358]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-06T22:10:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:23 smithi156 ceph-mon[134358]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T22:10:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:24 smithi190 ceph-mon[134891]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:24 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:24 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:24 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:25.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:24 smithi190 ceph-mon[134891]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T22:10:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:24 smithi057 ceph-mon[98492]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:24 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:24 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:24 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:24 smithi057 ceph-mon[98492]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T22:10:25.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:24 smithi156 ceph-mon[134358]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:25.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:24 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:25.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:24 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:25.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:24 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:25.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:24 smithi156 ceph-mon[134358]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T22:10:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:26 smithi057 ceph-mon[98492]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:26 smithi057 ceph-mon[98492]: osdmap e54: 8 total, 8 up, 8 in 2023-10-06T22:10:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:26 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/3388041613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]: dispatch 2023-10-06T22:10:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:26 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]: dispatch 2023-10-06T22:10:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:26 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]': finished 2023-10-06T22:10:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:26 smithi057 ceph-mon[98492]: osdmap e55: 9 total, 8 up, 9 in 2023-10-06T22:10:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:27.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:26 smithi156 ceph-mon[134358]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:26 smithi156 ceph-mon[134358]: osdmap e54: 8 total, 8 up, 8 in 2023-10-06T22:10:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:26 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/3388041613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]: dispatch 2023-10-06T22:10:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:26 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]: dispatch 2023-10-06T22:10:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:26 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]': finished 2023-10-06T22:10:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:26 smithi156 ceph-mon[134358]: osdmap e55: 9 total, 8 up, 9 in 2023-10-06T22:10:27.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:26 smithi190 ceph-mon[134891]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:26 smithi190 ceph-mon[134891]: osdmap e54: 8 total, 8 up, 8 in 2023-10-06T22:10:27.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:26 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/3388041613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]: dispatch 2023-10-06T22:10:27.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:26 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]: dispatch 2023-10-06T22:10:27.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:26 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae70781d-2670-4606-a280-604141f62861"}]': finished 2023-10-06T22:10:27.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:26 smithi190 ceph-mon[134891]: osdmap e55: 9 total, 8 up, 9 in 2023-10-06T22:10:27.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:27.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:27 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/2514670248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:28.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:27 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/2514670248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:28.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:27 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/2514670248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:28 smithi057 ceph-mon[98492]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:28 smithi156 ceph-mon[134358]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:29.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:28 smithi190 ceph-mon[134891]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:30.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:30 smithi057 ceph-mon[98492]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:31.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:30 smithi156 ceph-mon[134358]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:30 smithi190 ceph-mon[134891]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:32 smithi057 ceph-mon[98492]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:32.956 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:32 smithi190 ceph-mon[134891]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:33.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:32 smithi156 ceph-mon[134358]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:33.906 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T22:10:33.906 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:33.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T22:10:33.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:34.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T22:10:34.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:34.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:34 smithi190 ceph-mon[134891]: Deploying daemon osd.8 on smithi190 2023-10-06T22:10:34.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:34 smithi190 ceph-mon[134891]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:34 smithi057 ceph-mon[98492]: Deploying daemon osd.8 on smithi190 2023-10-06T22:10:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:34 smithi057 ceph-mon[98492]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:35.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:34 smithi156 ceph-mon[134358]: Deploying daemon osd.8 on smithi190 2023-10-06T22:10:35.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:34 smithi156 ceph-mon[134358]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:36 smithi057 ceph-mon[98492]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:37.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:36 smithi156 ceph-mon[134358]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:37.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:36 smithi190 ceph-mon[134891]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:37 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:37 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:37 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:38.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:37 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:38.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:37 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:38.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:37 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:37 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:37 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:37 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:38.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:38 smithi190 ceph-mon[134891]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:38 smithi057 ceph-mon[98492]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:39.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:38 smithi156 ceph-mon[134358]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:40.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.520+0000 7f4e527fc700 1 -- 172.21.15.190:0/2019005313 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e6002d2e0 con 0x7f4e4c0a1480 2023-10-06T22:10:40.523 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 8 on host 'smithi190' 2023-10-06T22:10:40.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4e4c0a1480 msgr2=0x7f4e4c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:40.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4e4c0a1480 0x7f4e4c0a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e60069a80 tx=0x7f4e5c006d20 comp rx=0 tx=0).stop 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e600ffe70 msgr2=0x7f4e60069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e600ffe70 0x7f4e60069520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400b530 tx=0x7f4e5400b8f0 comp rx=0 tx=0).stop 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 shutdown_connections 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4e4c0a1480 0x7f4e4c0a3940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e60101990 0x7f4e6019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e600ffe70 0x7f4e60069520 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 --2- 172.21.15.190:0/2019005313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e60100fd0 0x7f4e6019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 >> 172.21.15.190:0/2019005313 conn(0x7f4e600fb3f0 msgr2=0x7f4e601045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 shutdown_connections 2023-10-06T22:10:40.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:10:40.522+0000 7f4e672e0700 1 -- 172.21.15.190:0/2019005313 wait complete. 2023-10-06T22:10:40.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:40 smithi190 ceph-mon[134891]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:40.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:40 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:40.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:40 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:40.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:40 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:40.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:40 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:40 smithi057 ceph-mon[98492]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:41.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:40 smithi156 ceph-mon[134358]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:40 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:40 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:40 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:40 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:41.055 DEBUG:teuthology.orchestra.run.smithi190:osd.8> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.8.service 2023-10-06T22:10:41.057 INFO:tasks.cephadm:Deploying osd.9 on smithi057 with /dev/vg_nvme/lv_1... 2023-10-06T22:10:41.057 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T22:10:41.085 INFO:journalctl@ceph.osd.8.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:10:41.803 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:10:41.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:10:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:10:41.630+0000 7f9ed2c9c640 -1 osd.8 0 log_to_monitors true 2023-10-06T22:10:41.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:41 smithi190 ceph-mon[134891]: from='osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T22:10:41.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:41 smithi190 ceph-mon[134891]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T22:10:41.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:41 smithi057 ceph-mon[98492]: from='osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T22:10:41.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:41 smithi057 ceph-mon[98492]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T22:10:42.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:41 smithi156 ceph-mon[134358]: from='osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T22:10:42.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:41 smithi156 ceph-mon[134358]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T22:10:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:42 smithi057 ceph-mon[98492]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:42 smithi057 ceph-mon[98492]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T22:10:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:42 smithi057 ceph-mon[98492]: from='osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:10:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:42 smithi057 ceph-mon[98492]: osdmap e56: 9 total, 8 up, 9 in 2023-10-06T22:10:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:42 smithi057 ceph-mon[98492]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:10:43.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:42 smithi156 ceph-mon[134358]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:43.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:42 smithi156 ceph-mon[134358]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T22:10:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:42 smithi156 ceph-mon[134358]: from='osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:10:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:42 smithi156 ceph-mon[134358]: osdmap e56: 9 total, 8 up, 9 in 2023-10-06T22:10:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:42 smithi156 ceph-mon[134358]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:10:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-mon[134891]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-mon[134891]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T22:10:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-mon[134891]: from='osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:10:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-mon[134891]: osdmap e56: 9 total, 8 up, 9 in 2023-10-06T22:10:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-mon[134891]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:10:43.071 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:10:42.671+0000 7f9ec7b12700 -1 osd.8 0 waiting for initial osdmap 2023-10-06T22:10:43.071 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:10:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:10:42.686+0000 7f9ec1102700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:10:43.851 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T22:10:43.851 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T22:10:43.851 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-06T22:10:43.851 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-06T22:10:43.851 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0561135 s, 187 MB/s 2023-10-06T22:10:43.851 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-06T22:10:43.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:43 smithi057 ceph-mon[98492]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:10:43.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:43 smithi057 ceph-mon[98492]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T22:10:43.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:43.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:43.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:43 smithi057 ceph-mon[98492]: osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] boot 2023-10-06T22:10:43.997 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:43 smithi057 ceph-mon[98492]: osdmap e58: 9 total, 9 up, 9 in 2023-10-06T22:10:43.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:44.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:43 smithi156 ceph-mon[134358]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:10:44.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:43 smithi156 ceph-mon[134358]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T22:10:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:43 smithi156 ceph-mon[134358]: osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] boot 2023-10-06T22:10:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:43 smithi156 ceph-mon[134358]: osdmap e58: 9 total, 9 up, 9 in 2023-10-06T22:10:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:44.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:43 smithi190 ceph-mon[134891]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:10:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:43 smithi190 ceph-mon[134891]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T22:10:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:43 smithi190 ceph-mon[134891]: osd.8 [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] boot 2023-10-06T22:10:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:43 smithi190 ceph-mon[134891]: osdmap e58: 9 total, 9 up, 9 in 2023-10-06T22:10:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:10:44.384 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_1 2023-10-06T22:10:45.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi190 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi190 to 5722M 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:45.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:45.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:45.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:45.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:44 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:10:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:10:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi190 2023-10-06T22:10:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi190 to 5722M 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:45.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:45.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:45.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:45.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:45.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:44 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.123 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi190 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi190 to 5722M 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:10:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:44 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:10:45.963 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:45 smithi057 ceph-mon[98492]: osdmap e59: 9 total, 9 up, 9 in 2023-10-06T22:10:46.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:45 smithi156 ceph-mon[134358]: osdmap e59: 9 total, 9 up, 9 in 2023-10-06T22:10:46.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:45 smithi190 ceph-mon[134891]: osdmap e59: 9 total, 9 up, 9 in 2023-10-06T22:10:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.603+0000 7fc37631b700 1 -- 172.21.15.57:0/1999666510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370100f70 msgr2=0x7fc370101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.603+0000 7fc37631b700 1 --2- 172.21.15.57:0/1999666510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370100f70 0x7fc370101390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc3600099c0 tx=0x7fc360009cd0 comp rx=0 tx=0).stop 2023-10-06T22:10:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.604+0000 7fc37631b700 1 -- 172.21.15.57:0/1999666510 shutdown_connections 2023-10-06T22:10:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.604+0000 7fc37631b700 1 --2- 172.21.15.57:0/1999666510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370102b10 0x7fc3701093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.604+0000 7fc37631b700 1 --2- 172.21.15.57:0/1999666510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370100f70 0x7fc370101390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.604+0000 7fc37631b700 1 --2- 172.21.15.57:0/1999666510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370102170 0x7fc3701025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.604+0000 7fc37631b700 1 -- 172.21.15.57:0/1999666510 >> 172.21.15.57:0/1999666510 conn(0x7fc3700fc4d0 msgr2=0x7fc3700fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:10:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.604+0000 7fc37631b700 1 -- 172.21.15.57:0/1999666510 shutdown_connections 2023-10-06T22:10:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.604+0000 7fc37631b700 1 -- 172.21.15.57:0/1999666510 wait complete. 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.605+0000 7fc37631b700 1 Processor -- start 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.605+0000 7fc37631b700 1 -- start start 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc37631b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370100f70 0x7fc370196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc37631b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370102170 0x7fc3701971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc37631b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370102b10 0x7fc37019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc37631b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37019e800 con 0x7fc370102170 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc37631b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37019e940 con 0x7fc370102b10 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc37631b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37019eb00 con 0x7fc370100f70 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370102170 0x7fc3701971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370100f70 0x7fc370196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370100f70 0x7fc370196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52260/0 (socket says 172.21.15.57:52260) 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36ffff700 1 -- 172.21.15.57:0/32457959 learned_addr learned my addr 172.21.15.57:0/32457959 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36f7fe700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370102170 0x7fc3701971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53066/0 (socket says 172.21.15.57:53066) 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc3748b8700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370102b10 0x7fc37019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36ffff700 1 -- 172.21.15.57:0/32457959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370102b10 msgr2=0x7fc37019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36ffff700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370102b10 0x7fc37019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36ffff700 1 -- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370102170 msgr2=0x7fc3701971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.606+0000 7fc36ffff700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370102170 0x7fc3701971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc36ffff700 1 -- 172.21.15.57:0/32457959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600095f0 con 0x7fc370100f70 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc36f7fe700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370102170 0x7fc3701971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:10:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc36ffff700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370100f70 0x7fc370196c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3600036f0 tx=0x7fc360009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:10:46.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc36d7fa700 1 -- 172.21.15.57:0/32457959 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36001d070 con 0x7fc370100f70 2023-10-06T22:10:46.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37019ed80 con 0x7fc370100f70 2023-10-06T22:10:46.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc36d7fa700 1 -- 172.21.15.57:0/32457959 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc360003f90 con 0x7fc370100f70 2023-10-06T22:10:46.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc36d7fa700 1 -- 172.21.15.57:0/32457959 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360022e00 con 0x7fc370100f70 2023-10-06T22:10:46.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.607+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37019f190 con 0x7fc370100f70 2023-10-06T22:10:46.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.608+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc370108560 con 0x7fc370100f70 2023-10-06T22:10:46.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.611+0000 7fc36d7fa700 1 -- 172.21.15.57:0/32457959 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc360022500 con 0x7fc370100f70 2023-10-06T22:10:46.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.612+0000 7fc36d7fa700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3580a1500 0x7fc3580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:10:46.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.612+0000 7fc36d7fa700 1 -- 172.21.15.57:0/32457959 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fc3600cd870 con 0x7fc370100f70 2023-10-06T22:10:46.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.612+0000 7fc36f7fe700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3580a1500 0x7fc3580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:10:46.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.613+0000 7fc36f7fe700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3580a1500 0x7fc3580a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc370198190 tx=0x7fc36400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:10:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.614+0000 7fc36d7fa700 1 -- 172.21.15.57:0/32457959 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc360097850 con 0x7fc370100f70 2023-10-06T22:10:46.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:10:46.850+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc37002d2b0 con 0x7fc3580a1500 2023-10-06T22:10:47.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:46 smithi156 ceph-mon[134358]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:10:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:46 smithi156 ceph-mon[134358]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T22:10:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:46 smithi190 ceph-mon[134891]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:10:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:46 smithi190 ceph-mon[134891]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T22:10:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:46 smithi057 ceph-mon[98492]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:10:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:46 smithi057 ceph-mon[98492]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T22:10:47.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:47 smithi057 ceph-mon[98492]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T22:10:47.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:47 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:47.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:47 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:47.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:47 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:48.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:47 smithi156 ceph-mon[134358]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T22:10:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:47 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:47 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:47 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:48.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:47 smithi190 ceph-mon[134891]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T22:10:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:10:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:10:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:10:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:48 smithi057 ceph-mon[98492]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:48 smithi057 ceph-mon[98492]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:10:48.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:48 smithi057 ceph-mon[98492]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T22:10:49.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:48 smithi156 ceph-mon[134358]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:49.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:48 smithi156 ceph-mon[134358]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:10:49.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:48 smithi156 ceph-mon[134358]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T22:10:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:48 smithi190 ceph-mon[134891]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:10:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:48 smithi190 ceph-mon[134891]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:10:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:48 smithi190 ceph-mon[134891]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T22:10:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:50 smithi156 ceph-mon[134358]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-06T22:10:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:50 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1118245512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]: dispatch 2023-10-06T22:10:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:50 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]: dispatch 2023-10-06T22:10:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:50 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]': finished 2023-10-06T22:10:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:50 smithi156 ceph-mon[134358]: osdmap e63: 10 total, 9 up, 10 in 2023-10-06T22:10:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:50 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:10:51.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:50 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/254259155' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:50 smithi190 ceph-mon[134891]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-06T22:10:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:50 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1118245512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]: dispatch 2023-10-06T22:10:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:50 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]: dispatch 2023-10-06T22:10:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:50 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]': finished 2023-10-06T22:10:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:50 smithi190 ceph-mon[134891]: osdmap e63: 10 total, 9 up, 10 in 2023-10-06T22:10:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:50 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:10:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:50 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/254259155' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:50 smithi057 ceph-mon[98492]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-06T22:10:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:50 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1118245512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]: dispatch 2023-10-06T22:10:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:50 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]: dispatch 2023-10-06T22:10:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:50 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea6cc1cb-92e6-4cf1-a891-ca82e3e20079"}]': finished 2023-10-06T22:10:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:50 smithi057 ceph-mon[98492]: osdmap e63: 10 total, 9 up, 10 in 2023-10-06T22:10:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:50 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:10:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:50 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/254259155' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:10:52.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:52 smithi057 ceph-mon[98492]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-10-06T22:10:52.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:52 smithi156 ceph-mon[134358]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-10-06T22:10:52.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:52 smithi190 ceph-mon[134891]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-10-06T22:10:54.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:54 smithi057 ceph-mon[98492]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-10-06T22:10:55.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:54 smithi156 ceph-mon[134358]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-10-06T22:10:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:54 smithi190 ceph-mon[134891]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-10-06T22:10:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:56 smithi057 ceph-mon[98492]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:10:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:56 smithi156 ceph-mon[134358]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:10:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:56 smithi190 ceph-mon[134891]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:10:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:58 smithi057 ceph-mon[98492]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:10:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:58 smithi156 ceph-mon[134358]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:10:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:58 smithi190 ceph-mon[134891]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:10:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:59 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T22:10:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:10:59 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:00.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:59 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T22:11:00.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:10:59 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:00.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:59 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T22:11:00.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:10:59 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:01.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:00 smithi156 ceph-mon[134358]: Deploying daemon osd.9 on smithi057 2023-10-06T22:11:01.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:00 smithi156 ceph-mon[134358]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:00 smithi190 ceph-mon[134891]: Deploying daemon osd.9 on smithi057 2023-10-06T22:11:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:00 smithi190 ceph-mon[134891]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:00 smithi057 ceph-mon[98492]: Deploying daemon osd.9 on smithi057 2023-10-06T22:11:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:00 smithi057 ceph-mon[98492]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:02.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:02 smithi057 ceph-mon[98492]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:03.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:02 smithi156 ceph-mon[134358]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:02 smithi190 ceph-mon[134891]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:04.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:04 smithi057 ceph-mon[98492]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:05.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:04 smithi156 ceph-mon[134358]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:04 smithi190 ceph-mon[134891]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:06 smithi057 ceph-mon[98492]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:06.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:06.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:06.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:06.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:06 smithi156 ceph-mon[134358]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:06.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:06.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:06.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:06.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:06 smithi190 ceph-mon[134891]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:08.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.411+0000 7fc36d7fa700 1 -- 172.21.15.57:0/32457959 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc37002d2b0 con 0x7fc3580a1500 2023-10-06T22:11:08.414 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 9 on host 'smithi057' 2023-10-06T22:11:08.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.413+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3580a1500 msgr2=0x7fc3580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:08.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.413+0000 7fc37631b700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3580a1500 0x7fc3580a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc370198190 tx=0x7fc36400b410 comp rx=0 tx=0).stop 2023-10-06T22:11:08.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370100f70 msgr2=0x7fc370196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:08.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370100f70 0x7fc370196c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3600036f0 tx=0x7fc360009580 comp rx=0 tx=0).stop 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 shutdown_connections 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3580a1500 0x7fc3580a39c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc370102b10 0x7fc37019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc370100f70 0x7fc370196c80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 --2- 172.21.15.57:0/32457959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc370102170 0x7fc3701971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 >> 172.21.15.57:0/32457959 conn(0x7fc3700fc4d0 msgr2=0x7fc3701056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 shutdown_connections 2023-10-06T22:11:08.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:08.414+0000 7fc37631b700 1 -- 172.21.15.57:0/32457959 wait complete. 2023-10-06T22:11:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:08 smithi057 ceph-mon[98492]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:08.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:08 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:08.948 DEBUG:teuthology.orchestra.run.smithi057:osd.9> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.9.service 2023-10-06T22:11:08.953 INFO:tasks.cephadm:Deploying osd.10 on smithi156 with /dev/vg_nvme/lv_1... 2023-10-06T22:11:08.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T22:11:08.986 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:08 smithi156 ceph-mon[134358]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:08.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:08.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:08.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:08.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:08 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:09.028 INFO:journalctl@ceph.osd.9.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:11:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:08 smithi190 ceph-mon[134891]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:09.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:09.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:09.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:09.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:08 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:09.423 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:11:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:11:09.131+0000 7fdb24036640 -1 osd.9 0 log_to_monitors true 2023-10-06T22:11:09.525 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:11:09.901 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:09 smithi156 ceph-mon[134358]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T22:11:09.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:09 smithi057 ceph-mon[98492]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T22:11:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:09 smithi190 ceph-mon[134891]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T22:11:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:10 smithi057 ceph-mon[98492]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:10.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:10 smithi057 ceph-mon[98492]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T22:11:10.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:10 smithi057 ceph-mon[98492]: osdmap e64: 10 total, 9 up, 10 in 2023-10-06T22:11:10.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:10 smithi057 ceph-mon[98492]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:11:10.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:10 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:10.924 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:11:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:11:10.670+0000 7fdb18eac700 -1 osd.9 0 waiting for initial osdmap 2023-10-06T22:11:10.924 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:11:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:11:10.688+0000 7fdb154a2700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:11:11.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:10 smithi156 ceph-mon[134358]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:11.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:10 smithi156 ceph-mon[134358]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T22:11:11.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:10 smithi156 ceph-mon[134358]: osdmap e64: 10 total, 9 up, 10 in 2023-10-06T22:11:11.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:10 smithi156 ceph-mon[134358]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:11:11.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:10 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:10 smithi190 ceph-mon[134891]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:10 smithi190 ceph-mon[134891]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T22:11:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:10 smithi190 ceph-mon[134891]: osdmap e64: 10 total, 9 up, 10 in 2023-10-06T22:11:11.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:10 smithi190 ceph-mon[134891]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-06T22:11:11.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:10 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:11.095 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T22:11:11.095 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T22:11:11.095 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-10-06T22:11:11.096 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-10-06T22:11:11.096 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00828638 s, 1.3 GB/s 2023-10-06T22:11:11.096 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-10-06T22:11:11.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2023-10-06T22:11:12.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:11 smithi156 ceph-mon[134358]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:11:12.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:11 smithi156 ceph-mon[134358]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T22:11:12.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:12.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:11 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:12.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:11 smithi190 ceph-mon[134891]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:11:12.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:11 smithi190 ceph-mon[134891]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T22:11:12.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:12.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:11 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:12.080 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:11:12.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:11 smithi057 ceph-mon[98492]: from='osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-06T22:11:12.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:11 smithi057 ceph-mon[98492]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T22:11:12.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:12.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:11 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:11:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:11:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] boot 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: osdmap e66: 10 total, 10 up, 10 in 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:12.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:12.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:12.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] boot 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: osdmap e66: 10 total, 10 up, 10 in 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:13.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:13.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: osd.9 [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] boot 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: osdmap e66: 10 total, 10 up, 10 in 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:13.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:13.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:13.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 -- 172.21.15.156:0/1156550872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc101990 msgr2=0x7f59cc1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 --2- 172.21.15.156:0/1156550872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc101990 0x7f59cc1082b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b0a0 tx=0x7f59c400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 -- 172.21.15.156:0/1156550872 shutdown_connections 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 --2- 172.21.15.156:0/1156550872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc101990 0x7f59cc1082b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 --2- 172.21.15.156:0/1156550872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59cc0ffe70 0x7f59cc100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 --2- 172.21.15.156:0/1156550872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59cc100fd0 0x7f59cc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 -- 172.21.15.156:0/1156550872 >> 172.21.15.156:0/1156550872 conn(0x7f59cc0fb3f0 msgr2=0x7f59cc0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 -- 172.21.15.156:0/1156550872 shutdown_connections 2023-10-06T22:11:13.267 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.264+0000 7f59d115a700 1 -- 172.21.15.156:0/1156550872 wait complete. 2023-10-06T22:11:13.269 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 Processor -- start 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 -- start start 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc0ffe70 0x7f59cc104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59cc100fd0 0x7f59cc1026b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59cc101990 0x7f59cc102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc103370 con 0x7f59cc100fd0 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc1034e0 con 0x7f59cc0ffe70 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.265+0000 7f59d115a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc1a0d50 con 0x7f59cc101990 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc0ffe70 0x7f59cc104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc0ffe70 0x7f59cc104060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37806/0 (socket says 172.21.15.156:37806) 2023-10-06T22:11:13.270 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 -- 172.21.15.156:0/1538571986 learned_addr learned my addr 172.21.15.156:0/1538571986 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cb59e700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59cc101990 0x7f59cc102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59ca59c700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59cc100fd0 0x7f59cc1026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 -- 172.21.15.156:0/1538571986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59cc101990 msgr2=0x7f59cc102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59cc101990 0x7f59cc102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 -- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59cc100fd0 msgr2=0x7f59cc1026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59cc100fd0 0x7f59cc1026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 -- 172.21.15.156:0/1538571986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4009d00 con 0x7f59cc0ffe70 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59cad9d700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc0ffe70 0x7f59cc104060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00c920 tx=0x7f59bc00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59b3fff700 1 -- 172.21.15.156:0/1538571986 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc007b00 con 0x7f59cc0ffe70 2023-10-06T22:11:13.271 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59b3fff700 1 -- 172.21.15.156:0/1538571986 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59bc00ce70 con 0x7f59cc0ffe70 2023-10-06T22:11:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc1a0fd0 con 0x7f59cc0ffe70 2023-10-06T22:11:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59b3fff700 1 -- 172.21.15.156:0/1538571986 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc005460 con 0x7f59cc0ffe70 2023-10-06T22:11:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.266+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc1a1440 con 0x7f59cc0ffe70 2023-10-06T22:11:13.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.268+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc107eb0 con 0x7f59cc0ffe70 2023-10-06T22:11:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.268+0000 7f59b3fff700 1 -- 172.21.15.156:0/1538571986 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f59bc004750 con 0x7f59cc0ffe70 2023-10-06T22:11:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.269+0000 7f59b3fff700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f59b40a1480 0x7f59b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.269+0000 7f59b3fff700 1 -- 172.21.15.156:0/1538571986 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f59bc0cc120 con 0x7f59cc0ffe70 2023-10-06T22:11:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.269+0000 7f59ca59c700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f59b40a1480 0x7f59b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:13.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.269+0000 7f59ca59c700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f59b40a1480 0x7f59b40a3940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:13.275 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.273+0000 7f59b3fff700 1 -- 172.21.15.156:0/1538571986 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f59bc095f80 con 0x7f59cc0ffe70 2023-10-06T22:11:13.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:13.470+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f59cc02d250 con 0x7f59b40a1480 2023-10-06T22:11:14.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi057 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi057 to 4318M 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: osdmap e67: 10 total, 10 up, 10 in 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:11:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi057 2023-10-06T22:11:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi057 to 4318M 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: osdmap e67: 10 total, 10 up, 10 in 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:11:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi057 2023-10-06T22:11:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi057 to 4318M 2023-10-06T22:11:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: osdmap e67: 10 total, 10 up, 10 in 2023-10-06T22:11:14.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:14.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:14.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:14.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:14.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:11:14.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:11:14.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:14.968 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:14 smithi156 ceph-mon[134358]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:11:14.968 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:14 smithi156 ceph-mon[134358]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:14 smithi190 ceph-mon[134891]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:11:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:14 smithi190 ceph-mon[134891]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:14 smithi057 ceph-mon[98492]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:11:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:14 smithi057 ceph-mon[98492]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:16 smithi057 ceph-mon[98492]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:16 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/341287403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]: dispatch 2023-10-06T22:11:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:16 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]: dispatch 2023-10-06T22:11:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:16 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]': finished 2023-10-06T22:11:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:16 smithi057 ceph-mon[98492]: osdmap e68: 11 total, 10 up, 11 in 2023-10-06T22:11:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:16.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:16 smithi057 ceph-mon[98492]: from='client.? 172.21.15.156:0/2463537569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:11:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:16 smithi156 ceph-mon[134358]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:16 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/341287403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]: dispatch 2023-10-06T22:11:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:16 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]: dispatch 2023-10-06T22:11:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:16 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]': finished 2023-10-06T22:11:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:16 smithi156 ceph-mon[134358]: osdmap e68: 11 total, 10 up, 11 in 2023-10-06T22:11:17.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:17.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:16 smithi156 ceph-mon[134358]: from='client.? 172.21.15.156:0/2463537569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:11:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:16 smithi190 ceph-mon[134891]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:16 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/341287403' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]: dispatch 2023-10-06T22:11:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:16 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]: dispatch 2023-10-06T22:11:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:16 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d591b412-f98a-438b-bd2c-fd491fe9cf11"}]': finished 2023-10-06T22:11:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:16 smithi190 ceph-mon[134891]: osdmap e68: 11 total, 10 up, 11 in 2023-10-06T22:11:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:17.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:16 smithi190 ceph-mon[134891]: from='client.? 172.21.15.156:0/2463537569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:11:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:18 smithi057 ceph-mon[98492]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:19.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:18 smithi156 ceph-mon[134358]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:18 smithi190 ceph-mon[134891]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:20.868 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:20 smithi156 ceph-mon[134358]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:20.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:20 smithi057 ceph-mon[98492]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:21.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:20 smithi190 ceph-mon[134891]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:22 smithi057 ceph-mon[98492]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:22 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T22:11:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:22 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:22.951 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:22 smithi156 ceph-mon[134358]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:22.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:22 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T22:11:22.952 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:22 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:22 smithi190 ceph-mon[134891]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:22 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T22:11:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:22 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:23 smithi057 ceph-mon[98492]: Deploying daemon osd.10 on smithi156 2023-10-06T22:11:24.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:23 smithi156 ceph-mon[134358]: Deploying daemon osd.10 on smithi156 2023-10-06T22:11:24.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:23 smithi190 ceph-mon[134891]: Deploying daemon osd.10 on smithi156 2023-10-06T22:11:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:24 smithi057 ceph-mon[98492]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:25.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:24 smithi156 ceph-mon[134358]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:25.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:24 smithi190 ceph-mon[134891]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:26 smithi057 ceph-mon[98492]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:26.999 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:26 smithi156 ceph-mon[134358]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:26.999 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:26.999 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:26.999 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:26 smithi190 ceph-mon[134891]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:28.734 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 10 on host 'smithi156' 2023-10-06T22:11:28.735 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.732+0000 7f59b3fff700 1 -- 172.21.15.156:0/1538571986 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f59cc02d250 con 0x7f59b40a1480 2023-10-06T22:11:28.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f59b40a1480 msgr2=0x7f59b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:28.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f59b40a1480 0x7f59b40a3940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0008040 comp rx=0 tx=0).stop 2023-10-06T22:11:28.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc0ffe70 msgr2=0x7f59cc104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:28.736 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc0ffe70 0x7f59cc104060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00c920 tx=0x7f59bc00cc30 comp rx=0 tx=0).stop 2023-10-06T22:11:28.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 shutdown_connections 2023-10-06T22:11:28.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f59b40a1480 0x7f59b40a3940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:28.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59cc101990 0x7f59cc102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:28.737 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59cc0ffe70 0x7f59cc104060 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 --2- 172.21.15.156:0/1538571986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59cc100fd0 0x7f59cc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 >> 172.21.15.156:0/1538571986 conn(0x7f59cc0fb3f0 msgr2=0x7f59cc0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 shutdown_connections 2023-10-06T22:11:28.738 INFO:teuthology.orchestra.run.smithi156.stderr:2023-10-06T22:11:28.734+0000 7f59d115a700 1 -- 172.21.15.156:0/1538571986 wait complete. 2023-10-06T22:11:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:28 smithi057 ceph-mon[98492]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:29.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:28 smithi156 ceph-mon[134358]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:29.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:28 smithi190 ceph-mon[134891]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:29.176 DEBUG:teuthology.orchestra.run.smithi156:osd.10> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.10.service 2023-10-06T22:11:29.179 INFO:tasks.cephadm:Deploying osd.11 on smithi190 with /dev/vg_nvme/lv_1... 2023-10-06T22:11:29.179 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T22:11:29.208 INFO:journalctl@ceph.osd.10.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:11:29.748 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:11:29.900 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:29 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:29.900 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:29 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:29.900 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:29 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:29.901 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:29 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:29.970 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:29 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:29.970 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:29 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:29.970 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:29 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:29.970 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:29 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:29 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:29 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:29 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:29 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:30.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:11:30 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:11:30.139+0000 7f4bbb6e3640 -1 osd.10 0 log_to_monitors true 2023-10-06T22:11:30.840 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:30 smithi190 ceph-mon[134891]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:30.840 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:30 smithi190 ceph-mon[134891]: from='osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T22:11:30.840 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:30 smithi190 ceph-mon[134891]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T22:11:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:30 smithi156 ceph-mon[134358]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:30 smithi156 ceph-mon[134358]: from='osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T22:11:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:30 smithi156 ceph-mon[134358]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T22:11:31.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:30 smithi057 ceph-mon[98492]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:31.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:30 smithi057 ceph-mon[98492]: from='osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T22:11:31.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:30 smithi057 ceph-mon[98492]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T22:11:31.329 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T22:11:31.329 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T22:11:31.330 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-10-06T22:11:31.330 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-10-06T22:11:31.330 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112598 s, 931 MB/s 2023-10-06T22:11:31.330 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-10-06T22:11:31.793 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2023-10-06T22:11:31.799 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T22:11:31.799 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: osdmap e69: 11 total, 10 up, 11 in 2023-10-06T22:11:31.799 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:11:31.799 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi156 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:31.800 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:31.801 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:31.801 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi156 to 4292M 2023-10-06T22:11:31.801 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:31.801 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:31.801 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:31.801 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T22:11:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: osdmap e69: 11 total, 10 up, 11 in 2023-10-06T22:11:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi156 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi156 to 4292M 2023-10-06T22:11:32.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:32.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:32.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:31 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: osdmap e69: 11 total, 10 up, 11 in 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi156 2023-10-06T22:11:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi156 to 4292M 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:32.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:31 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:32.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:11:31.796+0000 7f4bb0559700 -1 osd.10 0 waiting for initial osdmap 2023-10-06T22:11:32.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:11:31 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:11:31.810+0000 7f4bac34e700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:11:32.355 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:11:32.954 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:11:32.955 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T22:11:32.955 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:32.955 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:32.955 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:32.955 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:32.955 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:32.955 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:32 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:11:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T22:11:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:33.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:33.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:33.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:33.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:32 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:33.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-10-06T22:11:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T22:11:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:33.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:32 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.669+0000 7f928fa7f700 1 -- 172.21.15.190:0/2032569102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92881007f0 msgr2=0x7f9288100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.669+0000 7f928fa7f700 1 --2- 172.21.15.190:0/2032569102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92881007f0 0x7f9288100c50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9284009990 tx=0x7f9284009ca0 comp rx=0 tx=0).stop 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- 172.21.15.190:0/2032569102 shutdown_connections 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 --2- 172.21.15.190:0/2032569102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9288101190 0x7f9288107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 --2- 172.21.15.190:0/2032569102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92880ff5f0 0x7f92880ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 --2- 172.21.15.190:0/2032569102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92881007f0 0x7f9288100c50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- 172.21.15.190:0/2032569102 >> 172.21.15.190:0/2032569102 conn(0x7f92880fab70 msgr2=0x7f92880fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:33.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- 172.21.15.190:0/2032569102 shutdown_connections 2023-10-06T22:11:33.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- 172.21.15.190:0/2032569102 wait complete. 2023-10-06T22:11:33.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 Processor -- start 2023-10-06T22:11:33.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- start start 2023-10-06T22:11:33.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92880ff5f0 0x7f92880fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:33.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92881007f0 0x7f928819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:33.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9288101190 0x7f928819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:33.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928819e740 con 0x7f92880ff5f0 2023-10-06T22:11:33.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928819e8b0 con 0x7f92881007f0 2023-10-06T22:11:33.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.670+0000 7f928fa7f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928819ea70 con 0x7f9288101190 2023-10-06T22:11:33.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9288101190 0x7f928819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:33.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9288101190 0x7f928819dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46486/0 (socket says 172.21.15.190:46486) 2023-10-06T22:11:33.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 -- 172.21.15.190:0/1479978543 learned_addr learned my addr 172.21.15.190:0/1479978543 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:11:33.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928d81b700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92880ff5f0 0x7f92880fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:33.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928d01a700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92881007f0 0x7f928819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:33.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 -- 172.21.15.190:0/1479978543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92881007f0 msgr2=0x7f928819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:33.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92881007f0 0x7f928819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 -- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92880ff5f0 msgr2=0x7f92880fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92880ff5f0 0x7f92880fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 -- 172.21.15.190:0/1479978543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840095f0 con 0x7f9288101190 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928d01a700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92881007f0 0x7f928819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928e01c700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9288101190 0x7f928819dfa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9280007d70 tx=0x7f928000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f927effd700 1 -- 172.21.15.190:0/1479978543 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000ad40 con 0x7f9288101190 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f927effd700 1 -- 172.21.15.190:0/1479978543 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92800128e0 con 0x7f9288101190 2023-10-06T22:11:33.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928819ed50 con 0x7f9288101190 2023-10-06T22:11:33.679 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f927effd700 1 -- 172.21.15.190:0/1479978543 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280011780 con 0x7f9288101190 2023-10-06T22:11:33.679 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.671+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928819f270 con 0x7f9288101190 2023-10-06T22:11:33.679 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.672+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9288107170 con 0x7f9288101190 2023-10-06T22:11:33.679 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.673+0000 7f927effd700 1 -- 172.21.15.190:0/1479978543 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9280012a50 con 0x7f9288101190 2023-10-06T22:11:33.680 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.674+0000 7f927effd700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f92740a1480 0x7f92740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:33.680 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.674+0000 7f927effd700 1 -- 172.21.15.190:0/1479978543 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f92800cca10 con 0x7f9288101190 2023-10-06T22:11:33.680 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.674+0000 7f928d81b700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f92740a1480 0x7f92740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:33.680 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.675+0000 7f928d81b700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f92740a1480 0x7f92740a3940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9278006fd0 tx=0x7f9278008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:33.680 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.678+0000 7f927effd700 1 -- 172.21.15.190:0/1479978543 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f92800966a0 con 0x7f9288101190 2023-10-06T22:11:33.883 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:33.881+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f928802d2b0 con 0x7f92740a1480 2023-10-06T22:11:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:33 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:11:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:33 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:11:34.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:33 smithi190 ceph-mon[134891]: osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] boot 2023-10-06T22:11:34.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:33 smithi190 ceph-mon[134891]: osdmap e71: 11 total, 11 up, 11 in 2023-10-06T22:11:34.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:34.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:33 smithi190 ceph-mon[134891]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:34.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:33 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:11:34.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:33 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:11:34.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:33 smithi057 ceph-mon[98492]: osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] boot 2023-10-06T22:11:34.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:33 smithi057 ceph-mon[98492]: osdmap e71: 11 total, 11 up, 11 in 2023-10-06T22:11:34.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:34.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:33 smithi057 ceph-mon[98492]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:34.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:33 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:11:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:33 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:11:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:33 smithi156 ceph-mon[134358]: osd.10 [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] boot 2023-10-06T22:11:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:33 smithi156 ceph-mon[134358]: osdmap e71: 11 total, 11 up, 11 in 2023-10-06T22:11:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:11:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:33 smithi156 ceph-mon[134358]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:34.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:34 smithi190 ceph-mon[134891]: osdmap e72: 11 total, 11 up, 11 in 2023-10-06T22:11:34.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:34 smithi190 ceph-mon[134891]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:11:34.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:11:34.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:11:34.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:34 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:34 smithi057 ceph-mon[98492]: osdmap e72: 11 total, 11 up, 11 in 2023-10-06T22:11:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:34 smithi057 ceph-mon[98492]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:11:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:11:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:11:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:34 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:34 smithi156 ceph-mon[134358]: osdmap e72: 11 total, 11 up, 11 in 2023-10-06T22:11:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:34 smithi156 ceph-mon[134358]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:11:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T22:11:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T22:11:35.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:34 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:36.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:35 smithi190 ceph-mon[134891]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T22:11:36.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:35 smithi190 ceph-mon[134891]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:36.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:35 smithi190 ceph-mon[134891]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T22:11:36.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:35 smithi057 ceph-mon[98492]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T22:11:36.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:35 smithi057 ceph-mon[98492]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:36.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:35 smithi057 ceph-mon[98492]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T22:11:36.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:35 smithi156 ceph-mon[134358]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T22:11:36.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:35 smithi156 ceph-mon[134358]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:36.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:35 smithi156 ceph-mon[134358]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T22:11:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:36 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/613450109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]: dispatch 2023-10-06T22:11:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:36 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]: dispatch 2023-10-06T22:11:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:36 smithi057 ceph-mon[98492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]': finished 2023-10-06T22:11:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:36 smithi057 ceph-mon[98492]: osdmap e75: 12 total, 11 up, 12 in 2023-10-06T22:11:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:37.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:36 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/613450109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]: dispatch 2023-10-06T22:11:37.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:36 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]: dispatch 2023-10-06T22:11:37.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:36 smithi156 ceph-mon[134358]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]': finished 2023-10-06T22:11:37.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:36 smithi156 ceph-mon[134358]: osdmap e75: 12 total, 11 up, 12 in 2023-10-06T22:11:37.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:36 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:36 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/613450109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]: dispatch 2023-10-06T22:11:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:36 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]: dispatch 2023-10-06T22:11:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:36 smithi190 ceph-mon[134891]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d41d0c-a9d4-45ea-9a40-6612ee98597e"}]': finished 2023-10-06T22:11:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:36 smithi190 ceph-mon[134891]: osdmap e75: 12 total, 11 up, 12 in 2023-10-06T22:11:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:36 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:38.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:37 smithi057 ceph-mon[98492]: from='client.? 172.21.15.190:0/1213981656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:11:38.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:37 smithi057 ceph-mon[98492]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:37 smithi156 ceph-mon[134358]: from='client.? 172.21.15.190:0/1213981656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:11:38.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:37 smithi156 ceph-mon[134358]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:38.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:37 smithi190 ceph-mon[134891]: from='client.? 172.21.15.190:0/1213981656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T22:11:38.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:37 smithi190 ceph-mon[134891]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:40.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:40 smithi190 ceph-mon[134891]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T22:11:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:40 smithi057 ceph-mon[98492]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T22:11:41.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:40 smithi156 ceph-mon[134358]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T22:11:42.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:42 smithi057 ceph-mon[98492]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T22:11:43.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:42 smithi156 ceph-mon[134358]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T22:11:43.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:42 smithi190 ceph-mon[134891]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T22:11:43.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T22:11:43.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:43.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T22:11:43.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:44.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T22:11:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:44.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:44 smithi190 ceph-mon[134891]: Deploying daemon osd.11 on smithi190 2023-10-06T22:11:44.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:44 smithi190 ceph-mon[134891]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:11:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:44 smithi057 ceph-mon[98492]: Deploying daemon osd.11 on smithi190 2023-10-06T22:11:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:44 smithi057 ceph-mon[98492]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:11:45.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:44 smithi156 ceph-mon[134358]: Deploying daemon osd.11 on smithi190 2023-10-06T22:11:45.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:44 smithi156 ceph-mon[134358]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T22:11:47.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:46 smithi156 ceph-mon[134358]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:11:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:46 smithi190 ceph-mon[134891]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:11:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:46 smithi057 ceph-mon[98492]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T22:11:49.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:48 smithi156 ceph-mon[134358]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-06T22:11:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:48 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:48 smithi190 ceph-mon[134891]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-06T22:11:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:48 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:48 smithi057 ceph-mon[98492]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-06T22:11:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:48 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:50.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:50 smithi190 ceph-mon[134891]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T22:11:51.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:50 smithi156 ceph-mon[134358]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T22:11:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:50 smithi057 ceph-mon[98492]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T22:11:51.350 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 11 on host 'smithi190' 2023-10-06T22:11:51.350 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.347+0000 7f927effd700 1 -- 172.21.15.190:0/1479978543 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f928802d2b0 con 0x7f92740a1480 2023-10-06T22:11:51.352 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f92740a1480 msgr2=0x7f92740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:51.352 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f92740a1480 0x7f92740a3940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9278006fd0 tx=0x7f9278008040 comp rx=0 tx=0).stop 2023-10-06T22:11:51.352 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9288101190 msgr2=0x7f928819dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:51.352 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9288101190 0x7f928819dfa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9280007d70 tx=0x7f928000a340 comp rx=0 tx=0).stop 2023-10-06T22:11:51.352 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 shutdown_connections 2023-10-06T22:11:51.352 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f92740a1480 0x7f92740a3940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:51.352 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9288101190 0x7f928819dfa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:51.353 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92880ff5f0 0x7f92880fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:51.353 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 --2- 172.21.15.190:0/1479978543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92881007f0 0x7f928819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:51.353 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 >> 172.21.15.190:0/1479978543 conn(0x7f92880fab70 msgr2=0x7f92881042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:51.353 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 shutdown_connections 2023-10-06T22:11:51.353 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:11:51.350+0000 7f928fa7f700 1 -- 172.21.15.190:0/1479978543 wait complete. 2023-10-06T22:11:51.852 DEBUG:teuthology.orchestra.run.smithi190:osd.11> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.11.service 2023-10-06T22:11:51.855 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-06T22:11:51.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-06T22:11:51.885 INFO:journalctl@ceph.osd.11.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:11:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:11:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:52 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:52 smithi057 ceph-mon[98492]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:52.710 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:11:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:11:52.376+0000 7fd0b37e1640 -1 osd.11 0 log_to_monitors true 2023-10-06T22:11:52.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:52 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:52 smithi190 ceph-mon[134891]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:52.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:52 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:52.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:52 smithi156 ceph-mon[134358]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:53.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:53 smithi057 ceph-mon[98492]: from='osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T22:11:53.420 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:53 smithi057 ceph-mon[98492]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T22:11:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:53 smithi190 ceph-mon[134891]: from='osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T22:11:53.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:53 smithi190 ceph-mon[134891]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T22:11:53.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:53 smithi156 ceph-mon[134358]: from='osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T22:11:53.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:53 smithi156 ceph-mon[134358]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T22:11:54.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 -- 172.21.15.57:0/1795477906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90102170 msgr2=0x7f8d901025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:54.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1795477906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90102170 0x7f8d901025d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009990 tx=0x7f8d84009ca0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 -- 172.21.15.57:0/1795477906 shutdown_connections 2023-10-06T22:11:54.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1795477906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d90102b10 0x7f8d901093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1795477906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90100f70 0x7f8d90101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1795477906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90102170 0x7f8d901025d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 -- 172.21.15.57:0/1795477906 >> 172.21.15.57:0/1795477906 conn(0x7f8d900fc4d0 msgr2=0x7f8d900fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:54.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 -- 172.21.15.57:0/1795477906 shutdown_connections 2023-10-06T22:11:54.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.115+0000 7f8d955ff700 1 -- 172.21.15.57:0/1795477906 wait complete. 2023-10-06T22:11:54.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.116+0000 7f8d955ff700 1 Processor -- start 2023-10-06T22:11:54.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.116+0000 7f8d955ff700 1 -- start start 2023-10-06T22:11:54.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d955ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90100f70 0x7f8d90196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:54.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d955ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90102170 0x7f8d901971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:54.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d955ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d90102b10 0x7f8d9019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d955ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019e810 con 0x7f8d90102b10 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d955ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019e950 con 0x7f8d90102170 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d955ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019eb10 con 0x7f8d90100f70 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90100f70 0x7f8d90196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d90102b10 0x7f8d9019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90100f70 0x7f8d90196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:55756/0 (socket says 172.21.15.57:55756) 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 -- 172.21.15.57:0/1326292406 learned_addr learned my addr 172.21.15.57:0/1326292406 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8e7fc700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90102170 0x7f8d901971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 -- 172.21.15.57:0/1326292406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90102170 msgr2=0x7f8d901971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90102170 0x7f8d901971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 -- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d90102b10 msgr2=0x7f8d9019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d90102b10 0x7f8d9019e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8effd700 1 -- 172.21.15.57:0/1326292406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840095f0 con 0x7f8d90100f70 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.117+0000 7f8d8e7fc700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90102170 0x7f8d901971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.118+0000 7f8d8effd700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90100f70 0x7f8d90196c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000d980 tx=0x7f8d8000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.118+0000 7f8d77fff700 1 -- 172.21.15.57:0/1326292406 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000b950 con 0x7f8d90100f70 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.118+0000 7f8d77fff700 1 -- 172.21.15.57:0/1326292406 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d80007620 con 0x7f8d90100f70 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.118+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9019edf0 con 0x7f8d90100f70 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.118+0000 7f8d77fff700 1 -- 172.21.15.57:0/1326292406 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000f3c0 con 0x7f8d90100f70 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.118+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9019f210 con 0x7f8d90100f70 2023-10-06T22:11:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.119+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d901085e0 con 0x7f8d90100f70 2023-10-06T22:11:54.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.121+0000 7f8d77fff700 1 -- 172.21.15.57:0/1326292406 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8d80010640 con 0x7f8d90100f70 2023-10-06T22:11:54.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.122+0000 7f8d77fff700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8d780a1480 0x7f8d780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:54.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.122+0000 7f8d77fff700 1 -- 172.21.15.57:0/1326292406 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f8d800cc770 con 0x7f8d90100f70 2023-10-06T22:11:54.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.122+0000 7f8d8e7fc700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8d780a1480 0x7f8d780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:54.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.123+0000 7f8d8e7fc700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8d780a1480 0x7f8d780a3940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400b2e0 tx=0x7f8d84005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:54.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.124+0000 7f8d77fff700 1 -- 172.21.15.57:0/1326292406 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8d80096300 con 0x7f8d90100f70 2023-10-06T22:11:54.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.335+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8d9010c7c0 con 0x7f8d90100f70 2023-10-06T22:11:54.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.336+0000 7f8d77fff700 1 -- 172.21.15.57:0/1326292406 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8d80099920 con 0x7f8d90100f70 2023-10-06T22:11:54.338 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:11:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8d780a1480 msgr2=0x7f8d780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8d780a1480 0x7f8d780a3940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400b2e0 tx=0x7f8d84005fd0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90100f70 msgr2=0x7f8d90196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90100f70 0x7f8d90196c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000d980 tx=0x7f8d8000dc90 comp rx=0 tx=0).stop 2023-10-06T22:11:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 shutdown_connections 2023-10-06T22:11:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8d780a1480 0x7f8d780a3940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d90102b10 0x7f8d9019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d90100f70 0x7f8d90196c90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 --2- 172.21.15.57:0/1326292406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d90102170 0x7f8d901971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 >> 172.21.15.57:0/1326292406 conn(0x7f8d900fc4d0 msgr2=0x7f8d90105720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 shutdown_connections 2023-10-06T22:11:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:54.339+0000 7f8d955ff700 1 -- 172.21.15.57:0/1326292406 wait complete. 2023-10-06T22:11:54.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:11:54.345+0000 7fd0a8657700 -1 osd.11 0 waiting for initial osdmap 2023-10-06T22:11:54.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:11:54.361+0000 7fd0a1c47700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T22:11:54.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-mon[134891]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T22:11:54.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-mon[134891]: from='osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:11:54.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-mon[134891]: osdmap e76: 12 total, 11 up, 12 in 2023-10-06T22:11:54.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:54.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-mon[134891]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:11:54.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-mon[134891]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:54.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:54 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1326292406' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T22:11:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:54 smithi057 ceph-mon[98492]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T22:11:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:54 smithi057 ceph-mon[98492]: from='osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:11:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:54 smithi057 ceph-mon[98492]: osdmap e76: 12 total, 11 up, 12 in 2023-10-06T22:11:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:54 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:54 smithi057 ceph-mon[98492]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:11:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:54 smithi057 ceph-mon[98492]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:54 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1326292406' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T22:11:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:54 smithi156 ceph-mon[134358]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T22:11:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:54 smithi156 ceph-mon[134358]: from='osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:11:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:54 smithi156 ceph-mon[134358]: osdmap e76: 12 total, 11 up, 12 in 2023-10-06T22:11:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:54 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:54 smithi156 ceph-mon[134358]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-10-06T22:11:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:54 smithi156 ceph-mon[134358]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:54 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1326292406' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T22:11:54.791 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1696630292,"num_in_osds":12,"osd_in_since":1696630296,"num_remapped_pgs":0} 2023-10-06T22:11:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:11:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: osdmap e77: 12 total, 11 up, 12 in 2023-10-06T22:11:55.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: Detected new or changed devices on smithi190 2023-10-06T22:11:55.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi190 to 4292M 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:55.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:55.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:55.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:55 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:11:55.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: osdmap e77: 12 total, 11 up, 12 in 2023-10-06T22:11:55.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: Detected new or changed devices on smithi190 2023-10-06T22:11:55.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi190 to 4292M 2023-10-06T22:11:55.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:55.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:55.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:55.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:55.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:55.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:55 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.792 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-06T22:11:55.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: osdmap e77: 12 total, 11 up, 12 in 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: Detected new or changed devices on smithi190 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi190 to 4292M 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:11:55.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:11:55.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:11:55.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:55 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:56.541 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:11:56.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:56 smithi057 ceph-mon[98492]: purged_snaps scrub starts 2023-10-06T22:11:56.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:56 smithi057 ceph-mon[98492]: purged_snaps scrub ok 2023-10-06T22:11:56.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:56 smithi057 ceph-mon[98492]: osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] boot 2023-10-06T22:11:56.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:56 smithi057 ceph-mon[98492]: osdmap e78: 12 total, 12 up, 12 in 2023-10-06T22:11:56.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:56 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:56.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:56 smithi057 ceph-mon[98492]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:56.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:56 smithi156 ceph-mon[134358]: purged_snaps scrub starts 2023-10-06T22:11:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:56 smithi156 ceph-mon[134358]: purged_snaps scrub ok 2023-10-06T22:11:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:56 smithi156 ceph-mon[134358]: osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] boot 2023-10-06T22:11:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:56 smithi156 ceph-mon[134358]: osdmap e78: 12 total, 12 up, 12 in 2023-10-06T22:11:57.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:56 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:57.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:56 smithi156 ceph-mon[134358]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:57.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:56 smithi190 ceph-mon[134891]: purged_snaps scrub starts 2023-10-06T22:11:57.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:56 smithi190 ceph-mon[134891]: purged_snaps scrub ok 2023-10-06T22:11:57.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:56 smithi190 ceph-mon[134891]: osd.11 [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] boot 2023-10-06T22:11:57.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:56 smithi190 ceph-mon[134891]: osdmap e78: 12 total, 12 up, 12 in 2023-10-06T22:11:57.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:56 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:11:57.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:56 smithi190 ceph-mon[134891]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T22:11:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:57 smithi057 ceph-mon[98492]: osdmap e79: 12 total, 12 up, 12 in 2023-10-06T22:11:57.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:57 smithi156 ceph-mon[134358]: osdmap e79: 12 total, 12 up, 12 in 2023-10-06T22:11:57.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:57 smithi190 ceph-mon[134891]: osdmap e79: 12 total, 12 up, 12 in 2023-10-06T22:11:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 -- 172.21.15.57:0/1020617061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 msgr2=0x7f0f80073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/1020617061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f80073340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009990 tx=0x7f0f74009ca0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 -- 172.21.15.57:0/1020617061 shutdown_connections 2023-10-06T22:11:58.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/1020617061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f80108e40 0x7f0f8010b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/1020617061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f80073340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/1020617061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 0x7f0f80073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 -- 172.21.15.57:0/1020617061 >> 172.21.15.57:0/1020617061 conn(0x7f0f800fc4d0 msgr2=0x7f0f800fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 -- 172.21.15.57:0/1020617061 shutdown_connections 2023-10-06T22:11:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.039+0000 7f0f86b1f700 1 -- 172.21.15.57:0/1020617061 wait complete. 2023-10-06T22:11:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 Processor -- start 2023-10-06T22:11:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 -- start start 2023-10-06T22:11:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 0x7f0f80198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f801993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f80108e40 0x7f0f801a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801a0a00 con 0x7f0f80074ee0 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801a0b40 con 0x7f0f80108e40 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.040+0000 7f0f86b1f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801a0cd0 con 0x7f0f80073910 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f801993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f848bb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 0x7f0f80198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f801993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51502/0 (socket says 172.21.15.57:51502) 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f848bb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 0x7f0f80198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:55778/0 (socket says 172.21.15.57:55778) 2023-10-06T22:11:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 -- 172.21.15.57:0/985797141 learned_addr learned my addr 172.21.15.57:0/985797141 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:11:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f850bc700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f80108e40 0x7f0f801a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 -- 172.21.15.57:0/985797141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 msgr2=0x7f0f80198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 0x7f0f80198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 -- 172.21.15.57:0/985797141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f80108e40 msgr2=0x7f0f801a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f80108e40 0x7f0f801a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 -- 172.21.15.57:0/985797141 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f740095f0 con 0x7f0f80074ee0 2023-10-06T22:11:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f848bb700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 0x7f0f80198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:11:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7ffff700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f801993b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00e9a0 tx=0x7f0f6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.041+0000 7f0f7dffb700 1 -- 172.21.15.57:0/985797141 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c009790 con 0x7f0f80074ee0 2023-10-06T22:11:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.042+0000 7f0f7dffb700 1 -- 172.21.15.57:0/985797141 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f6c00c810 con 0x7f0f80074ee0 2023-10-06T22:11:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.042+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f801a0fb0 con 0x7f0f80074ee0 2023-10-06T22:11:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.042+0000 7f0f7dffb700 1 -- 172.21.15.57:0/985797141 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c003c70 con 0x7f0f80074ee0 2023-10-06T22:11:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.042+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f801a1420 con 0x7f0f80074ee0 2023-10-06T22:11:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.043+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f801074f0 con 0x7f0f80074ee0 2023-10-06T22:11:58.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.045+0000 7f0f7dffb700 1 -- 172.21.15.57:0/985797141 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0f6c00cc80 con 0x7f0f80074ee0 2023-10-06T22:11:58.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.047+0000 7f0f7dffb700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0f700a1480 0x7f0f700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:11:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.047+0000 7f0f7dffb700 1 -- 172.21.15.57:0/985797141 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0f6c014070 con 0x7f0f80074ee0 2023-10-06T22:11:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.047+0000 7f0f848bb700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0f700a1480 0x7f0f700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:11:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.048+0000 7f0f848bb700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0f700a1480 0x7f0f700a3940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400b2e0 tx=0x7f0f74005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:11:58.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.051+0000 7f0f7dffb700 1 -- 172.21.15.57:0/985797141 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0f6c0964c0 con 0x7f0f80074ee0 2023-10-06T22:11:58.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.263+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0f8010e9c0 con 0x7f0f80074ee0 2023-10-06T22:11:58.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.263+0000 7f0f7dffb700 1 -- 172.21.15.57:0/985797141 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0f6c003dd0 con 0x7f0f80074ee0 2023-10-06T22:11:58.266 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:11:58.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0f700a1480 msgr2=0x7f0f700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:58.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0f700a1480 0x7f0f700a3940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400b2e0 tx=0x7f0f74005fb0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 msgr2=0x7f0f801993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:11:58.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f801993b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00e9a0 tx=0x7f0f6c00ecb0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 shutdown_connections 2023-10-06T22:11:58.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0f700a1480 0x7f0f700a3940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f80108e40 0x7f0f801a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f80073910 0x7f0f80198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 --2- 172.21.15.57:0/985797141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f80074ee0 0x7f0f801993b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:11:58.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 >> 172.21.15.57:0/985797141 conn(0x7f0f800fc4d0 msgr2=0x7f0f800fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:11:58.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 shutdown_connections 2023-10-06T22:11:58.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:11:58.266+0000 7f0f86b1f700 1 -- 172.21.15.57:0/985797141 wait complete. 2023-10-06T22:11:58.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:58 smithi057 ceph-mon[98492]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:11:58.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:11:58 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/985797141' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T22:11:58.720 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1696630315,"num_in_osds":12,"osd_in_since":1696630296,"num_remapped_pgs":0} 2023-10-06T22:11:58.721 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T22:11:58.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:58 smithi156 ceph-mon[134358]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:11:58.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:11:58 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/985797141' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T22:11:58.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:58 smithi190 ceph-mon[134891]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:11:58.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:11:58 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/985797141' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T22:11:59.460 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:12:00.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:00 smithi057 ceph-mon[98492]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.948+0000 7f1c50f35700 1 -- 172.21.15.57:0/885136214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 msgr2=0x7f1c4c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.948+0000 7f1c50f35700 1 --2- 172.21.15.57:0/885136214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c101400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c400099c0 tx=0x7f1c40009cd0 comp rx=0 tx=0).stop 2023-10-06T22:12:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.948+0000 7f1c50f35700 1 -- 172.21.15.57:0/885136214 shutdown_connections 2023-10-06T22:12:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.948+0000 7f1c50f35700 1 --2- 172.21.15.57:0/885136214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c101940 0x7f1c4c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.948+0000 7f1c50f35700 1 --2- 172.21.15.57:0/885136214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c106f90 0x7f1c4c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.948+0000 7f1c50f35700 1 --2- 172.21.15.57:0/885136214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c101400 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.949+0000 7f1c50f35700 1 -- 172.21.15.57:0/885136214 >> 172.21.15.57:0/885136214 conn(0x7f1c4c0780a0 msgr2=0x7f1c4c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.949+0000 7f1c50f35700 1 -- 172.21.15.57:0/885136214 shutdown_connections 2023-10-06T22:12:00.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.949+0000 7f1c50f35700 1 -- 172.21.15.57:0/885136214 wait complete. 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 Processor -- start 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 -- start start 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c101940 0x7f1c4c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c106f90 0x7f1c4c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19bf40 con 0x7f1c4c100f80 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19c0b0 con 0x7f1c4c101940 2023-10-06T22:12:00.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c50f35700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19c270 con 0x7f1c4c106f90 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c4a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c4a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51520/0 (socket says 172.21.15.57:51520) 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c4a59c700 1 -- 172.21.15.57:0/3505401871 learned_addr learned my addr 172.21.15.57:0/3505401871 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.950+0000 7f1c49d9b700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c101940 0x7f1c4c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c4ad9d700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c106f90 0x7f1c4c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c4a59c700 1 -- 172.21.15.57:0/3505401871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c106f90 msgr2=0x7f1c4c19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c4a59c700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c106f90 0x7f1c4c19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c4a59c700 1 -- 172.21.15.57:0/3505401871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c101940 msgr2=0x7f1c4c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c4a59c700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c101940 0x7f1c4c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c4a59c700 1 -- 172.21.15.57:0/3505401871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400095f0 con 0x7f1c4c100f80 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c49d9b700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c101940 0x7f1c4c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:00.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c4a59c700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c196f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00e9a0 tx=0x7f1c3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:00.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c3b7fe700 1 -- 172.21.15.57:0/3505401871 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c009790 con 0x7f1c4c100f80 2023-10-06T22:12:00.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.951+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c19c550 con 0x7f1c4c100f80 2023-10-06T22:12:00.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.952+0000 7f1c3b7fe700 1 -- 172.21.15.57:0/3505401871 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c3c010430 con 0x7f1c4c100f80 2023-10-06T22:12:00.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.952+0000 7f1c3b7fe700 1 -- 172.21.15.57:0/3505401871 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c018680 con 0x7f1c4c100f80 2023-10-06T22:12:00.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.952+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c1a3db0 con 0x7f1c4c100f80 2023-10-06T22:12:00.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.953+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c04ebb0 con 0x7f1c4c100f80 2023-10-06T22:12:00.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.955+0000 7f1c3b7fe700 1 -- 172.21.15.57:0/3505401871 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1c3c0105a0 con 0x7f1c4c100f80 2023-10-06T22:12:00.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.956+0000 7f1c3b7fe700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1c340a1480 0x7f1c340a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:00.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.956+0000 7f1c3b7fe700 1 -- 172.21.15.57:0/3505401871 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c3c014070 con 0x7f1c4c100f80 2023-10-06T22:12:00.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.956+0000 7f1c49d9b700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1c340a1480 0x7f1c340a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:00.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.957+0000 7f1c49d9b700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1c340a1480 0x7f1c340a3940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009990 tx=0x7f1c4001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:00.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:00.960+0000 7f1c3b7fe700 1 -- 172.21.15.57:0/3505401871 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1c3c052100 con 0x7f1c4c100f80 2023-10-06T22:12:01.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:00 smithi156 ceph-mon[134358]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:00 smithi190 ceph-mon[134891]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:01.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.170+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c4c1982e0 con 0x7f1c4c100f80 2023-10-06T22:12:01.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.171+0000 7f1c3b7fe700 1 -- 172.21.15.57:0/3505401871 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18412 (secure 0 0 0) 0x7f1c3c096510 con 0x7f1c4c100f80 2023-10-06T22:12:01.174 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:01.174 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":79,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:11:56.365760+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:12:01.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1c340a1480 msgr2=0x7f1c340a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:01.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1c340a1480 0x7f1c340a3940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009990 tx=0x7f1c4001a040 comp rx=0 tx=0).stop 2023-10-06T22:12:01.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 msgr2=0x7f1c4c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c196f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00e9a0 tx=0x7f1c3c00ecb0 comp rx=0 tx=0).stop 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 shutdown_connections 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1c340a1480 0x7f1c340a3940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c106f90 0x7f1c4c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c4c100f80 0x7f1c4c196f80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 --2- 172.21.15.57:0/3505401871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c4c101940 0x7f1c4c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 >> 172.21.15.57:0/3505401871 conn(0x7f1c4c0780a0 msgr2=0x7f1c4c0ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 shutdown_connections 2023-10-06T22:12:01.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:01.174+0000 7f1c50f35700 1 -- 172.21.15.57:0/3505401871 wait complete. 2023-10-06T22:12:01.736 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:12:01.737 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-06T22:12:02.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:01 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3505401871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:01 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3505401871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:02.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:01 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3505401871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:02.478 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:12:03.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:02 smithi156 ceph-mon[134358]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:03.042 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:02 smithi057 ceph-mon[98492]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:02 smithi190 ceph-mon[134891]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:03.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.965+0000 7f816ace7700 1 -- 172.21.15.57:0/1097148197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164101940 msgr2=0x7f816410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:03.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.965+0000 7f816ace7700 1 --2- 172.21.15.57:0/1097148197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164101940 0x7f816410ddb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b0d0 tx=0x7f815c00b3e0 comp rx=0 tx=0).stop 2023-10-06T22:12:03.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.965+0000 7f816ace7700 1 -- 172.21.15.57:0/1097148197 shutdown_connections 2023-10-06T22:12:03.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.965+0000 7f816ace7700 1 --2- 172.21.15.57:0/1097148197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164101940 0x7f816410ddb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:03.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.965+0000 7f816ace7700 1 --2- 172.21.15.57:0/1097148197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8164106f90 0x7f8164107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:03.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.965+0000 7f816ace7700 1 --2- 172.21.15.57:0/1097148197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8164100f80 0x7f8164101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:03.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.965+0000 7f816ace7700 1 -- 172.21.15.57:0/1097148197 >> 172.21.15.57:0/1097148197 conn(0x7f81640780a0 msgr2=0x7f81640784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:03.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.966+0000 7f816ace7700 1 -- 172.21.15.57:0/1097148197 shutdown_connections 2023-10-06T22:12:03.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.966+0000 7f816ace7700 1 -- 172.21.15.57:0/1097148197 wait complete. 2023-10-06T22:12:03.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 Processor -- start 2023-10-06T22:12:03.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 -- start start 2023-10-06T22:12:03.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8164100f80 0x7f8164199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8164101940 0x7f8164199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164106f90 0x7f816419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419e300 con 0x7f8164106f90 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419e470 con 0x7f8164100f80 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.967+0000 7f816ace7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419e630 con 0x7f8164101940 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164106f90 0x7f816419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8163fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8164101940 0x7f8164199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164106f90 0x7f816419dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37386/0 (socket says 172.21.15.57:37386) 2023-10-06T22:12:03.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 -- 172.21.15.57:0/1289880762 learned_addr learned my addr 172.21.15.57:0/1289880762 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:03.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8168a83700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8164100f80 0x7f8164199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:03.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 -- 172.21.15.57:0/1289880762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8164101940 msgr2=0x7f8164199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:03.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8164101940 0x7f8164199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:03.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 -- 172.21.15.57:0/1289880762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8164100f80 msgr2=0x7f8164199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:03.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8164100f80 0x7f8164199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:03.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 -- 172.21.15.57:0/1289880762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c009d00 con 0x7f8164106f90 2023-10-06T22:12:03.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.968+0000 7f8169284700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164106f90 0x7f816419dcf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f815c007820 tx=0x7f815c007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:03.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.969+0000 7f8161ffb700 1 -- 172.21.15.57:0/1289880762 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c00e040 con 0x7f8164106f90 2023-10-06T22:12:03.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.969+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816419e8b0 con 0x7f8164106f90 2023-10-06T22:12:03.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.969+0000 7f8161ffb700 1 -- 172.21.15.57:0/1289880762 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815c00bcb0 con 0x7f8164106f90 2023-10-06T22:12:03.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.969+0000 7f8161ffb700 1 -- 172.21.15.57:0/1289880762 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c005870 con 0x7f8164106f90 2023-10-06T22:12:03.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.969+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81641a6170 con 0x7f8164106f90 2023-10-06T22:12:03.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.970+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816419eb20 con 0x7f8164106f90 2023-10-06T22:12:03.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.972+0000 7f8161ffb700 1 -- 172.21.15.57:0/1289880762 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f815c004a50 con 0x7f8164106f90 2023-10-06T22:12:03.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.973+0000 7f8161ffb700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81540a1480 0x7f81540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:03.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.973+0000 7f8161ffb700 1 -- 172.21.15.57:0/1289880762 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f815c0cd660 con 0x7f8164106f90 2023-10-06T22:12:03.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.974+0000 7f8168a83700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81540a1480 0x7f81540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:03.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.974+0000 7f8168a83700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81540a1480 0x7f81540a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8158009be0 tx=0x7f8158009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:03.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:03.977+0000 7f8161ffb700 1 -- 172.21.15.57:0/1289880762 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f815c097150 con 0x7f8164106f90 2023-10-06T22:12:04.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.187+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8164110ea0 con 0x7f8164106f90 2023-10-06T22:12:04.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.187+0000 7f8161ffb700 1 -- 172.21.15.57:0/1289880762 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f815c09a770 con 0x7f8164106f90 2023-10-06T22:12:04.190 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:12:04.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81540a1480 msgr2=0x7f81540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:04.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81540a1480 0x7f81540a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8158009be0 tx=0x7f8158009260 comp rx=0 tx=0).stop 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164106f90 msgr2=0x7f816419dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164106f90 0x7f816419dcf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f815c007820 tx=0x7f815c007850 comp rx=0 tx=0).stop 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 shutdown_connections 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81540a1480 0x7f81540a3940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8164106f90 0x7f816419dcf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8164100f80 0x7f8164199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 --2- 172.21.15.57:0/1289880762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8164101940 0x7f8164199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 >> 172.21.15.57:0/1289880762 conn(0x7f81640780a0 msgr2=0x7f816410c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 shutdown_connections 2023-10-06T22:12:04.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:04.190+0000 7f816ace7700 1 -- 172.21.15.57:0/1289880762 wait complete. 2023-10-06T22:12:04.654 INFO:tasks.cephadm:Adding mds.a on smithi057 2023-10-06T22:12:04.654 INFO:tasks.cephadm:Adding mds.d on smithi057 2023-10-06T22:12:04.654 INFO:tasks.cephadm:Adding mds.g on smithi057 2023-10-06T22:12:04.654 INFO:tasks.cephadm:Adding mds.j on smithi057 2023-10-06T22:12:04.654 INFO:tasks.cephadm:Adding mds.b on smithi156 2023-10-06T22:12:04.655 INFO:tasks.cephadm:Adding mds.e on smithi156 2023-10-06T22:12:04.655 INFO:tasks.cephadm:Adding mds.h on smithi156 2023-10-06T22:12:04.655 INFO:tasks.cephadm:Adding mds.k on smithi156 2023-10-06T22:12:04.655 INFO:tasks.cephadm:Adding mds.c on smithi190 2023-10-06T22:12:04.655 INFO:tasks.cephadm:Adding mds.f on smithi190 2023-10-06T22:12:04.655 INFO:tasks.cephadm:Adding mds.i on smithi190 2023-10-06T22:12:04.655 INFO:tasks.cephadm:Adding mds.l on smithi190 2023-10-06T22:12:04.656 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l' 2023-10-06T22:12:04.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:04 smithi057 ceph-mon[98492]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:04.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:04 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1289880762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:05.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:04 smithi156 ceph-mon[134358]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:05.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:04 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1289880762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:04 smithi190 ceph-mon[134891]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:04 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1289880762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:05.229 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:12:06.572 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 -- 172.21.15.190:0/1973874663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc0ffe60 msgr2=0x7fe3dc100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:06.572 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/1973874663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc0ffe60 0x7fe3dc100280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009990 tx=0x7fe3d0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.572 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 -- 172.21.15.190:0/1973874663 shutdown_connections 2023-10-06T22:12:06.572 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/1973874663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc101a10 0x7fe3dc108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.572 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/1973874663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc0ffe60 0x7fe3dc100280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/1973874663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc101050 0x7fe3dc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 -- 172.21.15.190:0/1973874663 >> 172.21.15.190:0/1973874663 conn(0x7fe3dc0fb450 msgr2=0x7fe3dc0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 -- 172.21.15.190:0/1973874663 shutdown_connections 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.568+0000 7fe3e0acb700 1 -- 172.21.15.190:0/1973874663 wait complete. 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 Processor -- start 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 -- start start 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ffe60 0x7fe3dc069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc101050 0x7fe3dc19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc101a10 0x7fe3dc19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:06.573 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19e710 con 0x7fe3dc0ffe60 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19e880 con 0x7fe3dc101050 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3e0acb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19ea40 con 0x7fe3dc101a10 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3dad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc101a10 0x7fe3dc19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3dad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc101a10 0x7fe3dc19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46616/0 (socket says 172.21.15.190:46616) 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.569+0000 7fe3dad9d700 1 -- 172.21.15.190:0/738620351 learned_addr learned my addr 172.21.15.190:0/738620351 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3d9d9b700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc101050 0x7fe3dc19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3da59c700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ffe60 0x7fe3dc069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3dad9d700 1 -- 172.21.15.190:0/738620351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc101050 msgr2=0x7fe3dc19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3dad9d700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc101050 0x7fe3dc19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3dad9d700 1 -- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ffe60 msgr2=0x7fe3dc069e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:06.574 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3dad9d700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ffe60 0x7fe3dc069e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.575 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3dad9d700 1 -- 172.21.15.190:0/738620351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00095f0 con 0x7fe3dc101a10 2023-10-06T22:12:06.575 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3dad9d700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc101a10 0x7fe3dc19df70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400b970 tx=0x7fe3d400bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:06.575 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3cb7fe700 1 -- 172.21.15.190:0/738620351 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400c9b0 con 0x7fe3dc101a10 2023-10-06T22:12:06.575 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc19ed20 con 0x7fe3dc101a10 2023-10-06T22:12:06.575 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3cb7fe700 1 -- 172.21.15.190:0/738620351 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d4014440 con 0x7fe3dc101a10 2023-10-06T22:12:06.575 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3cb7fe700 1 -- 172.21.15.190:0/738620351 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40075d0 con 0x7fe3dc101a10 2023-10-06T22:12:06.576 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.570+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc19f1c0 con 0x7fe3dc101a10 2023-10-06T22:12:06.576 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.571+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc10b700 con 0x7fe3dc101a10 2023-10-06T22:12:06.576 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.572+0000 7fe3cb7fe700 1 -- 172.21.15.190:0/738620351 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe3d40145b0 con 0x7fe3dc101a10 2023-10-06T22:12:06.576 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.573+0000 7fe3cb7fe700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe3c40a1480 0x7fe3c40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:06.576 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.573+0000 7fe3cb7fe700 1 -- 172.21.15.190:0/738620351 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3d4006e40 con 0x7fe3dc101a10 2023-10-06T22:12:06.577 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.573+0000 7fe3da59c700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe3c40a1480 0x7fe3c40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:06.577 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.573+0000 7fe3da59c700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe3c40a1480 0x7fe3c40a3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b5c0 tx=0x7fe3d0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:06.578 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.576+0000 7fe3cb7fe700 1 -- 172.21.15.190:0/738620351 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe3d4095f30 con 0x7fe3dc101a10 2023-10-06T22:12:06.775 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.773+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3dc0613f0 con 0x7fe3c40a1480 2023-10-06T22:12:06.783 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.781+0000 7fe3cb7fe700 1 -- 172.21.15.190:0/738620351 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe3dc0613f0 con 0x7fe3c40a1480 2023-10-06T22:12:06.783 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mds.all update... 2023-10-06T22:12:06.785 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe3c40a1480 msgr2=0x7fe3c40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:06.785 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe3c40a1480 0x7fe3c40a3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b5c0 tx=0x7fe3d0005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc101a10 msgr2=0x7fe3dc19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc101a10 0x7fe3dc19df70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400b970 tx=0x7fe3d400bd30 comp rx=0 tx=0).stop 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 shutdown_connections 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe3c40a1480 0x7fe3c40a3940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3dc101a10 0x7fe3dc19df70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ffe60 0x7fe3dc069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 --2- 172.21.15.190:0/738620351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc101050 0x7fe3dc19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 >> 172.21.15.190:0/738620351 conn(0x7fe3dc0fb450 msgr2=0x7fe3dc0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 shutdown_connections 2023-10-06T22:12:06.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-06T22:12:06.783+0000 7fe3e0acb700 1 -- 172.21.15.190:0/738620351 wait complete. 2023-10-06T22:12:06.949 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:06 smithi190 ceph-mon[134891]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:07.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:06 smithi156 ceph-mon[134358]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:06 smithi057 ceph-mon[98492]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:07.277 DEBUG:teuthology.orchestra.run.smithi057:mds.a> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.a.service 2023-10-06T22:12:07.282 DEBUG:teuthology.orchestra.run.smithi057:mds.d> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.d.service 2023-10-06T22:12:07.328 DEBUG:teuthology.orchestra.run.smithi057:mds.g> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.g.service 2023-10-06T22:12:07.361 INFO:journalctl@ceph.mds.a.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:12:07.364 DEBUG:teuthology.orchestra.run.smithi057:mds.j> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.j.service 2023-10-06T22:12:07.402 INFO:journalctl@ceph.mds.d.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:12:07.407 DEBUG:teuthology.orchestra.run.smithi156:mds.b> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.b.service 2023-10-06T22:12:07.409 DEBUG:teuthology.orchestra.run.smithi156:mds.e> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service 2023-10-06T22:12:07.439 INFO:journalctl@ceph.mds.b.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:12:07.440 DEBUG:teuthology.orchestra.run.smithi156:mds.h> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.h.service 2023-10-06T22:12:07.441 INFO:journalctl@ceph.mds.g.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:12:07.466 INFO:journalctl@ceph.mds.e.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:12:07.467 DEBUG:teuthology.orchestra.run.smithi156:mds.k> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.k.service 2023-10-06T22:12:07.481 INFO:journalctl@ceph.mds.j.smithi057.stdout:-- Logs begin at Fri 2023-10-06 21:47:12 UTC. -- 2023-10-06T22:12:07.492 INFO:journalctl@ceph.mds.h.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:12:07.493 DEBUG:teuthology.orchestra.run.smithi190:mds.c> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.c.service 2023-10-06T22:12:07.495 DEBUG:teuthology.orchestra.run.smithi190:mds.f> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.f.service 2023-10-06T22:12:07.518 INFO:journalctl@ceph.mds.k.smithi156.stdout:-- Logs begin at Fri 2023-10-06 21:41:59 UTC. -- 2023-10-06T22:12:07.525 INFO:journalctl@ceph.mds.c.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:12:07.526 DEBUG:teuthology.orchestra.run.smithi190:mds.i> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.i.service 2023-10-06T22:12:07.553 INFO:journalctl@ceph.mds.f.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:12:07.554 DEBUG:teuthology.orchestra.run.smithi190:mds.l> sudo journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.l.service 2023-10-06T22:12:07.579 INFO:journalctl@ceph.mds.i.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:12:07.580 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-06T22:12:07.580 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:12:07.605 INFO:journalctl@ceph.mds.l.smithi190.stdout:-- Logs begin at Fri 2023-10-06 21:41:40 UTC. -- 2023-10-06T22:12:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 -- 172.21.15.57:0/2929420411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04102080 msgr2=0x7fac04102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 --2- 172.21.15.57:0/2929420411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04102080 0x7fac04102500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009990 tx=0x7fabf8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 -- 172.21.15.57:0/2929420411 shutdown_connections 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 --2- 172.21.15.57:0/2929420411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04102a40 0x7fac04109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 --2- 172.21.15.57:0/2929420411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04100ea0 0x7fac041012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 --2- 172.21.15.57:0/2929420411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04102080 0x7fac04102500 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 -- 172.21.15.57:0/2929420411 >> 172.21.15.57:0/2929420411 conn(0x7fac040fc420 msgr2=0x7fac040fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 -- 172.21.15.57:0/2929420411 shutdown_connections 2023-10-06T22:12:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.792+0000 7fac099ff700 1 -- 172.21.15.57:0/2929420411 wait complete. 2023-10-06T22:12:07.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 Processor -- start 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 -- start start 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04100ea0 0x7fac04074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04102080 0x7fac040732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04102a40 0x7fac040737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04073f80 con 0x7fac04102080 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040740f0 con 0x7fac04102a40 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.793+0000 7fac099ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac041a0c90 con 0x7fac04100ea0 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04100ea0 0x7fac04074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04100ea0 0x7fac04074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:42926/0 (socket says 172.21.15.57:42926) 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 -- 172.21.15.57:0/2423542703 learned_addr learned my addr 172.21.15.57:0/2423542703 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:07.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac027fc700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04102080 0x7fac040732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac037fe700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04102a40 0x7fac040737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 -- 172.21.15.57:0/2423542703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04102a40 msgr2=0x7fac040737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04102a40 0x7fac040737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 -- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04102080 msgr2=0x7fac040732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04102080 0x7fac040732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 -- 172.21.15.57:0/2423542703 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc009c40 con 0x7fac04100ea0 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac02ffd700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04100ea0 0x7fac04074c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e970 tx=0x7fabf400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.794+0000 7fac089fd700 1 -- 172.21.15.57:0/2423542703 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4009790 con 0x7fac04100ea0 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.795+0000 7fac089fd700 1 -- 172.21.15.57:0/2423542703 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabf400eec0 con 0x7fac04100ea0 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.795+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf80095f0 con 0x7fac04100ea0 2023-10-06T22:12:07.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.795+0000 7fac089fd700 1 -- 172.21.15.57:0/2423542703 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4003cf0 con 0x7fac04100ea0 2023-10-06T22:12:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.795+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac041a1260 con 0x7fac04100ea0 2023-10-06T22:12:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.796+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac04108510 con 0x7fac04100ea0 2023-10-06T22:12:07.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.798+0000 7fac089fd700 1 -- 172.21.15.57:0/2423542703 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fabf400ccd0 con 0x7fac04100ea0 2023-10-06T22:12:07.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.799+0000 7fac089fd700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabec0a1550 0x7fabec0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.800+0000 7fac089fd700 1 -- 172.21.15.57:0/2423542703 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fabf4014070 con 0x7fac04100ea0 2023-10-06T22:12:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.800+0000 7fac027fc700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabec0a1550 0x7fabec0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.800+0000 7fac027fc700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabec0a1550 0x7fabec0a3a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabf800b2e0 tx=0x7fabf8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:07.803+0000 7fac089fd700 1 -- 172.21.15.57:0/2423542703 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fabf4096580 con 0x7fac04100ea0 2023-10-06T22:12:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-10-06T22:12:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:07.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: Deploying daemon mds.l on smithi190 2023-10-06T22:12:07.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:07 smithi057 ceph-mon[98492]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:08.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.016+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac0410c760 con 0x7fac04100ea0 2023-10-06T22:12:08.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.018+0000 7fac089fd700 1 -- 172.21.15.57:0/2423542703 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18412 (secure 0 0 0) 0x7fabf4099ba0 con 0x7fac04100ea0 2023-10-06T22:12:08.021 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:08.021 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":79,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:11:56.365760+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:12:08.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabec0a1550 msgr2=0x7fabec0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabec0a1550 0x7fabec0a3a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabf800b2e0 tx=0x7fabf8005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04100ea0 msgr2=0x7fac04074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04100ea0 0x7fac04074c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e970 tx=0x7fabf400ec80 comp rx=0 tx=0).stop 2023-10-06T22:12:08.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 shutdown_connections 2023-10-06T22:12:08.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabec0a1550 0x7fabec0a3a10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac04102a40 0x7fac040737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac04100ea0 0x7fac04074c60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 --2- 172.21.15.57:0/2423542703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04102080 0x7fac040732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 >> 172.21.15.57:0/2423542703 conn(0x7fac040fc420 msgr2=0x7fac04105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:08.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 shutdown_connections 2023-10-06T22:12:08.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.021+0000 7fac099ff700 1 -- 172.21.15.57:0/2423542703 wait complete. 2023-10-06T22:12:08.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:12:08.043 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:12:08.055 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:08.055 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-10-06T22:12:08.055 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:08.055 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: Deploying daemon mds.l on smithi190 2023-10-06T22:12:08.056 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:07 smithi190 ceph-mon[134891]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:08.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 -- 172.21.15.57:0/1621594130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460100e90 msgr2=0x7f74601012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 --2- 172.21.15.57:0/1621594130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460100e90 0x7f74601012b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7454009990 tx=0x7f7454009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 -- 172.21.15.57:0/1621594130 shutdown_connections 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 --2- 172.21.15.57:0/1621594130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102a30 0x7f7460109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 --2- 172.21.15.57:0/1621594130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460100e90 0x7f74601012b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 --2- 172.21.15.57:0/1621594130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7460102070 0x7f74601024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 -- 172.21.15.57:0/1621594130 >> 172.21.15.57:0/1621594130 conn(0x7f74600fc430 msgr2=0x7f74600fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 -- 172.21.15.57:0/1621594130 shutdown_connections 2023-10-06T22:12:08.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.248+0000 7f7466fff700 1 -- 172.21.15.57:0/1621594130 wait complete. 2023-10-06T22:12:08.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 Processor -- start 2023-10-06T22:12:08.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 -- start start 2023-10-06T22:12:08.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7460100e90 0x7f7460198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102070 0x7f7460199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460102a30 0x7f74601a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74601a09c0 con 0x7f7460102a30 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74601a0b00 con 0x7f7460100e90 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.249+0000 7f7466fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74601a0c90 con 0x7f7460102070 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f745ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102070 0x7f7460199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f7464d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7460100e90 0x7f7460198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f745ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102070 0x7f7460199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:42938/0 (socket says 172.21.15.57:42938) 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f745ffff700 1 -- 172.21.15.57:0/3820346233 learned_addr learned my addr 172.21.15.57:0/3820346233 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f746559c700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460102a30 0x7f74601a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f7464d9b700 1 -- 172.21.15.57:0/3820346233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102070 msgr2=0x7f7460199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f7464d9b700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102070 0x7f7460199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f7464d9b700 1 -- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460102a30 msgr2=0x7f74601a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f7464d9b700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460102a30 0x7f74601a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f7464d9b700 1 -- 172.21.15.57:0/3820346233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540095f0 con 0x7f7460100e90 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f745ffff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102070 0x7f7460199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.250+0000 7f7464d9b700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7460100e90 0x7f7460198e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f745400b680 tx=0x7f7454009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.251+0000 7f745dffb700 1 -- 172.21.15.57:0/3820346233 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745401d070 con 0x7f7460100e90 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.251+0000 7f745dffb700 1 -- 172.21.15.57:0/3820346233 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7454021410 con 0x7f7460100e90 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.251+0000 7f745dffb700 1 -- 172.21.15.57:0/3820346233 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454017650 con 0x7f7460100e90 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.251+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74601a0f10 con 0x7f7460100e90 2023-10-06T22:12:08.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.251+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74601a1280 con 0x7f7460100e90 2023-10-06T22:12:08.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:08.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-10-06T22:12:08.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:08.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:08.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:08.256 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:08.257 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:08.257 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:08.257 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:08.257 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:08.257 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: Deploying daemon mds.l on smithi190 2023-10-06T22:12:08.257 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:07 smithi156 ceph-mon[134358]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:08.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.252+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7460193540 con 0x7f7460100e90 2023-10-06T22:12:08.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.254+0000 7f745dffb700 1 -- 172.21.15.57:0/3820346233 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7454021810 con 0x7f7460100e90 2023-10-06T22:12:08.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.255+0000 7f745dffb700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f74500a1500 0x7f74500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.255+0000 7f745dffb700 1 -- 172.21.15.57:0/3820346233 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f74540ce2b0 con 0x7f7460100e90 2023-10-06T22:12:08.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.255+0000 7f745ffff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f74500a1500 0x7f74500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.256+0000 7f745ffff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f74500a1500 0x7f74500a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f744c005950 tx=0x7f744c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:08.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.257+0000 7f745dffb700 1 -- 172.21.15.57:0/3820346233 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7454097da0 con 0x7f7460100e90 2023-10-06T22:12:08.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.465+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f746010c750 con 0x7f7460100e90 2023-10-06T22:12:08.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.467+0000 7f745dffb700 1 -- 172.21.15.57:0/3820346233 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f745409b3c0 con 0x7f7460100e90 2023-10-06T22:12:08.469 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:12:08.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f74500a1500 msgr2=0x7f74500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f74500a1500 0x7f74500a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f744c005950 tx=0x7f744c009260 comp rx=0 tx=0).stop 2023-10-06T22:12:08.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7460100e90 msgr2=0x7f7460198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7460100e90 0x7f7460198e40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f745400b680 tx=0x7f7454009580 comp rx=0 tx=0).stop 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 shutdown_connections 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f74500a1500 0x7f74500a39c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7460102a30 0x7f74601a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7460100e90 0x7f7460198e40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 --2- 172.21.15.57:0/3820346233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7460102070 0x7f7460199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 >> 172.21.15.57:0/3820346233 conn(0x7f74600fc430 msgr2=0x7f7460107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 shutdown_connections 2023-10-06T22:12:08.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.469+0000 7f7466fff700 1 -- 172.21.15.57:0/3820346233 wait complete. 2023-10-06T22:12:08.491 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 -- 172.21.15.57:0/2910554929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102080 msgr2=0x7fe520102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 --2- 172.21.15.57:0/2910554929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102080 0x7fe520102500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe514009990 tx=0x7fe514009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 -- 172.21.15.57:0/2910554929 shutdown_connections 2023-10-06T22:12:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 --2- 172.21.15.57:0/2910554929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe520102a40 0x7fe520109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 --2- 172.21.15.57:0/2910554929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520100ea0 0x7fe5201012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 --2- 172.21.15.57:0/2910554929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102080 0x7fe520102500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 -- 172.21.15.57:0/2910554929 >> 172.21.15.57:0/2910554929 conn(0x7fe5200fc420 msgr2=0x7fe5200fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:08.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 -- 172.21.15.57:0/2910554929 shutdown_connections 2023-10-06T22:12:08.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.691+0000 7fe5253ff700 1 -- 172.21.15.57:0/2910554929 wait complete. 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.692+0000 7fe5253ff700 1 Processor -- start 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe5253ff700 1 -- start start 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe5253ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe520100ea0 0x7fe520196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe5253ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520102080 0x7fe520197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe5253ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102a40 0x7fe52019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe5253ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52019e790 con 0x7fe520102080 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe5253ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52019e900 con 0x7fe520100ea0 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe5253ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52019eac0 con 0x7fe520102a40 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520102080 0x7fe520197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520102080 0x7fe520197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37422/0 (socket says 172.21.15.57:37422) 2023-10-06T22:12:08.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51e7fc700 1 -- 172.21.15.57:0/724418905 learned_addr learned my addr 172.21.15.57:0/724418905 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51f7fe700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102a40 0x7fe52019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51effd700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe520100ea0 0x7fe520196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51f7fe700 1 -- 172.21.15.57:0/724418905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe520100ea0 msgr2=0x7fe520196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51f7fe700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe520100ea0 0x7fe520196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51f7fe700 1 -- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520102080 msgr2=0x7fe520197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.693+0000 7fe51f7fe700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520102080 0x7fe520197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe51f7fe700 1 -- 172.21.15.57:0/724418905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510009520 con 0x7fe520102a40 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe51e7fc700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520102080 0x7fe520197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe51f7fe700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102a40 0x7fe52019e1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe51800efb0 tx=0x7fe51800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe507fff700 1 -- 172.21.15.57:0/724418905 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518003a00 con 0x7fe520102a40 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe507fff700 1 -- 172.21.15.57:0/724418905 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe51800f040 con 0x7fe520102a40 2023-10-06T22:12:08.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe518009c40 con 0x7fe520102a40 2023-10-06T22:12:08.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe507fff700 1 -- 172.21.15.57:0/724418905 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518004540 con 0x7fe520102a40 2023-10-06T22:12:08.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.694+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5140095f0 con 0x7fe520102a40 2023-10-06T22:12:08.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.695+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe520108510 con 0x7fe520102a40 2023-10-06T22:12:08.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.697+0000 7fe507fff700 1 -- 172.21.15.57:0/724418905 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe518003bf0 con 0x7fe520102a40 2023-10-06T22:12:08.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.698+0000 7fe507fff700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1550 0x7fe5080a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:08.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.698+0000 7fe507fff700 1 -- 172.21.15.57:0/724418905 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe5180cc440 con 0x7fe520102a40 2023-10-06T22:12:08.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.699+0000 7fe51effd700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1550 0x7fe5080a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:08.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.699+0000 7fe51effd700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1550 0x7fe5080a3a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe520198120 tx=0x7fe510009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:08.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.700+0000 7fe507fff700 1 -- 172.21.15.57:0/724418905 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe518095f30 con 0x7fe520102a40 2023-10-06T22:12:08.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.959+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe52010c9b0 con 0x7fe520102a40 2023-10-06T22:12:08.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.960+0000 7fe507fff700 1 -- 172.21.15.57:0/724418905 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fe518099550 con 0x7fe520102a40 2023-10-06T22:12:08.962 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:08.962 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1550 msgr2=0x7fe5080a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1550 0x7fe5080a3a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe520198120 tx=0x7fe510009260 comp rx=0 tx=0).stop 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102a40 msgr2=0x7fe52019e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102a40 0x7fe52019e1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe51800efb0 tx=0x7fe51800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 shutdown_connections 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1550 0x7fe5080a3a10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe520102a40 0x7fe52019e1c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe520100ea0 0x7fe520196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 --2- 172.21.15.57:0/724418905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe520102080 0x7fe520197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:08.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.962+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 >> 172.21.15.57:0/724418905 conn(0x7fe5200fc420 msgr2=0x7fe520105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:08.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.963+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 shutdown_connections 2023-10-06T22:12:08.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:08.963+0000 7fe5253ff700 1 -- 172.21.15.57:0/724418905 wait complete. 2023-10-06T22:12:08.966 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 1 2023-10-06T22:12:08.994 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:08 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2423542703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:08.994 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:08 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3820346233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:08.994 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:12:08 smithi190 systemd[1]: Starting Ceph mds.l for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:09.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:08 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2423542703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:09.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:08 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3820346233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:09.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:08 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2423542703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:09.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:08 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3820346233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:09.530 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:12:09 smithi190 podman[170189]: 2023-10-06T22:12:09.788 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mds[170229]: -- 172.21.15.190:0/2210993405 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b29152b1e0 con 0x55b291501000 2023-10-06T22:12:09.789 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:12:09 smithi190 bash[170189]: 7eb4897d64204d32bdd02d72739866339b75bb6d4cea41066bd193344147148f 2023-10-06T22:12:09.789 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-l[170205]: starting mds.l at 2023-10-06T22:12:09.789 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:12:09 smithi190 systemd[1]: Started Ceph mds.l for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/724418905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6832/1813464152,v1:172.21.15.190:6833/1813464152] up:boot 2023-10-06T22:12:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: fsmap 1 up:standby 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:10.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:09 smithi190 ceph-mon[134891]: Deploying daemon mds.k on smithi156 2023-10-06T22:12:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/724418905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6832/1813464152,v1:172.21.15.190:6833/1813464152] up:boot 2023-10-06T22:12:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: fsmap 1 up:standby 2023-10-06T22:12:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:12:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:10.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:10.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:10.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:10.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:09 smithi057 ceph-mon[98492]: Deploying daemon mds.k on smithi156 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/724418905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6832/1813464152,v1:172.21.15.190:6833/1813464152] up:boot 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: fsmap 1 up:standby 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:10.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:10.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:10.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:10.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:09 smithi156 ceph-mon[134358]: Deploying daemon mds.k on smithi156 2023-10-06T22:12:10.987 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:11.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 -- 172.21.15.57:0/635663199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102a40 msgr2=0x7fb444109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:11.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 --2- 172.21.15.57:0/635663199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102a40 0x7fb444109360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00b0a0 tx=0x7fb43c00b3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 -- 172.21.15.57:0/635663199 shutdown_connections 2023-10-06T22:12:11.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 --2- 172.21.15.57:0/635663199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102a40 0x7fb444109360 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 --2- 172.21.15.57:0/635663199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb444100ea0 0x7fb4441012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 --2- 172.21.15.57:0/635663199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444102080 0x7fb444102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 -- 172.21.15.57:0/635663199 >> 172.21.15.57:0/635663199 conn(0x7fb4440fc420 msgr2=0x7fb4440fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:11.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.199+0000 7fb4495ff700 1 -- 172.21.15.57:0/635663199 shutdown_connections 2023-10-06T22:12:11.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.200+0000 7fb4495ff700 1 -- 172.21.15.57:0/635663199 wait complete. 2023-10-06T22:12:11.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.200+0000 7fb4495ff700 1 Processor -- start 2023-10-06T22:12:11.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4495ff700 1 -- start start 2023-10-06T22:12:11.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4495ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444100ea0 0x7fb444074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:11.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4495ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102080 0x7fb4440732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:11.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4495ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb444102a40 0x7fb4440737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4495ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444073f80 con 0x7fb444100ea0 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4495ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440740f0 con 0x7fb444102080 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4495ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4441a0d10 con 0x7fb444102a40 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb442ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444100ea0 0x7fb444074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4427fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102080 0x7fb4440732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb442ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444100ea0 0x7fb444074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37428/0 (socket says 172.21.15.57:37428) 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb442ffd700 1 -- 172.21.15.57:0/3767306062 learned_addr learned my addr 172.21.15.57:0/3767306062 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.201+0000 7fb4437fe700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb444102a40 0x7fb4440737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb4427fc700 1 -- 172.21.15.57:0/3767306062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb444102a40 msgr2=0x7fb4440737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb4427fc700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb444102a40 0x7fb4440737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb4427fc700 1 -- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444100ea0 msgr2=0x7fb444074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb4427fc700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444100ea0 0x7fb444074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb4427fc700 1 -- 172.21.15.57:0/3767306062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb438009520 con 0x7fb444102080 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb442ffd700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444100ea0 0x7fb444074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb4427fc700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102080 0x7fb4440732a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9c0 tx=0x7fb43800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb42bfff700 1 -- 172.21.15.57:0/3767306062 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43800cb70 con 0x7fb444102080 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb42bfff700 1 -- 172.21.15.57:0/3767306062 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb43800ee60 con 0x7fb444102080 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb42bfff700 1 -- 172.21.15.57:0/3767306062 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb438018680 con 0x7fb444102080 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.202+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c009d00 con 0x7fb444102080 2023-10-06T22:12:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.203+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4441a1290 con 0x7fb444102080 2023-10-06T22:12:11.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.203+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4441083c0 con 0x7fb444102080 2023-10-06T22:12:11.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.206+0000 7fb42bfff700 1 -- 172.21.15.57:0/3767306062 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb438010430 con 0x7fb444102080 2023-10-06T22:12:11.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.207+0000 7fb42bfff700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb42c0a1550 0x7fb42c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:11.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.207+0000 7fb42bfff700 1 -- 172.21.15.57:0/3767306062 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb438014070 con 0x7fb444102080 2023-10-06T22:12:11.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.207+0000 7fb442ffd700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb42c0a1550 0x7fb42c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:11.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.208+0000 7fb442ffd700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb42c0a1550 0x7fb42c0a3a10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb434005950 tx=0x7fb434009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:11.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.209+0000 7fb42bfff700 1 -- 172.21.15.57:0/3767306062 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb438096a30 con 0x7fb444102080 2023-10-06T22:12:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.467+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb44402d2b0 con 0x7fb444102080 2023-10-06T22:12:11.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.468+0000 7fb42bfff700 1 -- 172.21.15.57:0/3767306062 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fb43800ce60 con 0x7fb444102080 2023-10-06T22:12:11.470 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:11.470 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb42c0a1550 msgr2=0x7fb42c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb42c0a1550 0x7fb42c0a3a10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb434005950 tx=0x7fb434009260 comp rx=0 tx=0).stop 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102080 msgr2=0x7fb4440732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102080 0x7fb4440732a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9c0 tx=0x7fb43800ecd0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 shutdown_connections 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb42c0a1550 0x7fb42c0a3a10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb444102a40 0x7fb4440737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb444100ea0 0x7fb444074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 --2- 172.21.15.57:0/3767306062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb444102080 0x7fb4440732a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:11.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 >> 172.21.15.57:0/3767306062 conn(0x7fb4440fc420 msgr2=0x7fb444105500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:11.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 shutdown_connections 2023-10-06T22:12:11.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:11.470+0000 7fb4495ff700 1 -- 172.21.15.57:0/3767306062 wait complete. 2023-10-06T22:12:11.474 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 2 2023-10-06T22:12:11.504 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:12:11 smithi156 systemd[1]: Starting Ceph mds.k for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:11.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:11 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3767306062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:11.947 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:11 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3767306062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:11.948 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:12:11 smithi156 podman[170890]: 2023-10-06T22:12:12.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:11 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3767306062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:12.253 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:12:11 smithi156 ceph-mds[170931]: -- 172.21.15.156:0/1304500844 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56399f9a91e0 con 0x56399f97e800 2023-10-06T22:12:12.254 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:12:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-k[170906]: starting mds.k at 2023-10-06T22:12:12.254 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:12:12 smithi156 bash[170890]: 0c8ab06a4d196df8b85d27df4569fec2008a0e595f2ef1148ecfdf776d02b6ac 2023-10-06T22:12:12.254 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:12:12 smithi156 systemd[1]: Started Ceph mds.k for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:12 smithi057 ceph-mon[98492]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:12 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:13.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:12 smithi156 ceph-mon[134358]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:12 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:12 smithi190 ceph-mon[134891]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:13.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:12 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:13.493 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:13 smithi057 ceph-mon[98492]: Deploying daemon mds.j on smithi057 2023-10-06T22:12:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:13 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6832/1142570288,v1:172.21.15.156:6833/1142570288] up:boot 2023-10-06T22:12:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:13 smithi057 ceph-mon[98492]: fsmap 2 up:standby 2023-10-06T22:12:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:13 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:12:13.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 -- 172.21.15.57:0/4050561519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8104a70 msgr2=0x7f55d8104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:13.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 --2- 172.21.15.57:0/4050561519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8104a70 0x7f55d8104ef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b0a0 tx=0x7f55d000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 -- 172.21.15.57:0/4050561519 shutdown_connections 2023-10-06T22:12:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 --2- 172.21.15.57:0/4050561519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d8104a70 0x7f55d8104ef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 --2- 172.21.15.57:0/4050561519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d80ff730 0x7f55d8101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 --2- 172.21.15.57:0/4050561519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55d81020a0 0x7f55d8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 -- 172.21.15.57:0/4050561519 >> 172.21.15.57:0/4050561519 conn(0x7f55d80fb310 msgr2=0x7f55d80fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 -- 172.21.15.57:0/4050561519 shutdown_connections 2023-10-06T22:12:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.704+0000 7f55df7ff700 1 -- 172.21.15.57:0/4050561519 wait complete. 2023-10-06T22:12:13.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.705+0000 7f55df7ff700 1 Processor -- start 2023-10-06T22:12:13.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.705+0000 7f55df7ff700 1 -- start start 2023-10-06T22:12:13.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55df7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d80ff730 0x7f55d819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:13.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55df7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55d81020a0 0x7f55d819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55df7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d8104a70 0x7f55d81a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55df7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d81a4e30 con 0x7f55d8104a70 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55df7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d81a4f70 con 0x7f55d80ff730 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55df7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d81a5100 con 0x7f55d81020a0 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55dd59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d80ff730 0x7f55d819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55ddd9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d8104a70 0x7f55d81a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55dd59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d80ff730 0x7f55d819d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:37916/0 (socket says 172.21.15.57:37916) 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55dd59b700 1 -- 172.21.15.57:0/1266920069 learned_addr learned my addr 172.21.15.57:0/1266920069 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55dcd9a700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55d81020a0 0x7f55d819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55ddd9c700 1 -- 172.21.15.57:0/1266920069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55d81020a0 msgr2=0x7f55d819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55ddd9c700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55d81020a0 0x7f55d819d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55ddd9c700 1 -- 172.21.15.57:0/1266920069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d80ff730 msgr2=0x7f55d819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.706+0000 7f55ddd9c700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d80ff730 0x7f55d819d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55ddd9c700 1 -- 172.21.15.57:0/1266920069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d0009d00 con 0x7f55d8104a70 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55dd59b700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d80ff730 0x7f55d819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55ddd9c700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d8104a70 0x7f55d81a4860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009fd0 tx=0x7f55d0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55ce7fc700 1 -- 172.21.15.57:0/1266920069 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000e040 con 0x7f55d8104a70 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55ce7fc700 1 -- 172.21.15.57:0/1266920069 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d0004c50 con 0x7f55d8104a70 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d81a5380 con 0x7f55d8104a70 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55ce7fc700 1 -- 172.21.15.57:0/1266920069 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d001b6c0 con 0x7f55d8104a70 2023-10-06T22:12:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.707+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d81a57a0 con 0x7f55d8104a70 2023-10-06T22:12:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.708+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d80fce90 con 0x7f55d8104a70 2023-10-06T22:12:13.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.710+0000 7f55ce7fc700 1 -- 172.21.15.57:0/1266920069 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f55d001d470 con 0x7f55d8104a70 2023-10-06T22:12:13.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.712+0000 7f55ce7fc700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55c40a14d0 0x7f55c40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:13.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.712+0000 7f55ce7fc700 1 -- 172.21.15.57:0/1266920069 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f55d00cd910 con 0x7f55d8104a70 2023-10-06T22:12:13.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.712+0000 7f55dd59b700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55c40a14d0 0x7f55c40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:13.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.713+0000 7f55dd59b700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55c40a14d0 0x7f55c40a3990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55d4011550 tx=0x7f55d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:13.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:13.716+0000 7f55ce7fc700 1 -- 172.21.15.57:0/1266920069 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f55d0097400 con 0x7f55d8104a70 2023-10-06T22:12:14.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:13 smithi156 ceph-mon[134358]: Deploying daemon mds.j on smithi057 2023-10-06T22:12:14.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:13 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6832/1142570288,v1:172.21.15.156:6833/1142570288] up:boot 2023-10-06T22:12:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:13 smithi156 ceph-mon[134358]: fsmap 2 up:standby 2023-10-06T22:12:14.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:13 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:12:14.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.014+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55d8112d00 con 0x7f55d8104a70 2023-10-06T22:12:14.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.015+0000 7f55ce7fc700 1 -- 172.21.15.57:0/1266920069 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f55d009aa20 con 0x7f55d8104a70 2023-10-06T22:12:14.018 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:14.018 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-06T22:12:14.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55c40a14d0 msgr2=0x7f55c40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:14.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55c40a14d0 0x7f55c40a3990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55d4011550 tx=0x7f55d4009260 comp rx=0 tx=0).stop 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d8104a70 msgr2=0x7f55d81a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d8104a70 0x7f55d81a4860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009fd0 tx=0x7f55d0007930 comp rx=0 tx=0).stop 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 shutdown_connections 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55c40a14d0 0x7f55c40a3990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d8104a70 0x7f55d81a4860 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55d80ff730 0x7f55d819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 --2- 172.21.15.57:0/1266920069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55d81020a0 0x7f55d819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 >> 172.21.15.57:0/1266920069 conn(0x7f55d80fb310 msgr2=0x7f55d81003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 shutdown_connections 2023-10-06T22:12:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:14.018+0000 7f55df7ff700 1 -- 172.21.15.57:0/1266920069 wait complete. 2023-10-06T22:12:14.022 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 3 2023-10-06T22:12:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:13 smithi190 ceph-mon[134891]: Deploying daemon mds.j on smithi057 2023-10-06T22:12:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:13 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6832/1142570288,v1:172.21.15.156:6833/1142570288] up:boot 2023-10-06T22:12:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:13 smithi190 ceph-mon[134891]: fsmap 2 up:standby 2023-10-06T22:12:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:13 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:12:14.907 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:14 smithi057 ceph-mon[98492]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:14.907 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:14 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1266920069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:14.908 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:12:14 smithi057 systemd[1]: Starting Ceph mds.j for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:15.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:14 smithi156 ceph-mon[134358]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:15.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:14 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1266920069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:14 smithi190 ceph-mon[134891]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:14 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1266920069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:15.163 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:12:14 smithi057 podman[149024]: 2023-10-06T22:12:15.423 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:12:15 smithi057 bash[149024]: 0a6d1f5a750d8bc8a99da6288bf18284264287fa42d2999d08add2b195bee1a8 2023-10-06T22:12:15.423 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:12:15 smithi057 ceph-mds[149064]: -- 172.21.15.57:0/3136038596 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56130180b1e0 con 0x5613017e0800 2023-10-06T22:12:15.423 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:12:15 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-j[149040]: starting mds.j at 2023-10-06T22:12:15.423 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:12:15 smithi057 systemd[1]: Started Ceph mds.j for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:16.041 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.259+0000 7f28fcbff700 1 -- 172.21.15.57:0/1913117740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8102080 msgr2=0x7f28f8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.259+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1913117740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8102080 0x7f28f8102500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0098f0 tx=0x7f28ec009c00 comp rx=0 tx=0).stop 2023-10-06T22:12:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.260+0000 7f28fcbff700 1 -- 172.21.15.57:0/1913117740 shutdown_connections 2023-10-06T22:12:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.260+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1913117740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28f8102a40 0x7f28f8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.260+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1913117740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8100ea0 0x7f28f81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.260+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1913117740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8102080 0x7f28f8102500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.260+0000 7f28fcbff700 1 -- 172.21.15.57:0/1913117740 >> 172.21.15.57:0/1913117740 conn(0x7f28f80fc420 msgr2=0x7f28f80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:16.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.260+0000 7f28fcbff700 1 -- 172.21.15.57:0/1913117740 shutdown_connections 2023-10-06T22:12:16.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.260+0000 7f28fcbff700 1 -- 172.21.15.57:0/1913117740 wait complete. 2023-10-06T22:12:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 Processor -- start 2023-10-06T22:12:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 -- start start 2023-10-06T22:12:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8100ea0 0x7f28f8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8102080 0x7f28f80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28f8102a40 0x7f28f80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8073f80 con 0x7f28f8102a40 2023-10-06T22:12:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80740f0 con 0x7f28f8100ea0 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.261+0000 7f28fcbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f81a0c60 con 0x7f28f8102080 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8102080 0x7f28f80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8102080 0x7f28f80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:38158/0 (socket says 172.21.15.57:38158) 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 -- 172.21.15.57:0/1709615765 learned_addr learned my addr 172.21.15.57:0/1709615765 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f659c700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8100ea0 0x7f28f8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f6d9d700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28f8102a40 0x7f28f80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 -- 172.21.15.57:0/1709615765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8100ea0 msgr2=0x7f28f8074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8100ea0 0x7f28f8074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 -- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28f8102a40 msgr2=0x7f28f80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28f8102a40 0x7f28f80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 -- 172.21.15.57:0/1709615765 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec009520 con 0x7f28f8102080 2023-10-06T22:12:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.262+0000 7f28f5d9b700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8102080 0x7f28f80732a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.263+0000 7f28e77fe700 1 -- 172.21.15.57:0/1709615765 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec01c070 con 0x7f28f8102080 2023-10-06T22:12:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.263+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f81a0ee0 con 0x7f28f8102080 2023-10-06T22:12:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.263+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f81a12a0 con 0x7f28f8102080 2023-10-06T22:12:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.263+0000 7f28e77fe700 1 -- 172.21.15.57:0/1709615765 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28ec00f7d0 con 0x7f28f8102080 2023-10-06T22:12:16.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.263+0000 7f28e77fe700 1 -- 172.21.15.57:0/1709615765 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec003ae0 con 0x7f28f8102080 2023-10-06T22:12:16.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.264+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f810c710 con 0x7f28f8102080 2023-10-06T22:12:16.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.266+0000 7f28e77fe700 1 -- 172.21.15.57:0/1709615765 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f28ec020030 con 0x7f28f8102080 2023-10-06T22:12:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.269+0000 7f28e77fe700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28e00a14d0 0x7f28e00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.269+0000 7f28e77fe700 1 -- 172.21.15.57:0/1709615765 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28ec0cdda0 con 0x7f28f8102080 2023-10-06T22:12:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.269+0000 7f28f659c700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28e00a14d0 0x7f28e00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.269+0000 7f28f659c700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28e00a14d0 0x7f28e00a3990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28f8101bf0 tx=0x7f28e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:16.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.270+0000 7f28e77fe700 1 -- 172.21.15.57:0/1709615765 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f28ec097890 con 0x7f28f8102080 2023-10-06T22:12:16.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:16.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:16.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:16.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: Deploying daemon mds.d on smithi057 2023-10-06T22:12:16.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:16.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.57:6834/1029690870,v1:172.21.15.57:6835/1029690870] up:boot 2023-10-06T22:12:16.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: fsmap 3 up:standby 2023-10-06T22:12:16.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:16 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T22:12:16.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.556+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28f8107e70 con 0x7f28f8102080 2023-10-06T22:12:16.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.558+0000 7f28e77fe700 1 -- 172.21.15.57:0/1709615765 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f28ec09aeb0 con 0x7f28f8102080 2023-10-06T22:12:16.560 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:16.561 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-06T22:12:16.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28e00a14d0 msgr2=0x7f28e00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:16.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28e00a14d0 0x7f28e00a3990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28f8101bf0 tx=0x7f28e8009260 comp rx=0 tx=0).stop 2023-10-06T22:12:16.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8102080 msgr2=0x7f28f80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8102080 0x7f28f80732a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec00b790 comp rx=0 tx=0).stop 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 shutdown_connections 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28e00a14d0 0x7f28e00a3990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28f8102a40 0x7f28f80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8100ea0 0x7f28f8074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 --2- 172.21.15.57:0/1709615765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f8102080 0x7f28f80732a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 >> 172.21.15.57:0/1709615765 conn(0x7f28f80fc420 msgr2=0x7f28f80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 shutdown_connections 2023-10-06T22:12:16.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:16.561+0000 7f28fcbff700 1 -- 172.21.15.57:0/1709615765 wait complete. 2023-10-06T22:12:16.565 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 4 2023-10-06T22:12:16.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:16.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:16.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:16.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: Deploying daemon mds.d on smithi057 2023-10-06T22:12:16.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:16.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.57:6834/1029690870,v1:172.21.15.57:6835/1029690870] up:boot 2023-10-06T22:12:16.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: fsmap 3 up:standby 2023-10-06T22:12:16.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:16 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: Deploying daemon mds.d on smithi057 2023-10-06T22:12:16.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:16.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.57:6834/1029690870,v1:172.21.15.57:6835/1029690870] up:boot 2023-10-06T22:12:16.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: fsmap 3 up:standby 2023-10-06T22:12:16.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:16 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T22:12:17.562 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:17 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1709615765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:17.563 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:12:17 smithi057 systemd[1]: Starting Ceph mds.d for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:17.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:17 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1709615765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:17.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:17 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1709615765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:18.288 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:12:18 smithi057 podman[149409]: 2023-10-06T22:12:18.583 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:18.648 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:18 smithi057 ceph-mon[98492]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:18.649 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:12:18 smithi057 bash[149409]: f55de95e1dcaede00ad9d3efd8cf74a7f27cea7b57a023724e6f3eb4cee0e4b9 2023-10-06T22:12:18.649 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:12:18 smithi057 ceph-mds[149449]: -- 172.21.15.57:0/480503538 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b8f4151e0 con 0x564b8f3ea800 2023-10-06T22:12:18.649 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:12:18 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-d[149425]: starting mds.d at 2023-10-06T22:12:18.649 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:12:18 smithi057 systemd[1]: Started Ceph mds.d for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:18.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:18 smithi156 ceph-mon[134358]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:18.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 -- 172.21.15.57:0/3438970011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102080 msgr2=0x7fde00102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:18.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 --2- 172.21.15.57:0/3438970011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102080 0x7fde00102500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009990 tx=0x7fddf4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:18.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 -- 172.21.15.57:0/3438970011 shutdown_connections 2023-10-06T22:12:18.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 --2- 172.21.15.57:0/3438970011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102a40 0x7fde00109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:18.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 --2- 172.21.15.57:0/3438970011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde00100ea0 0x7fde001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:18.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 --2- 172.21.15.57:0/3438970011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102080 0x7fde00102500 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:18.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 -- 172.21.15.57:0/3438970011 >> 172.21.15.57:0/3438970011 conn(0x7fde000fc420 msgr2=0x7fde000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:18.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 -- 172.21.15.57:0/3438970011 shutdown_connections 2023-10-06T22:12:18.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.790+0000 7fde067ff700 1 -- 172.21.15.57:0/3438970011 wait complete. 2023-10-06T22:12:18.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.791+0000 7fde067ff700 1 Processor -- start 2023-10-06T22:12:18.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.791+0000 7fde067ff700 1 -- start start 2023-10-06T22:12:18.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.791+0000 7fde067ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde00100ea0 0x7fde00198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:18.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fde067ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102080 0x7fde00199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:18.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fde067ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102a40 0x7fde001a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:18.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fde067ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde001a0990 con 0x7fde00102a40 2023-10-06T22:12:18.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fde067ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde001a0ad0 con 0x7fde00102080 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fde067ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde001a0c60 con 0x7fde00100ea0 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102080 0x7fde00199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102080 0x7fde00199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38014/0 (socket says 172.21.15.57:38014) 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 -- 172.21.15.57:0/2487410288 learned_addr learned my addr 172.21.15.57:0/2487410288 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddfffff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde00100ea0 0x7fde00198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fde04d9c700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102a40 0x7fde001a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 -- 172.21.15.57:0/2487410288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde00100ea0 msgr2=0x7fde00198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde00100ea0 0x7fde00198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 -- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102a40 msgr2=0x7fde001a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:18.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102a40 0x7fde001a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 -- 172.21.15.57:0/2487410288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40095f0 con 0x7fde00102080 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.792+0000 7fddff7fe700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102080 0x7fde00199350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddf4005720 tx=0x7fddf4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.793+0000 7fddfd7fa700 1 -- 172.21.15.57:0/2487410288 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf401d070 con 0x7fde00102080 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.793+0000 7fddfd7fa700 1 -- 172.21.15.57:0/2487410288 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddf4021410 con 0x7fde00102080 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.793+0000 7fddfd7fa700 1 -- 172.21.15.57:0/2487410288 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4017650 con 0x7fde00102080 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.793+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde001a0ee0 con 0x7fde00102080 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.793+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde001a1250 con 0x7fde00102080 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.794+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0010c7d0 con 0x7fde00102080 2023-10-06T22:12:18.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.796+0000 7fddfd7fa700 1 -- 172.21.15.57:0/2487410288 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fddf4021580 con 0x7fde00102080 2023-10-06T22:12:18.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.797+0000 7fddfd7fa700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdde80a14d0 0x7fdde80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:18.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.797+0000 7fddfd7fa700 1 -- 172.21.15.57:0/2487410288 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fddf40cdae0 con 0x7fde00102080 2023-10-06T22:12:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.797+0000 7fddfffff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdde80a14d0 0x7fdde80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.798+0000 7fddfffff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdde80a14d0 0x7fdde80a3990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde00101bf0 tx=0x7fddf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:18.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:18 smithi190 ceph-mon[134891]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:19.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:18.799+0000 7fddfd7fa700 1 -- 172.21.15.57:0/2487410288 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fddf40975d0 con 0x7fde00102080 2023-10-06T22:12:19.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.062+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde001a13e0 con 0x7fde00102080 2023-10-06T22:12:19.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.062+0000 7fddfd7fa700 1 -- 172.21.15.57:0/2487410288 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7fddf409abf0 con 0x7fde00102080 2023-10-06T22:12:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:19.065 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-06T22:12:19.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdde80a14d0 msgr2=0x7fdde80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:19.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdde80a14d0 0x7fdde80a3990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde00101bf0 tx=0x7fddf0009260 comp rx=0 tx=0).stop 2023-10-06T22:12:19.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102080 msgr2=0x7fde00199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:19.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102080 0x7fde00199350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddf4005720 tx=0x7fddf4009580 comp rx=0 tx=0).stop 2023-10-06T22:12:19.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 shutdown_connections 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdde80a14d0 0x7fdde80a3990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde00102a40 0x7fde001a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde00100ea0 0x7fde00198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 --2- 172.21.15.57:0/2487410288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde00102080 0x7fde00199350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 >> 172.21.15.57:0/2487410288 conn(0x7fde000fc420 msgr2=0x7fde000fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 shutdown_connections 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:19.065+0000 7fde067ff700 1 -- 172.21.15.57:0/2487410288 wait complete. 2023-10-06T22:12:19.069 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 5 2023-10-06T22:12:19.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.57:6836/2389529664,v1:172.21.15.57:6837/2389529664] up:boot 2023-10-06T22:12:19.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: fsmap 4 up:standby 2023-10-06T22:12:19.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T22:12:19.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:19.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:19.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:19.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: Deploying daemon mds.e on smithi156 2023-10-06T22:12:19.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:19 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2487410288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:19.603 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.57:6836/2389529664,v1:172.21.15.57:6837/2389529664] up:boot 2023-10-06T22:12:19.603 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: fsmap 4 up:standby 2023-10-06T22:12:19.603 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: Deploying daemon mds.e on smithi156 2023-10-06T22:12:19.604 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:19 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2487410288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:19.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.57:6836/2389529664,v1:172.21.15.57:6837/2389529664] up:boot 2023-10-06T22:12:19.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: fsmap 4 up:standby 2023-10-06T22:12:19.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: Deploying daemon mds.e on smithi156 2023-10-06T22:12:19.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:19 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2487410288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:20.004 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:12:19 smithi156 systemd[1]: Starting Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:20.399 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:20 smithi156 ceph-mon[134358]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:20.400 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:12:20 smithi156 podman[171231]: 2023-10-06T22:12:20.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:20 smithi057 ceph-mon[98492]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:20.753 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:12:20 smithi156 bash[171231]: 83419adac93b3ad2193d84b770e909f571ab50cfd3370be5722fa9e3ce120fe3 2023-10-06T22:12:20.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:12:20 smithi156 ceph-mds[171271]: -- 172.21.15.156:0/1283615029 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5574b00c51e0 con 0x5574b009ac00 2023-10-06T22:12:20.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:12:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: starting mds.e at 2023-10-06T22:12:20.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:12:20 smithi156 systemd[1]: Started Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:20.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:20 smithi190 ceph-mon[134891]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:21.087 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:21.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3124612957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 msgr2=0x7f8cd4109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3124612957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 0x7f8cd4109070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009f90 tx=0x7f8ccc00b330 comp rx=0 tx=0).stop 2023-10-06T22:12:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3124612957 shutdown_connections 2023-10-06T22:12:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3124612957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 0x7f8cd4109070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3124612957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd41019a0 0x7f8cd4103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3124612957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 0x7f8cd41066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3124612957 >> 172.21.15.57:0/3124612957 conn(0x7f8cd40fb3b0 msgr2=0x7f8cd40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3124612957 shutdown_connections 2023-10-06T22:12:21.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.300+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3124612957 wait complete. 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 Processor -- start 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 -- start start 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd41019a0 0x7f8cd4101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 0x7f8cd40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 0x7f8cd40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd4100630 con 0x7f8cd41042d0 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41007a0 con 0x7f8cd41019a0 2023-10-06T22:12:21.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.301+0000 7f8cdbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a0ca0 con 0x7f8cd4106c00 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 0x7f8cd40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 0x7f8cd40ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56234/0 (socket says 172.21.15.57:56234) 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 -- 172.21.15.57:0/3188706953 learned_addr learned my addr 172.21.15.57:0/3188706953 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd9d9b700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd41019a0 0x7f8cd4101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cda59c700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 0x7f8cd40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 -- 172.21.15.57:0/3188706953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 msgr2=0x7f8cd40ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 0x7f8cd40ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 -- 172.21.15.57:0/3188706953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd41019a0 msgr2=0x7f8cd4101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd41019a0 0x7f8cd4101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 -- 172.21.15.57:0/3188706953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40095f0 con 0x7f8cd41042d0 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.302+0000 7f8cd959a700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 0x7f8cd40ff950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400ec10 tx=0x7f8cc400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:21.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.303+0000 7f8ccaffd700 1 -- 172.21.15.57:0/3188706953 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400cc90 con 0x7f8cd41042d0 2023-10-06T22:12:21.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.303+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc009c40 con 0x7f8cd41042d0 2023-10-06T22:12:21.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.303+0000 7f8ccaffd700 1 -- 172.21.15.57:0/3188706953 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cc40044e0 con 0x7f8cd41042d0 2023-10-06T22:12:21.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.303+0000 7f8ccaffd700 1 -- 172.21.15.57:0/3188706953 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4010430 con 0x7f8cd41042d0 2023-10-06T22:12:21.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.303+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd41a12d0 con 0x7f8cd41042d0 2023-10-06T22:12:21.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.304+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd40fcf30 con 0x7f8cd41042d0 2023-10-06T22:12:21.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.306+0000 7f8ccaffd700 1 -- 172.21.15.57:0/3188706953 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8cc4003f90 con 0x7f8cd41042d0 2023-10-06T22:12:21.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.307+0000 7f8ccaffd700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8cc00a14d0 0x7f8cc00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:21.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.307+0000 7f8ccaffd700 1 -- 172.21.15.57:0/3188706953 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8cc4014070 con 0x7f8cd41042d0 2023-10-06T22:12:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.307+0000 7f8cd9d9b700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8cc00a14d0 0x7f8cc00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:21.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.308+0000 7f8cd9d9b700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8cc00a14d0 0x7f8cc00a3990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009c60 tx=0x7f8cd0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:21.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.310+0000 7f8ccaffd700 1 -- 172.21.15.57:0/3188706953 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8cc4096a10 con 0x7f8cd41042d0 2023-10-06T22:12:21.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.570+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cd402d270 con 0x7f8cd41042d0 2023-10-06T22:12:21.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.570+0000 7f8ccaffd700 1 -- 172.21.15.57:0/3188706953 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7f8cc4017e90 con 0x7f8cd41042d0 2023-10-06T22:12:21.573 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:21.573 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-06T22:12:21.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8cc00a14d0 msgr2=0x7f8cc00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:21.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8cc00a14d0 0x7f8cc00a3990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009c60 tx=0x7f8cd0009260 comp rx=0 tx=0).stop 2023-10-06T22:12:21.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 msgr2=0x7f8cd40ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:21.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 0x7f8cd40ff950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400ec10 tx=0x7f8cc400ef20 comp rx=0 tx=0).stop 2023-10-06T22:12:21.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 shutdown_connections 2023-10-06T22:12:21.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8cc00a14d0 0x7f8cc00a3990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cd4106c00 0x7f8cd40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cd41019a0 0x7f8cd4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 --2- 172.21.15.57:0/3188706953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8cd41042d0 0x7f8cd40ff950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:21.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 >> 172.21.15.57:0/3188706953 conn(0x7f8cd40fb3b0 msgr2=0x7f8cd4102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:21.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 shutdown_connections 2023-10-06T22:12:21.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:21.573+0000 7f8cdbfff700 1 -- 172.21.15.57:0/3188706953 wait complete. 2023-10-06T22:12:21.576 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-06T22:12:21.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: Deploying daemon mds.f on smithi190 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6834/2797676784,v1:172.21.15.156:6835/2797676784] up:boot 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: fsmap 5 up:standby 2023-10-06T22:12:21.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:21 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:12:21.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:21.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:21.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:21.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: Deploying daemon mds.f on smithi190 2023-10-06T22:12:21.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6834/2797676784,v1:172.21.15.156:6835/2797676784] up:boot 2023-10-06T22:12:21.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: fsmap 5 up:standby 2023-10-06T22:12:21.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:21 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: Deploying daemon mds.f on smithi190 2023-10-06T22:12:21.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6834/2797676784,v1:172.21.15.156:6835/2797676784] up:boot 2023-10-06T22:12:21.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: fsmap 5 up:standby 2023-10-06T22:12:21.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:21 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:12:22.485 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:12:22 smithi190 systemd[1]: Starting Ceph mds.f for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:22.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:22 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3188706953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:22.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:22 smithi156 ceph-mon[134358]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:22.795 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:12:22 smithi190 podman[170533]: 2023-10-06T22:12:22.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:22 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3188706953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:22.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:22 smithi190 ceph-mon[134891]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:22 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3188706953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:22 smithi057 ceph-mon[98492]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:23.069 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:12:22 smithi190 ceph-mds[170574]: -- 172.21.15.190:0/1467511533 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55de577971e0 con 0x55de5776cc00 2023-10-06T22:12:23.069 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:12:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: starting mds.f at 2023-10-06T22:12:23.070 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:12:22 smithi190 bash[170533]: 92969380c533e6845bd4c959f78e6d4916708ab37a7b82b9a1ad6152610ae66e 2023-10-06T22:12:23.070 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:12:22 smithi190 systemd[1]: Started Ceph mds.f for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:23.594 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:23.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 -- 172.21.15.57:0/321922730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c100ea0 msgr2=0x7f239c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 --2- 172.21.15.57:0/321922730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c100ea0 0x7f239c1012c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2398009990 tx=0x7f2398009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 -- 172.21.15.57:0/321922730 shutdown_connections 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 --2- 172.21.15.57:0/321922730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f239c102a40 0x7f239c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 --2- 172.21.15.57:0/321922730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c100ea0 0x7f239c1012c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 --2- 172.21.15.57:0/321922730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f239c102080 0x7f239c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 -- 172.21.15.57:0/321922730 >> 172.21.15.57:0/321922730 conn(0x7f239c0fc420 msgr2=0x7f239c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 -- 172.21.15.57:0/321922730 shutdown_connections 2023-10-06T22:12:23.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.808+0000 7f23a39ff700 1 -- 172.21.15.57:0/321922730 wait complete. 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 Processor -- start 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 -- start start 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f239c100ea0 0x7f239c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c102080 0x7f239c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f239c102a40 0x7f239c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c073f80 con 0x7f239c102a40 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0740f0 con 0x7f239c100ea0 2023-10-06T22:12:23.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.809+0000 7f23a39ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c1a0c50 con 0x7f239c102080 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c102080 0x7f239c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a1f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f239c102a40 0x7f239c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c102080 0x7f239c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36386/0 (socket says 172.21.15.57:36386) 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 -- 172.21.15.57:0/3462482472 learned_addr learned my addr 172.21.15.57:0/3462482472 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a179b700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f239c100ea0 0x7f239c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 -- 172.21.15.57:0/3462482472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f239c100ea0 msgr2=0x7f239c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f239c100ea0 0x7f239c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 -- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f239c102a40 msgr2=0x7f239c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f239c102a40 0x7f239c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:23.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 -- 172.21.15.57:0/3462482472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980095f0 con 0x7f239c102080 2023-10-06T22:12:23.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23a0f9a700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c102080 0x7f239c0732a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f238c00d980 tx=0x7f238c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:23.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.810+0000 7f23927fc700 1 -- 172.21.15.57:0/3462482472 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c00f890 con 0x7f239c102080 2023-10-06T22:12:23.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.811+0000 7f23927fc700 1 -- 172.21.15.57:0/3462482472 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f238c010460 con 0x7f239c102080 2023-10-06T22:12:23.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.811+0000 7f23927fc700 1 -- 172.21.15.57:0/3462482472 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0195c0 con 0x7f239c102080 2023-10-06T22:12:23.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.811+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c1a0f30 con 0x7f239c102080 2023-10-06T22:12:23.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.811+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c1a1360 con 0x7f239c102080 2023-10-06T22:12:23.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.812+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c10c760 con 0x7f239c102080 2023-10-06T22:12:23.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.814+0000 7f23927fc700 1 -- 172.21.15.57:0/3462482472 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f238c0105d0 con 0x7f239c102080 2023-10-06T22:12:23.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.815+0000 7f23927fc700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23880a1550 0x7f23880a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:23.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.815+0000 7f23927fc700 1 -- 172.21.15.57:0/3462482472 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f238c0cc6b0 con 0x7f239c102080 2023-10-06T22:12:23.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.815+0000 7f23a179b700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23880a1550 0x7f23880a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:23.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.816+0000 7f23a179b700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23880a1550 0x7f23880a3a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f239800b2e0 tx=0x7f2398005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:23.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:23.817+0000 7f23927fc700 1 -- 172.21.15.57:0/3462482472 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f238c096110 con 0x7f239c102080 2023-10-06T22:12:24.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.076+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f239c107e40 con 0x7f239c102080 2023-10-06T22:12:24.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.078+0000 7f23927fc700 1 -- 172.21.15.57:0/3462482472 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f238c099730 con 0x7f239c102080 2023-10-06T22:12:24.079 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:24.080 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-06T22:12:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23880a1550 msgr2=0x7f23880a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23880a1550 0x7f23880a3a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f239800b2e0 tx=0x7f2398005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c102080 msgr2=0x7f239c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c102080 0x7f239c0732a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f238c00d980 tx=0x7f238c00dc90 comp rx=0 tx=0).stop 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 shutdown_connections 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23880a1550 0x7f23880a3a10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f239c102a40 0x7f239c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f239c100ea0 0x7f239c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 --2- 172.21.15.57:0/3462482472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f239c102080 0x7f239c0732a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 >> 172.21.15.57:0/3462482472 conn(0x7f239c0fc420 msgr2=0x7f239c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 shutdown_connections 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:24.080+0000 7f23a39ff700 1 -- 172.21.15.57:0/3462482472 wait complete. 2023-10-06T22:12:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 7 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: Deploying daemon mds.c on smithi190 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:boot 2023-10-06T22:12:24.249 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: fsmap 6 up:standby 2023-10-06T22:12:24.250 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:12:24.250 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:23 smithi190 ceph-mon[134891]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:24.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: Deploying daemon mds.c on smithi190 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:boot 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: fsmap 6 up:standby 2023-10-06T22:12:24.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:12:24.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:23 smithi156 ceph-mon[134358]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:24.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: Deploying daemon mds.c on smithi190 2023-10-06T22:12:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:boot 2023-10-06T22:12:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: fsmap 6 up:standby 2023-10-06T22:12:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:12:24.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:23 smithi057 ceph-mon[98492]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:24.565 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:12:24 smithi190 systemd[1]: Starting Ceph mds.c for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:25.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:24 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3462482472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:25.080 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:12:24 smithi190 podman[170873]: 2023-10-06T22:12:25.080 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:12:25 smithi190 ceph-mds[170913]: -- 172.21.15.190:0/1079763177 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8dd2691e0 con 0x55f8dd23f000 2023-10-06T22:12:25.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:24 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3462482472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:25.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:24 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3462482472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:25.569 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:12:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c[170889]: starting mds.c at 2023-10-06T22:12:25.570 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:12:25 smithi190 bash[170873]: c20a81c897dc1284f66a74c7e683f78e05ae09cd85aa87e55e5a57c43b73aebd 2023-10-06T22:12:25.570 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:12:25 smithi190 systemd[1]: Started Ceph mds.c for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:26.102 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 -- 172.21.15.57:0/938685355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28100ea0 msgr2=0x7fcc281012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/938685355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28100ea0 0x7fcc281012c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009990 tx=0x7fcc18009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 -- 172.21.15.57:0/938685355 shutdown_connections 2023-10-06T22:12:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/938685355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc28102a40 0x7fcc28109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/938685355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28100ea0 0x7fcc281012c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/938685355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc28102080 0x7fcc28102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 -- 172.21.15.57:0/938685355 >> 172.21.15.57:0/938685355 conn(0x7fcc280fc420 msgr2=0x7fcc280fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 -- 172.21.15.57:0/938685355 shutdown_connections 2023-10-06T22:12:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.316+0000 7fcc2dfff700 1 -- 172.21.15.57:0/938685355 wait complete. 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.317+0000 7fcc2dfff700 1 Processor -- start 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc2dfff700 1 -- start start 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc2dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc28100ea0 0x7fcc28198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc2dfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc28102080 0x7fcc28199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc2dfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28102a40 0x7fcc281a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc2dfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc281a0980 con 0x7fcc28102a40 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc2dfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc281a0ac0 con 0x7fcc28102080 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc2dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc281a0c50 con 0x7fcc28100ea0 2023-10-06T22:12:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc27fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28102a40 0x7fcc281a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc27fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28102a40 0x7fcc281a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37532/0 (socket says 172.21.15.57:37532) 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc27fff700 1 -- 172.21.15.57:0/1654538037 learned_addr learned my addr 172.21.15.57:0/1654538037 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc26ffd700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc28102080 0x7fcc28199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.318+0000 7fcc277fe700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc28100ea0 0x7fcc28198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc27fff700 1 -- 172.21.15.57:0/1654538037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc28100ea0 msgr2=0x7fcc28198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc27fff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc28100ea0 0x7fcc28198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc27fff700 1 -- 172.21.15.57:0/1654538037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc28102080 msgr2=0x7fcc28199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc27fff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc28102080 0x7fcc28199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc27fff700 1 -- 172.21.15.57:0/1654538037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc180095f0 con 0x7fcc28102a40 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc277fe700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc28100ea0 0x7fcc28198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc26ffd700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc28102080 0x7fcc28199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc27fff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28102a40 0x7fcc281a03b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000ef30 tx=0x7fcc2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc24ff9700 1 -- 172.21.15.57:0/1654538037 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20018070 con 0x7fcc28102a40 2023-10-06T22:12:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc24ff9700 1 -- 172.21.15.57:0/1654538037 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc2000f040 con 0x7fcc28102a40 2023-10-06T22:12:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc281a0ed0 con 0x7fcc28102a40 2023-10-06T22:12:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc24ff9700 1 -- 172.21.15.57:0/1654538037 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000cdb0 con 0x7fcc28102a40 2023-10-06T22:12:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.319+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc281a1350 con 0x7fcc28102a40 2023-10-06T22:12:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.320+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc28193550 con 0x7fcc28102a40 2023-10-06T22:12:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.322+0000 7fcc24ff9700 1 -- 172.21.15.57:0/1654538037 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcc20003a00 con 0x7fcc28102a40 2023-10-06T22:12:26.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.324+0000 7fcc24ff9700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc100a1550 0x7fcc100a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:26.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.324+0000 7fcc24ff9700 1 -- 172.21.15.57:0/1654538037 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcc200cc450 con 0x7fcc28102a40 2023-10-06T22:12:26.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.324+0000 7fcc277fe700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc100a1550 0x7fcc100a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:26.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.324+0000 7fcc277fe700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc100a1550 0x7fcc100a3a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800b2e0 tx=0x7fcc18005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:26.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.328+0000 7fcc24ff9700 1 -- 172.21.15.57:0/1654538037 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcc20095f40 con 0x7fcc28102a40 2023-10-06T22:12:26.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:26.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:26.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:26.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: Deploying daemon mds.h on smithi156 2023-10-06T22:12:26.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:26.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] up:boot 2023-10-06T22:12:26.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: fsmap 7 up:standby 2023-10-06T22:12:26.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:26 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:12:26.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:26.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:26.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:26.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: Deploying daemon mds.h on smithi156 2023-10-06T22:12:26.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:26.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] up:boot 2023-10-06T22:12:26.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: fsmap 7 up:standby 2023-10-06T22:12:26.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:26 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:12:26.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: Deploying daemon mds.h on smithi156 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] up:boot 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: fsmap 7 up:standby 2023-10-06T22:12:26.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:26 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:12:26.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.588+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc2810c760 con 0x7fcc28102a40 2023-10-06T22:12:26.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.589+0000 7fcc24ff9700 1 -- 172.21.15.57:0/1654538037 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7fcc20099560 con 0x7fcc28102a40 2023-10-06T22:12:26.591 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:26.591 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-06T22:12:26.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc100a1550 msgr2=0x7fcc100a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc100a1550 0x7fcc100a3a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800b2e0 tx=0x7fcc18005fd0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28102a40 msgr2=0x7fcc281a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28102a40 0x7fcc281a03b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000ef30 tx=0x7fcc2000c3a0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 shutdown_connections 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc100a1550 0x7fcc100a3a10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc28102a40 0x7fcc281a03b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc28100ea0 0x7fcc28198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 --2- 172.21.15.57:0/1654538037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc28102080 0x7fcc28199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.591+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 >> 172.21.15.57:0/1654538037 conn(0x7fcc280fc420 msgr2=0x7fcc281079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:26.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.592+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 shutdown_connections 2023-10-06T22:12:26.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:26.592+0000 7fcc2dfff700 1 -- 172.21.15.57:0/1654538037 wait complete. 2023-10-06T22:12:26.595 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-10-06T22:12:27.004 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:12:26 smithi156 systemd[1]: Starting Ceph mds.h for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:27.341 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:27 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1654538037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:27.341 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:12:27 smithi156 podman[171573]: 2023-10-06T22:12:27.341 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:12:27 smithi156 bash[171573]: 794c647a2c7a468106958cf652562e44b8ace89214b7285d490f425a630e0f6e 2023-10-06T22:12:27.342 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:12:27 smithi156 ceph-mds[171613]: -- 172.21.15.156:0/3356685066 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c40f1dd1e0 con 0x55c40f1b3000 2023-10-06T22:12:27.342 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:12:27 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: starting mds.h at 2023-10-06T22:12:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:27 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1654538037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:27.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:27 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1654538037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:27.753 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:12:27 smithi156 systemd[1]: Started Ceph mds.h for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:28.613 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: Deploying daemon mds.i on smithi190 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:28.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:boot 2023-10-06T22:12:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: fsmap 8 up:standby 2023-10-06T22:12:28.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:28 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:12:28.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:28.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:28.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:28.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: Deploying daemon mds.i on smithi190 2023-10-06T22:12:28.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:28.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:boot 2023-10-06T22:12:28.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: fsmap 8 up:standby 2023-10-06T22:12:28.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:28 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:12:28.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: Deploying daemon mds.i on smithi190 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:boot 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: fsmap 8 up:standby 2023-10-06T22:12:28.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:28 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:12:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 -- 172.21.15.57:0/2392750041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588108db0 msgr2=0x7fd58810b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:28.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 --2- 172.21.15.57:0/2392750041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588108db0 0x7fd58810b240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b0a0 tx=0x7fd58000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 -- 172.21.15.57:0/2392750041 shutdown_connections 2023-10-06T22:12:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 --2- 172.21.15.57:0/2392750041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588108db0 0x7fd58810b240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 --2- 172.21.15.57:0/2392750041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588074ef0 0x7fd588073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 --2- 172.21.15.57:0/2392750041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd588073910 0x7fd588073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 -- 172.21.15.57:0/2392750041 >> 172.21.15.57:0/2392750041 conn(0x7fd5880fc420 msgr2=0x7fd5880fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 -- 172.21.15.57:0/2392750041 shutdown_connections 2023-10-06T22:12:28.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.825+0000 7fd5901ff700 1 -- 172.21.15.57:0/2392750041 wait complete. 2023-10-06T22:12:28.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 Processor -- start 2023-10-06T22:12:28.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 -- start start 2023-10-06T22:12:28.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588073910 0x7fd588198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:28.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588074ef0 0x7fd5881993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd588108db0 0x7fd5881a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a0a30 con 0x7fd588074ef0 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a0b70 con 0x7fd588108db0 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.826+0000 7fd5901ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881a0d00 con 0x7fd588073910 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588073910 0x7fd588198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588073910 0x7fd588198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36398/0 (socket says 172.21.15.57:36398) 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 -- 172.21.15.57:0/205922555 learned_addr learned my addr 172.21.15.57:0/205922555 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58d79a700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588074ef0 0x7fd5881993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58e79c700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd588108db0 0x7fd5881a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 -- 172.21.15.57:0/205922555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd588108db0 msgr2=0x7fd5881a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd588108db0 0x7fd5881a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 -- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588074ef0 msgr2=0x7fd5881993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588074ef0 0x7fd5881993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 -- 172.21.15.57:0/205922555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580009d00 con 0x7fd588073910 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.827+0000 7fd58df9b700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588073910 0x7fd588198eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd58400e9a0 tx=0x7fd58400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.828+0000 7fd57effd700 1 -- 172.21.15.57:0/205922555 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584009790 con 0x7fd588073910 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.828+0000 7fd57effd700 1 -- 172.21.15.57:0/205922555 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd584010430 con 0x7fd588073910 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.828+0000 7fd57effd700 1 -- 172.21.15.57:0/205922555 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584018680 con 0x7fd588073910 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.828+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5881a0fe0 con 0x7fd588073910 2023-10-06T22:12:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.828+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5881a13a0 con 0x7fd588073910 2023-10-06T22:12:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.829+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd588107b70 con 0x7fd588073910 2023-10-06T22:12:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.831+0000 7fd57effd700 1 -- 172.21.15.57:0/205922555 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd5840105a0 con 0x7fd588073910 2023-10-06T22:12:28.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.832+0000 7fd57effd700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5740a14d0 0x7fd5740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:28.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.832+0000 7fd57effd700 1 -- 172.21.15.57:0/205922555 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd584014070 con 0x7fd588073910 2023-10-06T22:12:28.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.832+0000 7fd58d79a700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5740a14d0 0x7fd5740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.833+0000 7fd58d79a700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5740a14d0 0x7fd5740a3990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd588074c10 tx=0x7fd578007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:28.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:28.834+0000 7fd57effd700 1 -- 172.21.15.57:0/205922555 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5840964f0 con 0x7fd588073910 2023-10-06T22:12:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.090+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd58810ebf0 con 0x7fd588073910 2023-10-06T22:12:29.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.091+0000 7fd57effd700 1 -- 172.21.15.57:0/205922555 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7fd584099b10 con 0x7fd588073910 2023-10-06T22:12:29.093 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:29.094 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-06T22:12:29.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5740a14d0 msgr2=0x7fd5740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:29.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5740a14d0 0x7fd5740a3990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd588074c10 tx=0x7fd578007400 comp rx=0 tx=0).stop 2023-10-06T22:12:29.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588073910 msgr2=0x7fd588198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:29.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588073910 0x7fd588198eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd58400e9a0 tx=0x7fd58400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 shutdown_connections 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5740a14d0 0x7fd5740a3990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd588108db0 0x7fd5881a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd588073910 0x7fd588198eb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 --2- 172.21.15.57:0/205922555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd588074ef0 0x7fd5881993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 >> 172.21.15.57:0/205922555 conn(0x7fd5880fc420 msgr2=0x7fd588104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 shutdown_connections 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:29.094+0000 7fd5901ff700 1 -- 172.21.15.57:0/205922555 wait complete. 2023-10-06T22:12:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 9 2023-10-06T22:12:29.319 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:29 smithi190 systemd[1]: Starting Ceph mds.i for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:29.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:29 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/205922555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:29.644 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:29 smithi190 podman[171213]: 2023-10-06T22:12:29.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:29 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/205922555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:29.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:29 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/205922555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:30.069 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:29 smithi190 ceph-mds[171253]: -- 172.21.15.190:0/1134252799 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55841ef771e0 con 0x55841ef4cc00 2023-10-06T22:12:30.069 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: starting mds.i at 2023-10-06T22:12:30.069 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:29 smithi190 bash[171213]: 602cb53cc294ac912a9f8a94cfe3ee7e461ad83f4fa7c8ed4d73ac3843e69cd6 2023-10-06T22:12:30.069 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:29 smithi190 systemd[1]: Started Ceph mds.i for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] up:boot 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: fsmap 9 up:standby 2023-10-06T22:12:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:30 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:12:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] up:boot 2023-10-06T22:12:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: fsmap 9 up:standby 2023-10-06T22:12:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:30 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:12:31.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] up:boot 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: fsmap 9 up:standby 2023-10-06T22:12:31.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:30 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:12:31.116 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.324+0000 7f6caf1ff700 1 -- 172.21.15.57:0/1596654202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8100ea0 msgr2=0x7f6ca81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.324+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/1596654202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8100ea0 0x7f6ca81012c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009990 tx=0x7f6c9c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.324+0000 7f6caf1ff700 1 -- 172.21.15.57:0/1596654202 shutdown_connections 2023-10-06T22:12:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.325+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/1596654202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8102a40 0x7f6ca8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.325+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/1596654202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8100ea0 0x7f6ca81012c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.325+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/1596654202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca8102080 0x7f6ca8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.325+0000 7f6caf1ff700 1 -- 172.21.15.57:0/1596654202 >> 172.21.15.57:0/1596654202 conn(0x7f6ca80fc420 msgr2=0x7f6ca80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.325+0000 7f6caf1ff700 1 -- 172.21.15.57:0/1596654202 shutdown_connections 2023-10-06T22:12:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.325+0000 7f6caf1ff700 1 -- 172.21.15.57:0/1596654202 wait complete. 2023-10-06T22:12:31.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 Processor -- start 2023-10-06T22:12:31.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 -- start start 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca8100ea0 0x7f6ca8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8102080 0x7f6ca8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8102a40 0x7f6ca81a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81a0970 con 0x7f6ca8102a40 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81a0ab0 con 0x7f6ca8100ea0 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6caf1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81a0c40 con 0x7f6ca8102080 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6cad79c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8102a40 0x7f6ca81a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6cacf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca8100ea0 0x7f6ca8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6ca7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8102080 0x7f6ca8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.326+0000 7f6cad79c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8102a40 0x7f6ca81a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37568/0 (socket says 172.21.15.57:37568) 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6cad79c700 1 -- 172.21.15.57:0/386998992 learned_addr learned my addr 172.21.15.57:0/386998992 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6cad79c700 1 -- 172.21.15.57:0/386998992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8102080 msgr2=0x7f6ca8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6cad79c700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8102080 0x7f6ca8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6cad79c700 1 -- 172.21.15.57:0/386998992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca8100ea0 msgr2=0x7f6ca8198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6cad79c700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca8100ea0 0x7f6ca8198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6cad79c700 1 -- 172.21.15.57:0/386998992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0095f0 con 0x7f6ca8102a40 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6cad79c700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8102a40 0x7f6ca81a03a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ef30 tx=0x7f6ca000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6ca5ffb700 1 -- 172.21.15.57:0/386998992 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0018070 con 0x7f6ca8102a40 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6ca5ffb700 1 -- 172.21.15.57:0/386998992 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ca000f040 con 0x7f6ca8102a40 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6ca5ffb700 1 -- 172.21.15.57:0/386998992 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000cd00 con 0x7f6ca8102a40 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.327+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca81a0ec0 con 0x7f6ca8102a40 2023-10-06T22:12:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.328+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca81a12f0 con 0x7f6ca8102a40 2023-10-06T22:12:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.329+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca810c760 con 0x7f6ca8102a40 2023-10-06T22:12:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.331+0000 7f6ca5ffb700 1 -- 172.21.15.57:0/386998992 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6ca0003a00 con 0x7f6ca8102a40 2023-10-06T22:12:31.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.332+0000 7f6ca5ffb700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6c980a1550 0x7f6c980a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:31.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.332+0000 7f6ca5ffb700 1 -- 172.21.15.57:0/386998992 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ca00ccd10 con 0x7f6ca8102a40 2023-10-06T22:12:31.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.333+0000 7f6cacf9b700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6c980a1550 0x7f6c980a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:31.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.333+0000 7f6cacf9b700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6c980a1550 0x7f6c980a3a10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b2e0 tx=0x7f6c9c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.334+0000 7f6ca5ffb700 1 -- 172.21.15.57:0/386998992 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6ca0096800 con 0x7f6ca8102a40 2023-10-06T22:12:31.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.618+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ca810c9a0 con 0x7f6ca8102a40 2023-10-06T22:12:31.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.619+0000 7f6ca5ffb700 1 -- 172.21.15.57:0/386998992 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7f6ca0099e20 con 0x7f6ca8102a40 2023-10-06T22:12:31.621 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:31.622 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6c980a1550 msgr2=0x7f6c980a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6c980a1550 0x7f6c980a3a10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b2e0 tx=0x7f6c9c005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8102a40 msgr2=0x7f6ca81a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8102a40 0x7f6ca81a03a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ef30 tx=0x7f6ca000c3a0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 shutdown_connections 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6c980a1550 0x7f6c980a3a10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ca8102a40 0x7f6ca81a03a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca8100ea0 0x7f6ca8198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 --2- 172.21.15.57:0/386998992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8102080 0x7f6ca8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:31.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 >> 172.21.15.57:0/386998992 conn(0x7f6ca80fc420 msgr2=0x7f6ca80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:31.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 shutdown_connections 2023-10-06T22:12:31.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:31.622+0000 7f6caf1ff700 1 -- 172.21.15.57:0/386998992 wait complete. 2023-10-06T22:12:31.625 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 10 2023-10-06T22:12:32.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:31 smithi156 ceph-mon[134358]: Deploying daemon mds.g on smithi057 2023-10-06T22:12:32.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:31 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/386998992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:31 smithi190 ceph-mon[134891]: Deploying daemon mds.g on smithi057 2023-10-06T22:12:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:31 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/386998992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:32.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:31 smithi057 ceph-mon[98492]: Deploying daemon mds.g on smithi057 2023-10-06T22:12:32.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:31 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/386998992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:32.423 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:12:32 smithi057 systemd[1]: Starting Ceph mds.g for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:32.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:32 smithi057 ceph-mon[98492]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:32.864 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:12:32 smithi057 podman[150031]: 2023-10-06T22:12:32.864 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:12:32 smithi057 bash[150031]: a33466e8eed0787cbbe482021c390073dd3427eaadd8ef32aa5266616e85e0b0 2023-10-06T22:12:32.865 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:12:32 smithi057 ceph-mds[150071]: -- 172.21.15.57:0/1583934587 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b0e5cb91e0 con 0x55b0e5c8f000 2023-10-06T22:12:32.865 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:12:32 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-g[150047]: starting mds.g at 2023-10-06T22:12:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:32 smithi190 ceph-mon[134891]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:33.172 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:12:32 smithi057 systemd[1]: Started Ceph mds.g for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:33.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:32 smithi156 ceph-mon[134358]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:33.644 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:33.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 -- 172.21.15.57:0/2737578676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102a30 msgr2=0x7fd708109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:33.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 --2- 172.21.15.57:0/2737578676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102a30 0x7fd708109350 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd70000b0a0 tx=0x7fd70000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:33.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 -- 172.21.15.57:0/2737578676 shutdown_connections 2023-10-06T22:12:33.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 --2- 172.21.15.57:0/2737578676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102a30 0x7fd708109350 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:33.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 --2- 172.21.15.57:0/2737578676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd708100e90 0x7fd7081012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:33.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 --2- 172.21.15.57:0/2737578676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd708102070 0x7fd7081024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:33.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 -- 172.21.15.57:0/2737578676 >> 172.21.15.57:0/2737578676 conn(0x7fd7080fc430 msgr2=0x7fd7080fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:33.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 -- 172.21.15.57:0/2737578676 shutdown_connections 2023-10-06T22:12:33.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.858+0000 7fd7103ff700 1 -- 172.21.15.57:0/2737578676 wait complete. 2023-10-06T22:12:33.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.859+0000 7fd7103ff700 1 Processor -- start 2023-10-06T22:12:33.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.859+0000 7fd7103ff700 1 -- start start 2023-10-06T22:12:33.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd7103ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd708100e90 0x7fd708074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:33.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd7103ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102070 0x7fd7080732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd7103ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd708102a30 0x7fd7080737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd7103ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708073f80 con 0x7fd708102070 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd7103ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080740f0 con 0x7fd708100e90 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd7103ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081a0d20 con 0x7fd708102a30 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd70d99a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102070 0x7fd7080732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd70d99a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102070 0x7fd7080732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46404/0 (socket says 172.21.15.57:46404) 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd70d99a700 1 -- 172.21.15.57:0/529175439 learned_addr learned my addr 172.21.15.57:0/529175439 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd70e19b700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd708100e90 0x7fd708074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.860+0000 7fd70e99c700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd708102a30 0x7fd7080737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd70d99a700 1 -- 172.21.15.57:0/529175439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd708102a30 msgr2=0x7fd7080737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:33.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd70d99a700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd708102a30 0x7fd7080737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd70d99a700 1 -- 172.21.15.57:0/529175439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd708100e90 msgr2=0x7fd708074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd70d99a700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd708100e90 0x7fd708074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd70d99a700 1 -- 172.21.15.57:0/529175439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009d00 con 0x7fd708102070 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd70d99a700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102070 0x7fd7080732a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800e970 tx=0x7fd6f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd6ff7fe700 1 -- 172.21.15.57:0/529175439 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8009790 con 0x7fd708102070 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd6ff7fe700 1 -- 172.21.15.57:0/529175439 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6f800eec0 con 0x7fd708102070 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7081a0fa0 con 0x7fd708102070 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd6ff7fe700 1 -- 172.21.15.57:0/529175439 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8003cf0 con 0x7fd708102070 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.861+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7081a14d0 con 0x7fd708102070 2023-10-06T22:12:33.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.862+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd708108430 con 0x7fd708102070 2023-10-06T22:12:33.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.864+0000 7fd6ff7fe700 1 -- 172.21.15.57:0/529175439 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd6f800ccd0 con 0x7fd708102070 2023-10-06T22:12:33.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.866+0000 7fd6ff7fe700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd6f40a1550 0x7fd6f40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:33.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.866+0000 7fd6ff7fe700 1 -- 172.21.15.57:0/529175439 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6f8014070 con 0x7fd708102070 2023-10-06T22:12:33.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.866+0000 7fd70e19b700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd6f40a1550 0x7fd6f40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:33.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.867+0000 7fd70e19b700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd6f40a1550 0x7fd6f40a3a10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd708101be0 tx=0x7fd70400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:33.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:33.869+0000 7fd6ff7fe700 1 -- 172.21.15.57:0/529175439 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd6f8096580 con 0x7fd708102070 2023-10-06T22:12:33.974 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:33.974 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:33.974 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: Deploying daemon mds.a on smithi057 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.57:6838/3033019918,v1:172.21.15.57:6839/3033019918] up:boot 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: fsmap 10 up:standby 2023-10-06T22:12:33.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:33 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:12:34.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.160+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd70802d020 con 0x7fd708102070 2023-10-06T22:12:34.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.160+0000 7fd6ff7fe700 1 -- 172.21.15.57:0/529175439 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7fd6f8099ba0 con 0x7fd708102070 2023-10-06T22:12:34.162 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:34.162 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14523,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T22:12:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd6f40a1550 msgr2=0x7fd6f40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd6f40a1550 0x7fd6f40a3a10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd708101be0 tx=0x7fd70400b410 comp rx=0 tx=0).stop 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102070 msgr2=0x7fd7080732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102070 0x7fd7080732a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800e970 tx=0x7fd6f800ec80 comp rx=0 tx=0).stop 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 shutdown_connections 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd6f40a1550 0x7fd6f40a3a10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd708102a30 0x7fd7080737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd708100e90 0x7fd708074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 --2- 172.21.15.57:0/529175439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd708102070 0x7fd7080732a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 >> 172.21.15.57:0/529175439 conn(0x7fd7080fc430 msgr2=0x7fd708105570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 shutdown_connections 2023-10-06T22:12:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:34.163+0000 7fd7103ff700 1 -- 172.21.15.57:0/529175439 wait complete. 2023-10-06T22:12:34.166 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 11 2023-10-06T22:12:34.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: Deploying daemon mds.a on smithi057 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.57:6838/3033019918,v1:172.21.15.57:6839/3033019918] up:boot 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: fsmap 10 up:standby 2023-10-06T22:12:34.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:33 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:12:34.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:34.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:34.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:34.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:34.322 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:34.322 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:34.322 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: Deploying daemon mds.a on smithi057 2023-10-06T22:12:34.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:34.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.57:6838/3033019918,v1:172.21.15.57:6839/3033019918] up:boot 2023-10-06T22:12:34.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: fsmap 10 up:standby 2023-10-06T22:12:34.323 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:33 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:12:35.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:34 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/529175439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:35.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:34 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/529175439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:35.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:34 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/529175439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:35.673 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:12:35 smithi057 systemd[1]: Starting Ceph mds.a for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:36.094 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:12:35 smithi057 podman[150419]: 2023-10-06T22:12:36.185 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 -- 172.21.15.57:0/1445984479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8100ea0 msgr2=0x7f08f81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 --2- 172.21.15.57:0/1445984479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8100ea0 0x7f08f81012c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009990 tx=0x7f08e8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 -- 172.21.15.57:0/1445984479 shutdown_connections 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 --2- 172.21.15.57:0/1445984479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08f8102a40 0x7f08f8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 --2- 172.21.15.57:0/1445984479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8100ea0 0x7f08f81012c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 --2- 172.21.15.57:0/1445984479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08f8102080 0x7f08f8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 -- 172.21.15.57:0/1445984479 >> 172.21.15.57:0/1445984479 conn(0x7f08f80fc420 msgr2=0x7f08f80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.393+0000 7f08fddff700 1 -- 172.21.15.57:0/1445984479 shutdown_connections 2023-10-06T22:12:36.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.394+0000 7f08fddff700 1 -- 172.21.15.57:0/1445984479 wait complete. 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 Processor -- start 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 -- start start 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08f8100ea0 0x7f08f8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8102080 0x7f08f8197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08f8102a40 0x7f08f819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819e770 con 0x7f08f8102a40 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819e8b0 con 0x7f08f8102080 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.395+0000 7f08fddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819ea40 con 0x7f08f8100ea0 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08f8102a40 0x7f08f819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8102080 0x7f08f8197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08f8100ea0 0x7f08f8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:36.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8102080 0x7f08f8197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34464/0 (socket says 172.21.15.57:34464) 2023-10-06T22:12:36.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 -- 172.21.15.57:0/1659080255 learned_addr learned my addr 172.21.15.57:0/1659080255 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 -- 172.21.15.57:0/1659080255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08f8100ea0 msgr2=0x7f08f8196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08f8100ea0 0x7f08f8196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 -- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08f8102a40 msgr2=0x7f08f819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08f8102a40 0x7f08f819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 -- 172.21.15.57:0/1659080255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80095f0 con 0x7f08f8102080 2023-10-06T22:12:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f77fe700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08f8100ea0 0x7f08f8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f7fff700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08f8102a40 0x7f08f819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f6ffd700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8102080 0x7f08f8197130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00e970 tx=0x7f08ec00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f4ff9700 1 -- 172.21.15.57:0/1659080255 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec009790 con 0x7f08f8102080 2023-10-06T22:12:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f4ff9700 1 -- 172.21.15.57:0/1659080255 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08ec00c7e0 con 0x7f08f8102080 2023-10-06T22:12:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08f4ff9700 1 -- 172.21.15.57:0/1659080255 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec003c70 con 0x7f08f8102080 2023-10-06T22:12:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.396+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f819ed20 con 0x7f08f8102080 2023-10-06T22:12:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.397+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f819f180 con 0x7f08f8102080 2023-10-06T22:12:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.397+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f810c760 con 0x7f08f8102080 2023-10-06T22:12:36.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.400+0000 7f08f4ff9700 1 -- 172.21.15.57:0/1659080255 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f08ec00cc50 con 0x7f08f8102080 2023-10-06T22:12:36.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.401+0000 7f08f4ff9700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f08e00a14d0 0x7f08e00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:36.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.401+0000 7f08f4ff9700 1 -- 172.21.15.57:0/1659080255 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f08ec014070 con 0x7f08f8102080 2023-10-06T22:12:36.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.401+0000 7f08f77fe700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f08e00a14d0 0x7f08e00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:36.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.402+0000 7f08f77fe700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f08e00a14d0 0x7f08e00a3990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b2e0 tx=0x7f08e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:36.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.403+0000 7f08f4ff9700 1 -- 172.21.15.57:0/1659080255 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f08ec096900 con 0x7f08f8102080 2023-10-06T22:12:36.422 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:12:36 smithi057 bash[150419]: 86c3e142d113d167159346974e729a0b30f437149729cb3bf361192dbd744cbf 2023-10-06T22:12:36.423 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mds[150459]: -- 172.21.15.57:0/1815755536 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5650aada11e0 con 0x5650aad76c00 2023-10-06T22:12:36.423 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-a[150435]: starting mds.a at 2023-10-06T22:12:36.423 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:12:36 smithi057 systemd[1]: Started Ceph mds.a for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:36.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.659+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08f819f3d0 con 0x7f08f8102080 2023-10-06T22:12:36.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.660+0000 7f08f4ff9700 1 -- 172.21.15.57:0/1659080255 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7f08ec003dd0 con 0x7f08f8102080 2023-10-06T22:12:36.661 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:36.661 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14523,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f08e00a14d0 msgr2=0x7f08e00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f08e00a14d0 0x7f08e00a3990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b2e0 tx=0x7f08e8005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8102080 msgr2=0x7f08f8197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8102080 0x7f08f8197130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00e970 tx=0x7f08ec00ec80 comp rx=0 tx=0).stop 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 shutdown_connections 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f08e00a14d0 0x7f08e00a3990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08f8102a40 0x7f08f819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08f8100ea0 0x7f08f8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 --2- 172.21.15.57:0/1659080255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8102080 0x7f08f8197130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:36.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.662+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 >> 172.21.15.57:0/1659080255 conn(0x7f08f80fc420 msgr2=0x7f08f80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:36.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.663+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 shutdown_connections 2023-10-06T22:12:36.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:36.663+0000 7f08fddff700 1 -- 172.21.15.57:0/1659080255 wait complete. 2023-10-06T22:12:36.665 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 11 2023-10-06T22:12:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mon[98492]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:36.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:36.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:36 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:36.926 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:36 smithi156 ceph-mon[134358]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:36.926 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:36 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:36.926 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:36 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:36.926 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:36 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:36.926 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:36 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:36.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:36 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:36.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:36 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:37.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:36 smithi190 ceph-mon[134891]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:37.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:36 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:37.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:36 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:37.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:36 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:37.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:36 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T22:12:37.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:36 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T22:12:37.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:36 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:37.837 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:12:37 smithi156 systemd[1]: Starting Ceph mds.b for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:12:37.837 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:37 smithi156 ceph-mon[134358]: Deploying daemon mds.b on smithi156 2023-10-06T22:12:37.837 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:37 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1659080255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:37.837 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:37 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.57:6840/490100321,v1:172.21.15.57:6841/490100321] up:boot 2023-10-06T22:12:37.837 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:37 smithi156 ceph-mon[134358]: fsmap 11 up:standby 2023-10-06T22:12:37.837 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:37 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:12:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:37 smithi057 ceph-mon[98492]: Deploying daemon mds.b on smithi156 2023-10-06T22:12:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:37 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1659080255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:37 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.57:6840/490100321,v1:172.21.15.57:6841/490100321] up:boot 2023-10-06T22:12:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:37 smithi057 ceph-mon[98492]: fsmap 11 up:standby 2023-10-06T22:12:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:37 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:12:38.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:37 smithi190 ceph-mon[134891]: Deploying daemon mds.b on smithi156 2023-10-06T22:12:38.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:37 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1659080255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:38.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:37 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.57:6840/490100321,v1:172.21.15.57:6841/490100321] up:boot 2023-10-06T22:12:38.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:37 smithi190 ceph-mon[134891]: fsmap 11 up:standby 2023-10-06T22:12:38.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:37 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:12:38.421 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:12:38 smithi156 podman[171912]: 2023-10-06T22:12:38.421 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-mds[171952]: -- 172.21.15.156:0/1608177602 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea9270b1e0 con 0x55ea926e0c00 2023-10-06T22:12:38.421 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-b[171928]: starting mds.b at 2023-10-06T22:12:38.422 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:12:38 smithi156 bash[171912]: 836c3f2609a5aff55967b79d748c13bc054fb0c03e8346a228bfe510ab29e865 2023-10-06T22:12:38.684 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:38.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-mon[134358]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:38.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:38 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.754 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:12:38 smithi156 systemd[1]: Started Ceph mds.b for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:12:38.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 -- 172.21.15.57:0/1681339572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c01090a0 msgr2=0x7f14c010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:38.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1681339572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c01090a0 0x7f14c010f950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f14b800b0d0 tx=0x7f14b800b3e0 comp rx=0 tx=0).stop 2023-10-06T22:12:38.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 -- 172.21.15.57:0/1681339572 shutdown_connections 2023-10-06T22:12:38.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1681339572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c01090a0 0x7f14c010f950 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:38.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1681339572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c0107510 0x7f14c0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:38.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1681339572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 0x7f14c0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:38.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 -- 172.21.15.57:0/1681339572 >> 172.21.15.57:0/1681339572 conn(0x7f14c0076490 msgr2=0x7f14c00788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:38.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 -- 172.21.15.57:0/1681339572 shutdown_connections 2023-10-06T22:12:38.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.901+0000 7f14c4bff700 1 -- 172.21.15.57:0/1681339572 wait complete. 2023-10-06T22:12:38.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.902+0000 7f14c4bff700 1 Processor -- start 2023-10-06T22:12:38.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.902+0000 7f14c4bff700 1 -- start start 2023-10-06T22:12:38.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14c4bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0107510 0x7f14c019d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:38.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14c4bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 0x7f14c019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:38.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14c4bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c01090a0 0x7f14c01a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:38.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14c4bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01a4da0 con 0x7f14c0107510 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14c4bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01a4ee0 con 0x7f14c01090a0 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14c4bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01a5070 con 0x7f14c0108700 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14be59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0107510 0x7f14c019d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 0x7f14c019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 0x7f14c019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52728/0 (socket says 172.21.15.57:52728) 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 -- 172.21.15.57:0/1569986669 learned_addr learned my addr 172.21.15.57:0/1569986669 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14bed9d700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c01090a0 0x7f14c01a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 -- 172.21.15.57:0/1569986669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c01090a0 msgr2=0x7f14c01a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:38.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c01090a0 0x7f14c01a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 -- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0107510 msgr2=0x7f14c019d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0107510 0x7f14c019d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14b7fff700 1 -- 172.21.15.57:0/1569986669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8009d00 con 0x7f14c0108700 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.903+0000 7f14be59c700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0107510 0x7f14c019d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.904+0000 7f14b7fff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 0x7f14c019d760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14a800e9a0 tx=0x7f14a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.904+0000 7f14b77fe700 1 -- 172.21.15.57:0/1569986669 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8009790 con 0x7f14c0108700 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.904+0000 7f14b77fe700 1 -- 172.21.15.57:0/1569986669 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a8010430 con 0x7f14c0108700 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.904+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c01a5350 con 0x7f14c0108700 2023-10-06T22:12:38.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.904+0000 7f14b77fe700 1 -- 172.21.15.57:0/1569986669 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8018680 con 0x7f14c0108700 2023-10-06T22:12:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.904+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c01a5860 con 0x7f14c0108700 2023-10-06T22:12:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.905+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c0112d20 con 0x7f14c0108700 2023-10-06T22:12:38.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.909+0000 7f14b77fe700 1 -- 172.21.15.57:0/1569986669 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f14a80105a0 con 0x7f14c0108700 2023-10-06T22:12:38.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.910+0000 7f14b77fe700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f14ac0a14d0 0x7f14ac0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:38.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.910+0000 7f14b77fe700 1 -- 172.21.15.57:0/1569986669 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14a8014070 con 0x7f14c0108700 2023-10-06T22:12:38.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.911+0000 7f14be59c700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f14ac0a14d0 0x7f14ac0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:38.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.911+0000 7f14b77fe700 1 -- 172.21.15.57:0/1569986669 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f14a8096510 con 0x7f14c0108700 2023-10-06T22:12:38.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:38.911+0000 7f14be59c700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f14ac0a14d0 0x7f14ac0a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14c0108560 tx=0x7f14b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:38 smithi057 ceph-mon[98492]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:38.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:38 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:39.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:38 smithi190 ceph-mon[134891]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:39.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:39.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:39.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:39.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:39.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:38 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:39.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.187+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14c010e340 con 0x7f14c0108700 2023-10-06T22:12:39.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.189+0000 7f14b77fe700 1 -- 172.21.15.57:0/1569986669 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7f14a8099b30 con 0x7f14c0108700 2023-10-06T22:12:39.190 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:39.190 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14523,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f14ac0a14d0 msgr2=0x7f14ac0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f14ac0a14d0 0x7f14ac0a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14c0108560 tx=0x7f14b0009260 comp rx=0 tx=0).stop 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 msgr2=0x7f14c019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 0x7f14c019d760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14a800e9a0 tx=0x7f14a800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 shutdown_connections 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f14ac0a14d0 0x7f14ac0a3990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c01090a0 0x7f14c01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0107510 0x7f14c019d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 --2- 172.21.15.57:0/1569986669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0108700 0x7f14c019d760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 >> 172.21.15.57:0/1569986669 conn(0x7f14c0076490 msgr2=0x7f14c0078660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:39.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 shutdown_connections 2023-10-06T22:12:39.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.191+0000 7f14c4bff700 1 -- 172.21.15.57:0/1569986669 wait complete. 2023-10-06T22:12:39.194 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-06T22:12:39.211 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:12:39.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.420+0000 7f7634bff700 1 -- 172.21.15.57:0/1985721936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7630106c00 msgr2=0x7f7630109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.420+0000 7f7634bff700 1 --2- 172.21.15.57:0/1985721936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7630106c00 0x7f7630109070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f762800b0a0 tx=0x7f762800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.420+0000 7f7634bff700 1 -- 172.21.15.57:0/1985721936 shutdown_connections 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.420+0000 7f7634bff700 1 --2- 172.21.15.57:0/1985721936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7630106c00 0x7f7630109070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.420+0000 7f7634bff700 1 --2- 172.21.15.57:0/1985721936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76301019a0 0x7f7630103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.420+0000 7f7634bff700 1 --2- 172.21.15.57:0/1985721936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301042d0 0x7f76301066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.420+0000 7f7634bff700 1 -- 172.21.15.57:0/1985721936 >> 172.21.15.57:0/1985721936 conn(0x7f76300fb3b0 msgr2=0x7f76300fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.421+0000 7f7634bff700 1 -- 172.21.15.57:0/1985721936 shutdown_connections 2023-10-06T22:12:39.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.421+0000 7f7634bff700 1 -- 172.21.15.57:0/1985721936 wait complete. 2023-10-06T22:12:39.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 Processor -- start 2023-10-06T22:12:39.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 -- start start 2023-10-06T22:12:39.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301019a0 0x7f7630198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76301042d0 0x7f7630199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7630106c00 0x7f76301a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76301a0970 con 0x7f76301042d0 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76301a0ab0 con 0x7f7630106c00 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f7634bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76301a0c40 con 0x7f76301019a0 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f762dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76301042d0 0x7f7630199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.422+0000 7f762e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301019a0 0x7f7630198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301019a0 0x7f7630198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52750/0 (socket says 172.21.15.57:52750) 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 -- 172.21.15.57:0/3618449668 learned_addr learned my addr 172.21.15.57:0/3618449668 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762ed9d700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7630106c00 0x7f76301a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 -- 172.21.15.57:0/3618449668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7630106c00 msgr2=0x7f76301a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7630106c00 0x7f76301a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 -- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76301042d0 msgr2=0x7f7630199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76301042d0 0x7f7630199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 -- 172.21.15.57:0/3618449668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628009d00 con 0x7f76301019a0 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f762e59c700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301019a0 0x7f7630198df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f762000e9a0 tx=0x7f762000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f761f7fe700 1 -- 172.21.15.57:0/3618449668 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620009790 con 0x7f76301019a0 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f761f7fe700 1 -- 172.21.15.57:0/3618449668 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7620010430 con 0x7f76301019a0 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76301a0f20 con 0x7f76301019a0 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.423+0000 7f761f7fe700 1 -- 172.21.15.57:0/3618449668 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620018680 con 0x7f76301019a0 2023-10-06T22:12:39.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.424+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76301a1340 con 0x7f76301019a0 2023-10-06T22:12:39.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.425+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7630108780 con 0x7f76301019a0 2023-10-06T22:12:39.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.427+0000 7f761f7fe700 1 -- 172.21.15.57:0/3618449668 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f76200105a0 con 0x7f76301019a0 2023-10-06T22:12:39.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.428+0000 7f761f7fe700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76180a14d0 0x7f76180a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.428+0000 7f761f7fe700 1 -- 172.21.15.57:0/3618449668 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7620014070 con 0x7f76301019a0 2023-10-06T22:12:39.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.428+0000 7f762dd9b700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76180a14d0 0x7f76180a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.429+0000 7f762dd9b700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76180a14d0 0x7f76180a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7624009c60 tx=0x7f7624009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:39.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.432+0000 7f761f7fe700 1 -- 172.21.15.57:0/3618449668 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7620052100 con 0x7f76301019a0 2023-10-06T22:12:39.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.659+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f763010c700 con 0x7f76301019a0 2023-10-06T22:12:39.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.669+0000 7f761f7fe700 1 -- 172.21.15.57:0/3618449668 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18412 (secure 0 0 0) 0x7f7620096500 con 0x7f76301019a0 2023-10-06T22:12:39.671 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":79,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:11:56.365760+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:12:39.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76180a14d0 msgr2=0x7f76180a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76180a14d0 0x7f76180a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7624009c60 tx=0x7f7624009260 comp rx=0 tx=0).stop 2023-10-06T22:12:39.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301019a0 msgr2=0x7f7630198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301019a0 0x7f7630198df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f762000e9a0 tx=0x7f762000ecb0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 shutdown_connections 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76180a14d0 0x7f76180a3990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7630106c00 0x7f76301a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76301019a0 0x7f7630198df0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 --2- 172.21.15.57:0/3618449668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76301042d0 0x7f7630199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 >> 172.21.15.57:0/3618449668 conn(0x7f76300fb3b0 msgr2=0x7f7630100020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 shutdown_connections 2023-10-06T22:12:39.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.672+0000 7f7634bff700 1 -- 172.21.15.57:0/3618449668 wait complete. 2023-10-06T22:12:39.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:12:39.694 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/1140885481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541030f0 msgr2=0x7f2d54103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/1140885481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541030f0 0x7f2d54103510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009990 tx=0x7f2d44009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/1140885481 shutdown_connections 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/1140885481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54104c80 0x7f2d5410b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/1140885481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541030f0 0x7f2d54103510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/1140885481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d541042e0 0x7f2d54104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/1140885481 >> 172.21.15.57:0/1140885481 conn(0x7f2d540fe660 msgr2=0x7f2d54100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:39.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/1140885481 shutdown_connections 2023-10-06T22:12:39.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.909+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/1140885481 wait complete. 2023-10-06T22:12:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.910+0000 7f2d5a1ff700 1 Processor -- start 2023-10-06T22:12:39.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d5a1ff700 1 -- start start 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d5a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d541030f0 0x7f2d54198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d5a1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541042e0 0x7f2d54199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d5a1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54104c80 0x7f2d541a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d5a1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d541a0990 con 0x7f2d54104c80 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d5a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d541a0ad0 con 0x7f2d541030f0 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d5a1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d541a0c60 con 0x7f2d541042e0 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d52ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541042e0 0x7f2d54199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d541030f0 0x7f2d54198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d52ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541042e0 0x7f2d54199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52762/0 (socket says 172.21.15.57:52762) 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d52ffd700 1 -- 172.21.15.57:0/4133924334 learned_addr learned my addr 172.21.15.57:0/4133924334 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.911+0000 7f2d53fff700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54104c80 0x7f2d541a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d52ffd700 1 -- 172.21.15.57:0/4133924334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d541030f0 msgr2=0x7f2d54198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d52ffd700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d541030f0 0x7f2d54198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d52ffd700 1 -- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54104c80 msgr2=0x7f2d541a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d52ffd700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54104c80 0x7f2d541a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d52ffd700 1 -- 172.21.15.57:0/4133924334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d48009520 con 0x7f2d541042e0 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d52ffd700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541042e0 0x7f2d54199350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800eb30 tx=0x7f2d4800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d50ff9700 1 -- 172.21.15.57:0/4133924334 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48009880 con 0x7f2d541042e0 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d50ff9700 1 -- 172.21.15.57:0/4133924334 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d480044e0 con 0x7f2d541042e0 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d50ff9700 1 -- 172.21.15.57:0/4133924334 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48003d80 con 0x7f2d541042e0 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d440095f0 con 0x7f2d541042e0 2023-10-06T22:12:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.912+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d541a1260 con 0x7f2d541042e0 2023-10-06T22:12:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.913+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5410a680 con 0x7f2d541042e0 2023-10-06T22:12:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.915+0000 7f2d50ff9700 1 -- 172.21.15.57:0/4133924334 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2d48003f20 con 0x7f2d541042e0 2023-10-06T22:12:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.916+0000 7f2d50ff9700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2d3c0a1500 0x7f2d3c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.917+0000 7f2d50ff9700 1 -- 172.21.15.57:0/4133924334 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2d48014070 con 0x7f2d541042e0 2023-10-06T22:12:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.917+0000 7f2d537fe700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2d3c0a1500 0x7f2d3c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.917+0000 7f2d537fe700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2d3c0a1500 0x7f2d3c0a39c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400b2e0 tx=0x7f2d44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:39.918+0000 7f2d50ff9700 1 -- 172.21.15.57:0/4133924334 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2d48096c00 con 0x7f2d541042e0 2023-10-06T22:12:39.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:39 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:39.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:39 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] up:boot 2023-10-06T22:12:39.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:39 smithi057 ceph-mon[98492]: fsmap 12 up:standby 2023-10-06T22:12:39.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:39 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:12:39.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:39 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1569986669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:40.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:39 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:40.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:39 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] up:boot 2023-10-06T22:12:40.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:39 smithi156 ceph-mon[134358]: fsmap 12 up:standby 2023-10-06T22:12:40.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:39 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:12:40.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:39 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1569986669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:39 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:39 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] up:boot 2023-10-06T22:12:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:39 smithi190 ceph-mon[134891]: fsmap 12 up:standby 2023-10-06T22:12:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:39 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:12:40.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:39 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1569986669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:40.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.165+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d5410eaf0 con 0x7f2d541042e0 2023-10-06T22:12:40.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.165+0000 7f2d50ff9700 1 -- 172.21.15.57:0/4133924334 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d4809a220 con 0x7f2d541042e0 2023-10-06T22:12:40.167 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2d3c0a1500 msgr2=0x7f2d3c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2d3c0a1500 0x7f2d3c0a39c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400b2e0 tx=0x7f2d44005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541042e0 msgr2=0x7f2d54199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541042e0 0x7f2d54199350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800eb30 tx=0x7f2d4800ee40 comp rx=0 tx=0).stop 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 shutdown_connections 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2d3c0a1500 0x7f2d3c0a39c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d54104c80 0x7f2d541a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d541030f0 0x7f2d54198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 --2- 172.21.15.57:0/4133924334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d541042e0 0x7f2d54199350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 >> 172.21.15.57:0/4133924334 conn(0x7f2d540fe660 msgr2=0x7f2d541077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 shutdown_connections 2023-10-06T22:12:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.168+0000 7f2d5a1ff700 1 -- 172.21.15.57:0/4133924334 wait complete. 2023-10-06T22:12:40.191 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T22:12:40.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1950672835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb514101950 msgr2=0x7fb514108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1950672835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb514101950 0x7fb514108200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00b0a0 tx=0x7fb50c00b3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1950672835 shutdown_connections 2023-10-06T22:12:40.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1950672835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb514101950 0x7fb514108200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1950672835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5140ffdc0 0x7fb5141001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1950672835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb514100fb0 0x7fb514101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1950672835 >> 172.21.15.57:0/1950672835 conn(0x7fb5140fb310 msgr2=0x7fb5140fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:40.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1950672835 shutdown_connections 2023-10-06T22:12:40.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.405+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1950672835 wait complete. 2023-10-06T22:12:40.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.406+0000 7fb51c1ff700 1 Processor -- start 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.406+0000 7fb51c1ff700 1 -- start start 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51c1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5140ffdc0 0x7fb514103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51c1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb514100fb0 0x7fb514102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51c1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb514101950 0x7fb5141028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51c1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb514103070 con 0x7fb514100fb0 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51c1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5141031e0 con 0x7fb514101950 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51c1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5141a0d00 con 0x7fb5140ffdc0 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5140ffdc0 0x7fb514103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5140ffdc0 0x7fb514103d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52784/0 (socket says 172.21.15.57:52784) 2023-10-06T22:12:40.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 -- 172.21.15.57:0/1499022261 learned_addr learned my addr 172.21.15.57:0/1499022261 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51979a700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb514100fb0 0x7fb514102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51a79c700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb514101950 0x7fb5141028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 -- 172.21.15.57:0/1499022261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb514101950 msgr2=0x7fb5141028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb514101950 0x7fb5141028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 -- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb514100fb0 msgr2=0x7fb514102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb514100fb0 0x7fb514102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb519f9b700 1 -- 172.21.15.57:0/1499022261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c009d00 con 0x7fb5140ffdc0 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51979a700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb514100fb0 0x7fb514102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.407+0000 7fb51a79c700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb514101950 0x7fb5141028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:40.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.408+0000 7fb519f9b700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5140ffdc0 0x7fb514103d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e970 tx=0x7fb51000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:40.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.408+0000 7fb50affd700 1 -- 172.21.15.57:0/1499022261 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510009790 con 0x7fb5140ffdc0 2023-10-06T22:12:40.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.408+0000 7fb50affd700 1 -- 172.21.15.57:0/1499022261 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb51000c7e0 con 0x7fb5140ffdc0 2023-10-06T22:12:40.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.408+0000 7fb50affd700 1 -- 172.21.15.57:0/1499022261 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510003c70 con 0x7fb5140ffdc0 2023-10-06T22:12:40.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.408+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5141a0fe0 con 0x7fb5140ffdc0 2023-10-06T22:12:40.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.408+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5141a1400 con 0x7fb5140ffdc0 2023-10-06T22:12:40.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.409+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51410b630 con 0x7fb5140ffdc0 2023-10-06T22:12:40.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.411+0000 7fb50affd700 1 -- 172.21.15.57:0/1499022261 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb51000cc50 con 0x7fb5140ffdc0 2023-10-06T22:12:40.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.412+0000 7fb50affd700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb5000a1550 0x7fb5000a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.413+0000 7fb50affd700 1 -- 172.21.15.57:0/1499022261 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb510014070 con 0x7fb5140ffdc0 2023-10-06T22:12:40.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.413+0000 7fb51979a700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb5000a1550 0x7fb5000a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.413+0000 7fb51979a700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb5000a1550 0x7fb5000a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb504011490 tx=0x7fb504009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:40.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.415+0000 7fb50affd700 1 -- 172.21.15.57:0/1499022261 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb510096860 con 0x7fb5140ffdc0 2023-10-06T22:12:40.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.643+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb51410b870 con 0x7fb5140ffdc0 2023-10-06T22:12:40.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.644+0000 7fb50affd700 1 -- 172.21.15.57:0/1499022261 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb510003dd0 con 0x7fb5140ffdc0 2023-10-06T22:12:40.645 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:40.646 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-06T22:12:40.646 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:12:40.646 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-10-06T22:12:40.646 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-10-06T22:12:40.646 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:12:40.646 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-06T22:12:40.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb5000a1550 msgr2=0x7fb5000a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb5000a1550 0x7fb5000a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb504011490 tx=0x7fb504009260 comp rx=0 tx=0).stop 2023-10-06T22:12:40.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5140ffdc0 msgr2=0x7fb514103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5140ffdc0 0x7fb514103d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e970 tx=0x7fb51000ec80 comp rx=0 tx=0).stop 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 shutdown_connections 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb5000a1550 0x7fb5000a3a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb514101950 0x7fb5141028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5140ffdc0 0x7fb514103d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 --2- 172.21.15.57:0/1499022261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb514100fb0 0x7fb514102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 >> 172.21.15.57:0/1499022261 conn(0x7fb5140fb310 msgr2=0x7fb514068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 shutdown_connections 2023-10-06T22:12:40.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.647+0000 7fb51c1ff700 1 -- 172.21.15.57:0/1499022261 wait complete. 2023-10-06T22:12:40.668 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T22:12:40.668 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T22:12:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 -- 172.21.15.57:0/4063680765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 msgr2=0x7f4518109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 --2- 172.21.15.57:0/4063680765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 0x7f4518109350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f90 tx=0x7f451000b330 comp rx=0 tx=0).stop 2023-10-06T22:12:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 -- 172.21.15.57:0/4063680765 shutdown_connections 2023-10-06T22:12:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 --2- 172.21.15.57:0/4063680765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 0x7f4518109350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 --2- 172.21.15.57:0/4063680765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518100e90 0x7f45181012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 --2- 172.21.15.57:0/4063680765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518102070 0x7f45181024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 -- 172.21.15.57:0/4063680765 >> 172.21.15.57:0/4063680765 conn(0x7f45180fc430 msgr2=0x7f45180fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:40.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 -- 172.21.15.57:0/4063680765 shutdown_connections 2023-10-06T22:12:40.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.883+0000 7f451f5ff700 1 -- 172.21.15.57:0/4063680765 wait complete. 2023-10-06T22:12:40.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.884+0000 7f451f5ff700 1 Processor -- start 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451f5ff700 1 -- start start 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451f5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518100e90 0x7f4518196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451f5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518102070 0x7f4518197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451f5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 0x7f451819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451f5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819e870 con 0x7f4518102070 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451f5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819e9e0 con 0x7f4518102a30 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451f5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819eba0 con 0x7f4518100e90 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451d39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518100e90 0x7f4518196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451d39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518100e90 0x7f4518196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52804/0 (socket says 172.21.15.57:52804) 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451d39b700 1 -- 172.21.15.57:0/1387057432 learned_addr learned my addr 172.21.15.57:0/1387057432 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451cb9a700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518102070 0x7f4518197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451db9c700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 0x7f451819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451d39b700 1 -- 172.21.15.57:0/1387057432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 msgr2=0x7f451819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451d39b700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 0x7f451819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.885+0000 7f451d39b700 1 -- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518102070 msgr2=0x7f4518197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f451d39b700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518102070 0x7f4518197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f451d39b700 1 -- 172.21.15.57:0/1387057432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140095f0 con 0x7f4518100e90 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f451d39b700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518100e90 0x7f4518196d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f451400ebf0 tx=0x7f451400ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f450e7fc700 1 -- 172.21.15.57:0/1387057432 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514009a10 con 0x7f4518100e90 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f450e7fc700 1 -- 172.21.15.57:0/1387057432 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451400cc90 con 0x7f4518100e90 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f450e7fc700 1 -- 172.21.15.57:0/1387057432 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514018750 con 0x7f4518100e90 2023-10-06T22:12:40.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4510009c40 con 0x7f4518100e90 2023-10-06T22:12:40.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.886+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451819f1b0 con 0x7f4518100e90 2023-10-06T22:12:40.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.887+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451810c750 con 0x7f4518100e90 2023-10-06T22:12:40.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.889+0000 7f450e7fc700 1 -- 172.21.15.57:0/1387057432 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f45140188b0 con 0x7f4518100e90 2023-10-06T22:12:40.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.890+0000 7f450e7fc700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f45040a1480 0x7f45040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:40.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.891+0000 7f450e7fc700 1 -- 172.21.15.57:0/1387057432 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4514014070 con 0x7f4518100e90 2023-10-06T22:12:40.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.891+0000 7f451cb9a700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f45040a1480 0x7f45040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:40.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.891+0000 7f451cb9a700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f45040a1480 0x7f45040a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4518100bb0 tx=0x7f4508009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:40.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:40.894+0000 7f450e7fc700 1 -- 172.21.15.57:0/1387057432 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4514096910 con 0x7f4518100e90 2023-10-06T22:12:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:40 smithi057 ceph-mon[98492]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:40 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3618449668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:40 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/4133924334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:40 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:40.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:40 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1499022261' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T22:12:41.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:40 smithi156 ceph-mon[134358]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:40 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3618449668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:40 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/4133924334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:40 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:41.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:40 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1499022261' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T22:12:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:40 smithi190 ceph-mon[134891]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:40 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3618449668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:40 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/4133924334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:12:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:40 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:41.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:40 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1499022261' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T22:12:41.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.111+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4518107e00 con 0x7f4518100e90 2023-10-06T22:12:41.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.675+0000 7f450e7fc700 1 -- 172.21.15.57:0/1387057432 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4514099f30 con 0x7f4518100e90 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f45040a1480 msgr2=0x7f45040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f45040a1480 0x7f45040a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4518100bb0 tx=0x7f4508009260 comp rx=0 tx=0).stop 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518100e90 msgr2=0x7f4518196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518100e90 0x7f4518196d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f451400ebf0 tx=0x7f451400ef00 comp rx=0 tx=0).stop 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 shutdown_connections 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f45040a1480 0x7f45040a3940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4518102a30 0x7f451819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4518100e90 0x7f4518196d30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 --2- 172.21.15.57:0/1387057432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518102070 0x7f4518197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 >> 172.21.15.57:0/1387057432 conn(0x7f45180fc430 msgr2=0x7f45180fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:41.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 shutdown_connections 2023-10-06T22:12:41.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.679+0000 7f451f5ff700 1 -- 172.21.15.57:0/1387057432 wait complete. 2023-10-06T22:12:41.682 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-10-06T22:12:41.700 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T22:12:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.919+0000 7f1a597ff700 1 -- 172.21.15.57:0/1763983488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 msgr2=0x7f1a5410b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.919+0000 7f1a597ff700 1 --2- 172.21.15.57:0/1763983488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a5410b550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fc0 tx=0x7f1a4c00b330 comp rx=0 tx=0).stop 2023-10-06T22:12:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.919+0000 7f1a597ff700 1 -- 172.21.15.57:0/1763983488 shutdown_connections 2023-10-06T22:12:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.919+0000 7f1a597ff700 1 --2- 172.21.15.57:0/1763983488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a5410b550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.919+0000 7f1a597ff700 1 --2- 172.21.15.57:0/1763983488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a54103110 0x7f1a54103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.919+0000 7f1a597ff700 1 --2- 172.21.15.57:0/1763983488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a54104300 0x7f1a54104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.919+0000 7f1a597ff700 1 -- 172.21.15.57:0/1763983488 >> 172.21.15.57:0/1763983488 conn(0x7f1a540fe660 msgr2=0x7f1a54100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:41.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.920+0000 7f1a597ff700 1 -- 172.21.15.57:0/1763983488 shutdown_connections 2023-10-06T22:12:41.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.920+0000 7f1a597ff700 1 -- 172.21.15.57:0/1763983488 wait complete. 2023-10-06T22:12:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 Processor -- start 2023-10-06T22:12:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 -- start start 2023-10-06T22:12:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a54103110 0x7f1a54198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a54104300 0x7f1a54199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a541a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541a0a70 con 0x7f1a54103110 2023-10-06T22:12:41.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541a0bb0 con 0x7f1a54104ca0 2023-10-06T22:12:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a597ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541a0d40 con 0x7f1a54104300 2023-10-06T22:12:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a52ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a54103110 0x7f1a54198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a541a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a541a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38940/0 (socket says 172.21.15.57:38940) 2023-10-06T22:12:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.921+0000 7f1a537fe700 1 -- 172.21.15.57:0/2761122570 learned_addr learned my addr 172.21.15.57:0/2761122570 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:41.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a527fc700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a54104300 0x7f1a54199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:41.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:41 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1387057432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T22:12:41.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:41 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T22:12:41.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:41 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:41.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:41 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:41.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:41 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:41.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:41 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a537fe700 1 -- 172.21.15.57:0/2761122570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a54104300 msgr2=0x7f1a54199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a537fe700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a54104300 0x7f1a54199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a537fe700 1 -- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a54103110 msgr2=0x7f1a54198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a537fe700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a54103110 0x7f1a54198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a537fe700 1 -- 172.21.15.57:0/2761122570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44009520 con 0x7f1a54104ca0 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a52ffd700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a54103110 0x7f1a54198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a537fe700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a541a04a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c004c40 tx=0x7f1a4c004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:41.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a3bfff700 1 -- 172.21.15.57:0/2761122570 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00e040 con 0x7f1a54104ca0 2023-10-06T22:12:41.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.922+0000 7f1a3bfff700 1 -- 172.21.15.57:0/2761122570 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4c00bbf0 con 0x7f1a54104ca0 2023-10-06T22:12:41.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.923+0000 7f1a3bfff700 1 -- 172.21.15.57:0/2761122570 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0052a0 con 0x7f1a54104ca0 2023-10-06T22:12:41.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.923+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c009c40 con 0x7f1a54104ca0 2023-10-06T22:12:41.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.923+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a541a1200 con 0x7f1a54104ca0 2023-10-06T22:12:41.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.924+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5410eb10 con 0x7f1a54104ca0 2023-10-06T22:12:41.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.926+0000 7f1a3bfff700 1 -- 172.21.15.57:0/2761122570 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1a4c01d470 con 0x7f1a54104ca0 2023-10-06T22:12:41.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.928+0000 7f1a3bfff700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a3c0a1500 0x7f1a3c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:41.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.928+0000 7f1a3bfff700 1 -- 172.21.15.57:0/2761122570 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f1a4c0cdf90 con 0x7f1a54104ca0 2023-10-06T22:12:41.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.928+0000 7f1a52ffd700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a3c0a1500 0x7f1a3c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:41.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.928+0000 7f1a52ffd700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a3c0a1500 0x7f1a3c0a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a54104160 tx=0x7f1a44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:41.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:41.929+0000 7f1a3bfff700 1 -- 172.21.15.57:0/2761122570 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1a4c0979b0 con 0x7f1a54104ca0 2023-10-06T22:12:42.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:41 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1387057432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T22:12:42.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:41 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T22:12:42.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:41 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:41 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:41 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:41 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:41 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1387057432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T22:12:42.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:41 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T22:12:42.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:41 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:41 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:41 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:41 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:42.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.150+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a541a13f0 con 0x7f1a54104ca0 2023-10-06T22:12:42.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.682+0000 7f1a3bfff700 1 -- 172.21.15.57:0/2761122570 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1a4c09afd0 con 0x7f1a54104ca0 2023-10-06T22:12:42.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.691+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a3c0a1500 msgr2=0x7f1a3c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:42.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.691+0000 7f1a597ff700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a3c0a1500 0x7f1a3c0a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a54104160 tx=0x7f1a44009260 comp rx=0 tx=0).stop 2023-10-06T22:12:42.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.691+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 msgr2=0x7f1a541a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:42.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.691+0000 7f1a597ff700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a541a04a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c004c40 tx=0x7f1a4c004d20 comp rx=0 tx=0).stop 2023-10-06T22:12:42.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.694+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 shutdown_connections 2023-10-06T22:12:42.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.694+0000 7f1a597ff700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1a3c0a1500 0x7f1a3c0a39c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.694+0000 7f1a597ff700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a54104ca0 0x7f1a541a04a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.694+0000 7f1a597ff700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a54103110 0x7f1a54198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.694+0000 7f1a597ff700 1 --2- 172.21.15.57:0/2761122570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a54104300 0x7f1a54199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.694+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 >> 172.21.15.57:0/2761122570 conn(0x7f1a540fe660 msgr2=0x7f1a54100820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:42.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.698+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 shutdown_connections 2023-10-06T22:12:42.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.700+0000 7f1a597ff700 1 -- 172.21.15.57:0/2761122570 wait complete. 2023-10-06T22:12:42.703 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-10-06T22:12:42.723 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T22:12:42.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/3716788150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4102a30 msgr2=0x7fb7e4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/3716788150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4102a30 0x7fb7e4109350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009f90 tx=0x7fb7dc00b330 comp rx=0 tx=0).stop 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/3716788150 shutdown_connections 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/3716788150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4102a30 0x7fb7e4109350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/3716788150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e4100e90 0x7fb7e41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/3716788150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7e4102070 0x7fb7e41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/3716788150 >> 172.21.15.57:0/3716788150 conn(0x7fb7e40fc430 msgr2=0x7fb7e40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/3716788150 shutdown_connections 2023-10-06T22:12:42.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.940+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/3716788150 wait complete. 2023-10-06T22:12:42.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.941+0000 7fb7ea9ff700 1 Processor -- start 2023-10-06T22:12:42.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.941+0000 7fb7ea9ff700 1 -- start start 2023-10-06T22:12:42.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7ea9ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4100e90 0x7fb7e4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7ea9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7e4102070 0x7fb7e40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7ea9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e4102a30 0x7fb7e40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7ea9ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e4073f80 con 0x7fb7e4100e90 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7ea9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e40740f0 con 0x7fb7e4102070 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7ea9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41a0d60 con 0x7fb7e4102a30 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4100e90 0x7fb7e4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7e4102070 0x7fb7e40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4100e90 0x7fb7e4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59148/0 (socket says 172.21.15.57:59148) 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e3fff700 1 -- 172.21.15.57:0/691906847 learned_addr learned my addr 172.21.15.57:0/691906847 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e8f9c700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e4102a30 0x7fb7e40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e37fe700 1 -- 172.21.15.57:0/691906847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e4102a30 msgr2=0x7fb7e40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:42.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e37fe700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e4102a30 0x7fb7e40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e37fe700 1 -- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4100e90 msgr2=0x7fb7e4074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e37fe700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4100e90 0x7fb7e4074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e37fe700 1 -- 172.21.15.57:0/691906847 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc009c40 con 0x7fb7e4102070 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.942+0000 7fb7e3fff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4100e90 0x7fb7e4074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.943+0000 7fb7e37fe700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7e4102070 0x7fb7e40732a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005fd0 tx=0x7fb7d800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.943+0000 7fb7e17fa700 1 -- 172.21.15.57:0/691906847 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8004220 con 0x7fb7e4102070 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.943+0000 7fb7e17fa700 1 -- 172.21.15.57:0/691906847 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d8004380 con 0x7fb7e4102070 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.943+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e41a0fe0 con 0x7fb7e4102070 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.943+0000 7fb7e17fa700 1 -- 172.21.15.57:0/691906847 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8003be0 con 0x7fb7e4102070 2023-10-06T22:12:42.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.943+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e41a14c0 con 0x7fb7e4102070 2023-10-06T22:12:42.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.944+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e410c750 con 0x7fb7e4102070 2023-10-06T22:12:42.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.946+0000 7fb7e17fa700 1 -- 172.21.15.57:0/691906847 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb7d8003d40 con 0x7fb7e4102070 2023-10-06T22:12:42.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.948+0000 7fb7e17fa700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb7cc0a1480 0x7fb7cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:42.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.948+0000 7fb7e17fa700 1 -- 172.21.15.57:0/691906847 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fb7d8014070 con 0x7fb7e4102070 2023-10-06T22:12:42.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.948+0000 7fb7e3fff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb7cc0a1480 0x7fb7cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:42.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.949+0000 7fb7e3fff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb7cc0a1480 0x7fb7cc0a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005950 tx=0x7fb7d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:42.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:42.949+0000 7fb7e17fa700 1 -- 172.21.15.57:0/691906847 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb7d8096640 con 0x7fb7e4102070 2023-10-06T22:12:43.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:42 smithi156 ceph-mon[134358]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:43.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:42 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T22:12:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:42 smithi156 ceph-mon[134358]: osdmap e80: 12 total, 12 up, 12 in 2023-10-06T22:12:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:42 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2761122570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T22:12:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:42 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T22:12:43.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:42 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:43.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:42 smithi190 ceph-mon[134891]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:43.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:42 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T22:12:43.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:42 smithi190 ceph-mon[134891]: osdmap e80: 12 total, 12 up, 12 in 2023-10-06T22:12:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:42 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2761122570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T22:12:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:42 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T22:12:43.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:42 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:42 smithi057 ceph-mon[98492]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:42 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T22:12:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:42 smithi057 ceph-mon[98492]: osdmap e80: 12 total, 12 up, 12 in 2023-10-06T22:12:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:42 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2761122570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T22:12:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:42 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T22:12:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:42 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:43.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.211+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7e4107df0 con 0x7fb7e4102070 2023-10-06T22:12:43.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.716+0000 7fb7e17fa700 1 -- 172.21.15.57:0/691906847 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb7d80173d0 con 0x7fb7e4102070 2023-10-06T22:12:43.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.719+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb7cc0a1480 msgr2=0x7fb7cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:43.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.719+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb7cc0a1480 0x7fb7cc0a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005950 tx=0x7fb7d4009260 comp rx=0 tx=0).stop 2023-10-06T22:12:43.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.719+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7e4102070 msgr2=0x7fb7e40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:43.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.719+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7e4102070 0x7fb7e40732a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005fd0 tx=0x7fb7d800eca0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.720+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 shutdown_connections 2023-10-06T22:12:43.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.720+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb7cc0a1480 0x7fb7cc0a3940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.720+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e4102a30 0x7fb7e40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.720+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7e4100e90 0x7fb7e4074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.720+0000 7fb7ea9ff700 1 --2- 172.21.15.57:0/691906847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7e4102070 0x7fb7e40732a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.720+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 >> 172.21.15.57:0/691906847 conn(0x7fb7e40fc430 msgr2=0x7fb7e40fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:43.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.722+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 shutdown_connections 2023-10-06T22:12:43.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.722+0000 7fb7ea9ff700 1 -- 172.21.15.57:0/691906847 wait complete. 2023-10-06T22:12:43.724 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 2 and data pool 3 2023-10-06T22:12:43.743 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:12:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.960+0000 7fd8145ff700 1 -- 172.21.15.57:0/2297987251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c102a40 msgr2=0x7fd80c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.960+0000 7fd8145ff700 1 --2- 172.21.15.57:0/2297987251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c102a40 0x7fd80c109360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b0a0 tx=0x7fd80400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.960+0000 7fd8145ff700 1 -- 172.21.15.57:0/2297987251 shutdown_connections 2023-10-06T22:12:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.960+0000 7fd8145ff700 1 --2- 172.21.15.57:0/2297987251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c102a40 0x7fd80c109360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.960+0000 7fd8145ff700 1 --2- 172.21.15.57:0/2297987251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd80c100ea0 0x7fd80c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.960+0000 7fd8145ff700 1 --2- 172.21.15.57:0/2297987251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80c102080 0x7fd80c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.960+0000 7fd8145ff700 1 -- 172.21.15.57:0/2297987251 >> 172.21.15.57:0/2297987251 conn(0x7fd80c0fc420 msgr2=0x7fd80c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.961+0000 7fd8145ff700 1 -- 172.21.15.57:0/2297987251 shutdown_connections 2023-10-06T22:12:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.961+0000 7fd8145ff700 1 -- 172.21.15.57:0/2297987251 wait complete. 2023-10-06T22:12:43.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 Processor -- start 2023-10-06T22:12:43.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 -- start start 2023-10-06T22:12:43.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c100ea0 0x7fd80c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:43.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80c102080 0x7fd80c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:43.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd80c102a40 0x7fd80c19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:43.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c19e790 con 0x7fd80c102a40 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c19e900 con 0x7fd80c102080 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd8145ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c19eac0 con 0x7fd80c100ea0 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd81239b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c100ea0 0x7fd80c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd81239b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c100ea0 0x7fd80c196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:54226/0 (socket says 172.21.15.57:54226) 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd81239b700 1 -- 172.21.15.57:0/1461841626 learned_addr learned my addr 172.21.15.57:0/1461841626 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd812b9c700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd80c102a40 0x7fd80c19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.962+0000 7fd811b9a700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80c102080 0x7fd80c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd812b9c700 1 -- 172.21.15.57:0/1461841626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c100ea0 msgr2=0x7fd80c196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd812b9c700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c100ea0 0x7fd80c196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd812b9c700 1 -- 172.21.15.57:0/1461841626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80c102080 msgr2=0x7fd80c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd812b9c700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80c102080 0x7fd80c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd812b9c700 1 -- 172.21.15.57:0/1461841626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd804009d00 con 0x7fd80c102a40 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd81239b700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c100ea0 0x7fd80c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd812b9c700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd80c102a40 0x7fd80c19e1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd804009fd0 tx=0x7fd804007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd8037fe700 1 -- 172.21.15.57:0/1461841626 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80400e040 con 0x7fd80c102a40 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd8037fe700 1 -- 172.21.15.57:0/1461841626 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd804004c50 con 0x7fd80c102a40 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd8037fe700 1 -- 172.21.15.57:0/1461841626 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80401b6c0 con 0x7fd80c102a40 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.963+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c19ed40 con 0x7fd80c102a40 2023-10-06T22:12:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.964+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c19f110 con 0x7fd80c102a40 2023-10-06T22:12:43.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.965+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c108410 con 0x7fd80c102a40 2023-10-06T22:12:43.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.967+0000 7fd8037fe700 1 -- 172.21.15.57:0/1461841626 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd80401d470 con 0x7fd80c102a40 2023-10-06T22:12:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.968+0000 7fd8037fe700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd7f80a1550 0x7fd7f80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.968+0000 7fd8037fe700 1 -- 172.21.15.57:0/1461841626 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd8040ce190 con 0x7fd80c102a40 2023-10-06T22:12:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.968+0000 7fd81239b700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd7f80a1550 0x7fd7f80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.969+0000 7fd81239b700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd7f80a1550 0x7fd7f80a3a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd80c198120 tx=0x7fd808009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:43.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:43.970+0000 7fd8037fe700 1 -- 172.21.15.57:0/1461841626 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd8040979d0 con 0x7fd80c102a40 2023-10-06T22:12:44.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/691906847' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T22:12:44.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:43 smithi156 ceph-mon[134358]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-10-06T22:12:44.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:44.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:44.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T22:12:44.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T22:12:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/691906847' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T22:12:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T22:12:44.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:43 smithi190 ceph-mon[134891]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-10-06T22:12:44.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:44.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:44.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T22:12:44.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T22:12:44.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:44.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:44.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:12:44.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/691906847' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T22:12:44.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T22:12:44.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:43 smithi057 ceph-mon[98492]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-10-06T22:12:44.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.222+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd80c02d290 con 0x7fd80c102a40 2023-10-06T22:12:44.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.224+0000 7fd8037fe700 1 -- 172.21.15.57:0/1461841626 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22122 (secure 0 0 0) 0x7fd80409aff0 con 0x7fd80c102a40 2023-10-06T22:12:44.226 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:44.226 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":82,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:12:43.212537+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:12:44.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd7f80a1550 msgr2=0x7fd7f80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd7f80a1550 0x7fd7f80a3a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd80c198120 tx=0x7fd808009260 comp rx=0 tx=0).stop 2023-10-06T22:12:44.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd80c102a40 msgr2=0x7fd80c19e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd80c102a40 0x7fd80c19e1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd804009fd0 tx=0x7fd804007930 comp rx=0 tx=0).stop 2023-10-06T22:12:44.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 shutdown_connections 2023-10-06T22:12:44.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd7f80a1550 0x7fd7f80a3a10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd80c102a40 0x7fd80c19e1c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80c100ea0 0x7fd80c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 --2- 172.21.15.57:0/1461841626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd80c102080 0x7fd80c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 >> 172.21.15.57:0/1461841626 conn(0x7fd80c0fc420 msgr2=0x7fd80c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:44.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 shutdown_connections 2023-10-06T22:12:44.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.228+0000 7fd8145ff700 1 -- 172.21.15.57:0/1461841626 wait complete. 2023-10-06T22:12:44.253 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:12:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 -- 172.21.15.57:0/2630345507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 msgr2=0x7f5b100ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 --2- 172.21.15.57:0/2630345507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b100ffbb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009990 tx=0x7f5b0c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 -- 172.21.15.57:0/2630345507 shutdown_connections 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 --2- 172.21.15.57:0/2630345507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b10100ab0 0x7f5b10100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 --2- 172.21.15.57:0/2630345507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b100ffbb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 --2- 172.21.15.57:0/2630345507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b101000f0 0x7f5b10100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 -- 172.21.15.57:0/2630345507 >> 172.21.15.57:0/2630345507 conn(0x7f5b100fb3b0 msgr2=0x7f5b100fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 -- 172.21.15.57:0/2630345507 shutdown_connections 2023-10-06T22:12:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.473+0000 7f5b181ff700 1 -- 172.21.15.57:0/2630345507 wait complete. 2023-10-06T22:12:44.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.474+0000 7f5b181ff700 1 Processor -- start 2023-10-06T22:12:44.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b181ff700 1 -- start start 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b181ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b10198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b181ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b101000f0 0x7f5b101992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b181ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b10100ab0 0x7f5b101a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b181ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101a08f0 con 0x7f5b100ff790 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b181ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101a0a60 con 0x7f5b10100ab0 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b181ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101a0c20 con 0x7f5b101000f0 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.475+0000 7f5b15f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b10198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b1679c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b10100ab0 0x7f5b101a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b10198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59184/0 (socket says 172.21.15.57:59184) 2023-10-06T22:12:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 -- 172.21.15.57:0/1551596708 learned_addr learned my addr 172.21.15.57:0/1551596708 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 -- 172.21.15.57:0/1551596708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b101000f0 msgr2=0x7f5b101992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b101000f0 0x7f5b101992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 -- 172.21.15.57:0/1551596708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b10100ab0 msgr2=0x7f5b101a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b10100ab0 0x7f5b101a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 -- 172.21.15.57:0/1551596708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0095f0 con 0x7f5b100ff790 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b1679c700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b10100ab0 0x7f5b101a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.476+0000 7f5b15f9b700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b10198db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c004740 tx=0x7f5b0c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.477+0000 7f5b06ffd700 1 -- 172.21.15.57:0/1551596708 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c01d070 con 0x7f5b100ff790 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.477+0000 7f5b06ffd700 1 -- 172.21.15.57:0/1551596708 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0c004830 con 0x7f5b100ff790 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.477+0000 7f5b06ffd700 1 -- 172.21.15.57:0/1551596708 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c005450 con 0x7f5b100ff790 2023-10-06T22:12:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.477+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b101a0ea0 con 0x7f5b100ff790 2023-10-06T22:12:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.477+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b101a1210 con 0x7f5b100ff790 2023-10-06T22:12:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.478+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1010a570 con 0x7f5b100ff790 2023-10-06T22:12:44.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.480+0000 7f5b06ffd700 1 -- 172.21.15.57:0/1551596708 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5b0c00fb00 con 0x7f5b100ff790 2023-10-06T22:12:44.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.481+0000 7f5b06ffd700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5afc0a14d0 0x7f5afc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.482+0000 7f5b06ffd700 1 -- 172.21.15.57:0/1551596708 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5b0c0cde70 con 0x7f5b100ff790 2023-10-06T22:12:44.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.482+0000 7f5b1579a700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5afc0a14d0 0x7f5afc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:44.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.482+0000 7f5b1579a700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5afc0a14d0 0x7f5afc0a3990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b00005950 tx=0x7f5b000058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:44.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.486+0000 7f5b06ffd700 1 -- 172.21.15.57:0/1551596708 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5b0c05b100 con 0x7f5b100ff790 2023-10-06T22:12:44.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.726+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b1002d1e0 con 0x7f5b100ff790 2023-10-06T22:12:44.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.727+0000 7f5b06ffd700 1 -- 172.21.15.57:0/1551596708 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22109 (secure 0 0 0) 0x7f5b0c097610 con 0x7f5b100ff790 2023-10-06T22:12:44.729 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:44.729 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":83,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:12:44.676689+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5afc0a14d0 msgr2=0x7f5afc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5afc0a14d0 0x7f5afc0a3990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b00005950 tx=0x7f5b000058e0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 msgr2=0x7f5b10198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b10198db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c004740 tx=0x7f5b0c009580 comp rx=0 tx=0).stop 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 shutdown_connections 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5afc0a14d0 0x7f5afc0a3990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b10100ab0 0x7f5b101a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b100ff790 0x7f5b10198db0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 --2- 172.21.15.57:0/1551596708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b101000f0 0x7f5b101992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 >> 172.21.15.57:0/1551596708 conn(0x7f5b100fb3b0 msgr2=0x7f5b100fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:44.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 shutdown_connections 2023-10-06T22:12:44.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.730+0000 7f5b181ff700 1 -- 172.21.15.57:0/1551596708 wait complete. 2023-10-06T22:12:44.752 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 -- 172.21.15.57:0/1386335911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 msgr2=0x7fbf98073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/1386335911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98073340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009990 tx=0x7fbf94009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 -- 172.21.15.57:0/1386335911 shutdown_connections 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/1386335911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf98108da0 0x7fbf9810b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/1386335911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98073340 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/1386335911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf98073910 0x7fbf98073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 -- 172.21.15.57:0/1386335911 >> 172.21.15.57:0/1386335911 conn(0x7fbf980fc430 msgr2=0x7fbf980fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:44.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 -- 172.21.15.57:0/1386335911 shutdown_connections 2023-10-06T22:12:44.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.970+0000 7fbfa07ff700 1 -- 172.21.15.57:0/1386335911 wait complete. 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.971+0000 7fbfa07ff700 1 Processor -- start 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbfa07ff700 1 -- start start 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbfa07ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf98073910 0x7fbf98198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbfa07ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbfa07ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf98108da0 0x7fbf981a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbfa07ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981a0970 con 0x7fbf98074ef0 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbfa07ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981a0ab0 con 0x7fbf98073910 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbfa07ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981a0c40 con 0x7fbf98108da0 2023-10-06T22:12:44.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbf9dd9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbf9dd9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59204/0 (socket says 172.21.15.57:59204) 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbf9dd9a700 1 -- 172.21.15.57:0/264624529 learned_addr learned my addr 172.21.15.57:0/264624529 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbf9e59b700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf98073910 0x7fbf98198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.972+0000 7fbf9ed9c700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf98108da0 0x7fbf981a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf9dd9a700 1 -- 172.21.15.57:0/264624529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf98108da0 msgr2=0x7fbf981a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf9dd9a700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf98108da0 0x7fbf981a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf9dd9a700 1 -- 172.21.15.57:0/264624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf98073910 msgr2=0x7fbf98198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf9dd9a700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf98073910 0x7fbf98198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf9dd9a700 1 -- 172.21.15.57:0/264624529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940095f0 con 0x7fbf98074ef0 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf9e59b700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf98073910 0x7fbf98198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:44.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf9dd9a700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98199330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800d980 tx=0x7fbf8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:44.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf8f7fe700 1 -- 172.21.15.57:0/264624529 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800b950 con 0x7fbf98074ef0 2023-10-06T22:12:44.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf8f7fe700 1 -- 172.21.15.57:0/264624529 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf88007620 con 0x7fbf98074ef0 2023-10-06T22:12:44.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf981a0f20 con 0x7fbf98074ef0 2023-10-06T22:12:44.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbf8f7fe700 1 -- 172.21.15.57:0/264624529 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800f3c0 con 0x7fbf98074ef0 2023-10-06T22:12:44.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.973+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf981a13a0 con 0x7fbf98074ef0 2023-10-06T22:12:44.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.974+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf98107450 con 0x7fbf98074ef0 2023-10-06T22:12:44.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.977+0000 7fbf8f7fe700 1 -- 172.21.15.57:0/264624529 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbf88010640 con 0x7fbf98074ef0 2023-10-06T22:12:44.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.978+0000 7fbf8f7fe700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbf840a1480 0x7fbf840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:44.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.978+0000 7fbf8f7fe700 1 -- 172.21.15.57:0/264624529 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbf880cd350 con 0x7fbf98074ef0 2023-10-06T22:12:44.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.978+0000 7fbf9e59b700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbf840a1480 0x7fbf840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:44.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.979+0000 7fbf9e59b700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbf840a1480 0x7fbf840a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b2e0 tx=0x7fbf94005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:44.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:44.980+0000 7fbf8f7fe700 1 -- 172.21.15.57:0/264624529 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf88096a60 con 0x7fbf98074ef0 2023-10-06T22:12:45.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:44 smithi156 ceph-mon[134358]: pgmap v270: 97 pgs: 15 creating+peering, 7 active+clean, 75 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:44 smithi156 ceph-mon[134358]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T22:12:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:44 smithi156 ceph-mon[134358]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T22:12:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:44 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T22:12:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:44 smithi156 ceph-mon[134358]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-10-06T22:12:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:44 smithi156 ceph-mon[134358]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-10-06T22:12:45.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:44 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1461841626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:45.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:44 smithi190 ceph-mon[134891]: pgmap v270: 97 pgs: 15 creating+peering, 7 active+clean, 75 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:45.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:44 smithi190 ceph-mon[134891]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T22:12:45.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:44 smithi190 ceph-mon[134891]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T22:12:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:44 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T22:12:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:44 smithi190 ceph-mon[134891]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-10-06T22:12:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:44 smithi190 ceph-mon[134891]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-10-06T22:12:45.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:44 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1461841626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:44 smithi057 ceph-mon[98492]: pgmap v270: 97 pgs: 15 creating+peering, 7 active+clean, 75 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:12:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:44 smithi057 ceph-mon[98492]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T22:12:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:44 smithi057 ceph-mon[98492]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T22:12:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:44 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T22:12:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:44 smithi057 ceph-mon[98492]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-10-06T22:12:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:44 smithi057 ceph-mon[98492]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-10-06T22:12:45.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:44 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1461841626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:45.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.244+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbf9802d270 con 0x7fbf98074ef0 2023-10-06T22:12:45.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.724+0000 7fbf8f7fe700 1 -- 172.21.15.57:0/264624529 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbf8809a080 con 0x7fbf98074ef0 2023-10-06T22:12:45.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbf840a1480 msgr2=0x7fbf840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbf840a1480 0x7fbf840a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b2e0 tx=0x7fbf94005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 msgr2=0x7fbf98199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98199330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800d980 tx=0x7fbf8800dc90 comp rx=0 tx=0).stop 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 shutdown_connections 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fbf840a1480 0x7fbf840a3940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf98108da0 0x7fbf981a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf98073910 0x7fbf98198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 --2- 172.21.15.57:0/264624529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf98074ef0 0x7fbf98199330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 >> 172.21.15.57:0/264624529 conn(0x7fbf980fc430 msgr2=0x7fbf980fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:45.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 shutdown_connections 2023-10-06T22:12:45.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.728+0000 7fbfa07ff700 1 -- 172.21.15.57:0/264624529 wait complete. 2023-10-06T22:12:45.749 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-06T22:12:45.749 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-06T22:12:45.819 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: 2023-10-06T22:12:45.440+0000 7eff0ff7e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T22:12:45.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-mon[134891]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T22:12:45.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] up:active 2023-10-06T22:12:45.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-mon[134891]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:45.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-mon[134891]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:45.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1551596708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:45.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/264624529' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T22:12:45.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:45 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:45.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.961+0000 7fdb58fff700 1 -- 172.21.15.57:0/1029132823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb541042b0 msgr2=0x7fdb541066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:45.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.961+0000 7fdb58fff700 1 --2- 172.21.15.57:0/1029132823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb541042b0 0x7fdb541066a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009990 tx=0x7fdb48009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.962+0000 7fdb58fff700 1 -- 172.21.15.57:0/1029132823 shutdown_connections 2023-10-06T22:12:45.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.962+0000 7fdb58fff700 1 --2- 172.21.15.57:0/1029132823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb54106be0 0x7fdb54109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.962+0000 7fdb58fff700 1 --2- 172.21.15.57:0/1029132823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb54101980 0x7fdb54103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.962+0000 7fdb58fff700 1 --2- 172.21.15.57:0/1029132823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb541042b0 0x7fdb541066a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.962+0000 7fdb58fff700 1 -- 172.21.15.57:0/1029132823 >> 172.21.15.57:0/1029132823 conn(0x7fdb540fb370 msgr2=0x7fdb540fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:45.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.962+0000 7fdb58fff700 1 -- 172.21.15.57:0/1029132823 shutdown_connections 2023-10-06T22:12:45.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.962+0000 7fdb58fff700 1 -- 172.21.15.57:0/1029132823 wait complete. 2023-10-06T22:12:45.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 Processor -- start 2023-10-06T22:12:45.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 -- start start 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb54101980 0x7fdb54198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb541042b0 0x7fdb54199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb54106be0 0x7fdb541a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541a0900 con 0x7fdb541042b0 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541a0a70 con 0x7fdb54106be0 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb58fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541a0c30 con 0x7fdb54101980 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb51d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb541042b0 0x7fdb54199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb51d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb541042b0 0x7fdb54199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59218/0 (socket says 172.21.15.57:59218) 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb51d9b700 1 -- 172.21.15.57:0/3850842903 learned_addr learned my addr 172.21.15.57:0/3850842903 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:45.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb52d9d700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb54106be0 0x7fdb541a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.963+0000 7fdb5259c700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb54101980 0x7fdb54198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb52d9d700 1 -- 172.21.15.57:0/3850842903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb54101980 msgr2=0x7fdb54198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb52d9d700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb54101980 0x7fdb54198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb52d9d700 1 -- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb541042b0 msgr2=0x7fdb54199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb52d9d700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb541042b0 0x7fdb54199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb52d9d700 1 -- 172.21.15.57:0/3850842903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480095f0 con 0x7fdb54106be0 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb51d9b700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb541042b0 0x7fdb54199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb52d9d700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb54106be0 0x7fdb541a0330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00efc0 tx=0x7fdb4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb437fe700 1 -- 172.21.15.57:0/3850842903 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c018070 con 0x7fdb54106be0 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb437fe700 1 -- 172.21.15.57:0/3850842903 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4c00f040 con 0x7fdb54106be0 2023-10-06T22:12:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb437fe700 1 -- 172.21.15.57:0/3850842903 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00cd40 con 0x7fdb54106be0 2023-10-06T22:12:45.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb541a0eb0 con 0x7fdb54106be0 2023-10-06T22:12:45.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.964+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb541a1280 con 0x7fdb54106be0 2023-10-06T22:12:45.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.965+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5410e960 con 0x7fdb54106be0 2023-10-06T22:12:45.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.968+0000 7fdb437fe700 1 -- 172.21.15.57:0/3850842903 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdb4c003a00 con 0x7fdb54106be0 2023-10-06T22:12:45.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.971+0000 7fdb437fe700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:45.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.971+0000 7fdb437fe700 1 -- 172.21.15.57:0/3850842903 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdb4c0cc870 con 0x7fdb54106be0 2023-10-06T22:12:45.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.971+0000 7fdb5259c700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:45.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.971+0000 7fdb437fe700 1 -- 172.21.15.57:0/3850842903 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb4c096010 con 0x7fdb54106be0 2023-10-06T22:12:45.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:45.971+0000 7fdb5259c700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdb5419a290 tx=0x7fdb44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:46.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:45 smithi156 ceph-mon[134358]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T22:12:46.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:45 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] up:active 2023-10-06T22:12:46.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:45 smithi156 ceph-mon[134358]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:46.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:45 smithi156 ceph-mon[134358]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:46.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:45 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1551596708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:46.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:45 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/264624529' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T22:12:46.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:45 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:45 smithi057 ceph-mon[98492]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T22:12:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:45 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] up:active 2023-10-06T22:12:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:45 smithi057 ceph-mon[98492]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:45 smithi057 ceph-mon[98492]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:45 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1551596708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:45 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/264624529' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T22:12:46.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:45 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:46.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.245+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fdb541a1470 con 0x7fdb54106be0 2023-10-06T22:12:46.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.729+0000 7fdb437fe700 1 -- 172.21.15.57:0/3850842903 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fdb4c099630 con 0x7fdb54106be0 2023-10-06T22:12:46.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdb3c0a1500 msgr2=0x7fdb3c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:46.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdb5419a290 tx=0x7fdb44009260 comp rx=0 tx=0).stop 2023-10-06T22:12:46.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb54106be0 msgr2=0x7fdb541a0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:46.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb54106be0 0x7fdb541a0330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00efc0 tx=0x7fdb4c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 shutdown_connections 2023-10-06T22:12:46.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb54106be0 0x7fdb541a0330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb54101980 0x7fdb54198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 --2- 172.21.15.57:0/3850842903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb541042b0 0x7fdb54199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.732+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 >> 172.21.15.57:0/3850842903 conn(0x7fdb540fb370 msgr2=0x7fdb54102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:46.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.733+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 shutdown_connections 2023-10-06T22:12:46.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.733+0000 7fdb58fff700 1 -- 172.21.15.57:0/3850842903 wait complete. 2023-10-06T22:12:46.755 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-06T22:12:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 -- 172.21.15.57:0/2424561060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 msgr2=0x7f23681012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 --2- 172.21.15.57:0/2424561060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 0x7f23681012b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2364009990 tx=0x7f2364009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 -- 172.21.15.57:0/2424561060 shutdown_connections 2023-10-06T22:12:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 --2- 172.21.15.57:0/2424561060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2368102a30 0x7f2368109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 --2- 172.21.15.57:0/2424561060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 0x7f23681012b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 --2- 172.21.15.57:0/2424561060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 0x7f23681024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 -- 172.21.15.57:0/2424561060 >> 172.21.15.57:0/2424561060 conn(0x7f23680fc430 msgr2=0x7f23680fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 -- 172.21.15.57:0/2424561060 shutdown_connections 2023-10-06T22:12:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.961+0000 7f23701ff700 1 -- 172.21.15.57:0/2424561060 wait complete. 2023-10-06T22:12:46.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.962+0000 7f23701ff700 1 Processor -- start 2023-10-06T22:12:46.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.962+0000 7f23701ff700 1 -- start start 2023-10-06T22:12:46.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f23701ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 0x7f2368196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f23701ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 0x7f2368197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f23701ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2368102a30 0x7f236819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f23701ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819e770 con 0x7f2368102070 2023-10-06T22:12:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f23701ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819e8e0 con 0x7f2368100e90 2023-10-06T22:12:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f23701ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819eaa0 con 0x7f2368102a30 2023-10-06T22:12:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 0x7f2368197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 0x7f2368197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59230/0 (socket says 172.21.15.57:59230) 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 -- 172.21.15.57:0/1408782181 learned_addr learned my addr 172.21.15.57:0/1408782181 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236df9b700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 0x7f2368196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236e79c700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2368102a30 0x7f236819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 -- 172.21.15.57:0/1408782181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2368102a30 msgr2=0x7f236819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2368102a30 0x7f236819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 -- 172.21.15.57:0/1408782181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 msgr2=0x7f2368196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 0x7f2368196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236d79a700 1 -- 172.21.15.57:0/1408782181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009520 con 0x7f2368102070 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.963+0000 7f236e79c700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2368102a30 0x7f236819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.964+0000 7f236d79a700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 0x7f2368197170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f235800ea50 tx=0x7f235800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.964+0000 7f235effd700 1 -- 172.21.15.57:0/1408782181 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800cbd0 con 0x7f2368102070 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.964+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23640095f0 con 0x7f2368102070 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.964+0000 7f235effd700 1 -- 172.21.15.57:0/1408782181 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f235800cd30 con 0x7f2368102070 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.964+0000 7f235effd700 1 -- 172.21.15.57:0/1408782181 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358018680 con 0x7f2368102070 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.964+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236819f150 con 0x7f2368102070 2023-10-06T22:12:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.965+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368107e60 con 0x7f2368102070 2023-10-06T22:12:46.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.967+0000 7f235effd700 1 -- 172.21.15.57:0/1408782181 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f23580188a0 con 0x7f2368102070 2023-10-06T22:12:46.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.968+0000 7f235effd700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23540a1500 0x7f23540a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.968+0000 7f235effd700 1 -- 172.21.15.57:0/1408782181 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2358014070 con 0x7f2368102070 2023-10-06T22:12:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.969+0000 7f236df9b700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23540a1500 0x7f23540a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:46.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.969+0000 7f236df9b700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23540a1500 0x7f23540a39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2368198100 tx=0x7f2364005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:46.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:46.972+0000 7f235effd700 1 -- 172.21.15.57:0/1408782181 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2358096820 con 0x7f2368102070 2023-10-06T22:12:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: pgmap v273: 97 pgs: 4 creating+activating, 15 creating+peering, 41 active+clean, 37 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2023-10-06T22:12:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-06T22:12:47.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: Cluster is now healthy 2023-10-06T22:12:47.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/264624529' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T22:12:47.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:47.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3850842903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T22:12:47.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T22:12:47.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:47.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:47.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:46 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: pgmap v273: 97 pgs: 4 creating+activating, 15 creating+peering, 41 active+clean, 37 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2023-10-06T22:12:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: Cluster is now healthy 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/264624529' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3850842903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:47.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:46 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: pgmap v273: 97 pgs: 4 creating+activating, 15 creating+peering, 41 active+clean, 37 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: Cluster is now healthy 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/264624529' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T22:12:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:47.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3850842903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T22:12:47.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-06T22:12:47.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:12:47.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:12:47.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:46 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:12:47.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.231+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f236810c700 con 0x7f2368102070 2023-10-06T22:12:47.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.735+0000 7f235effd700 1 -- 172.21.15.57:0/1408782181 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f2358099e40 con 0x7f2368102070 2023-10-06T22:12:47.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.738+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23540a1500 msgr2=0x7f23540a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:47.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.738+0000 7f23701ff700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23540a1500 0x7f23540a39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2368198100 tx=0x7f2364005fd0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.738+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 msgr2=0x7f2368197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:47.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.738+0000 7f23701ff700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 0x7f2368197170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f235800ea50 tx=0x7f235800ed60 comp rx=0 tx=0).stop 2023-10-06T22:12:47.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 shutdown_connections 2023-10-06T22:12:47.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f23540a1500 0x7f23540a39c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2368102a30 0x7f236819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2368100e90 0x7f2368196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 --2- 172.21.15.57:0/1408782181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2368102070 0x7f2368197170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 >> 172.21.15.57:0/1408782181 conn(0x7f23680fc430 msgr2=0x7f23680fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:47.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 shutdown_connections 2023-10-06T22:12:47.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.739+0000 7f23701ff700 1 -- 172.21.15.57:0/1408782181 wait complete. 2023-10-06T22:12:47.767 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-06T22:12:47.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/3162454834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4100e90 msgr2=0x7fc9b41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/3162454834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4100e90 0x7fc9b41012b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00098f0 tx=0x7fc9b0009c00 comp rx=0 tx=0).stop 2023-10-06T22:12:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/3162454834 shutdown_connections 2023-10-06T22:12:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/3162454834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b4102a30 0x7fc9b4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/3162454834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4100e90 0x7fc9b41012b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/3162454834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4102070 0x7fc9b41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/3162454834 >> 172.21.15.57:0/3162454834 conn(0x7fc9b40fc430 msgr2=0x7fc9b40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:47.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/3162454834 shutdown_connections 2023-10-06T22:12:47.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.975+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/3162454834 wait complete. 2023-10-06T22:12:47.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.976+0000 7fc9bc1ff700 1 Processor -- start 2023-10-06T22:12:47.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.976+0000 7fc9bc1ff700 1 -- start start 2023-10-06T22:12:47.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.976+0000 7fc9bc1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4100e90 0x7fc9b4196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:47.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.976+0000 7fc9bc1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b4102070 0x7fc9b41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:47.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9bc1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4102a30 0x7fc9b419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:47.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9bc1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b419e7c0 con 0x7fc9b4100e90 2023-10-06T22:12:47.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9bc1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b419e930 con 0x7fc9b4102070 2023-10-06T22:12:47.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9bc1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b419eaf0 con 0x7fc9b4102a30 2023-10-06T22:12:47.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4100e90 0x7fc9b4196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:47.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4100e90 0x7fc9b4196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59240/0 (socket says 172.21.15.57:59240) 2023-10-06T22:12:47.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 -- 172.21.15.57:0/49967289 learned_addr learned my addr 172.21.15.57:0/49967289 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:47.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b979a700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b4102070 0x7fc9b41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:47.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9ba79c700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4102a30 0x7fc9b419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:47.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 -- 172.21.15.57:0/49967289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4102a30 msgr2=0x7fc9b419e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4102a30 0x7fc9b419e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 -- 172.21.15.57:0/49967289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b4102070 msgr2=0x7fc9b41971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b4102070 0x7fc9b41971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 -- 172.21.15.57:0/49967289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0009520 con 0x7fc9b4100e90 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.977+0000 7fc9b9f9b700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4100e90 0x7fc9b4196c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000c00 tx=0x7fc9b000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.978+0000 7fc9aaffd700 1 -- 172.21.15.57:0/49967289 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b001c070 con 0x7fc9b4100e90 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.978+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b419ed70 con 0x7fc9b4100e90 2023-10-06T22:12:47.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.978+0000 7fc9aaffd700 1 -- 172.21.15.57:0/49967289 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9b000f7d0 con 0x7fc9b4100e90 2023-10-06T22:12:47.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.978+0000 7fc9aaffd700 1 -- 172.21.15.57:0/49967289 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00039b0 con 0x7fc9b4100e90 2023-10-06T22:12:47.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.978+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b419f1a0 con 0x7fc9b4100e90 2023-10-06T22:12:47.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.979+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b410c700 con 0x7fc9b4100e90 2023-10-06T22:12:47.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.981+0000 7fc9aaffd700 1 -- 172.21.15.57:0/49967289 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc9b0020030 con 0x7fc9b4100e90 2023-10-06T22:12:47.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.982+0000 7fc9aaffd700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc9a00a1480 0x7fc9a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:47.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.983+0000 7fc9aaffd700 1 -- 172.21.15.57:0/49967289 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9b00ce0a0 con 0x7fc9b4100e90 2023-10-06T22:12:47.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.983+0000 7fc9b979a700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc9a00a1480 0x7fc9a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:47.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.983+0000 7fc9b979a700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc9a00a1480 0x7fc9a00a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4100bb0 tx=0x7fc9a4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:47.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:47.984+0000 7fc9aaffd700 1 -- 172.21.15.57:0/49967289 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc9b0097840 con 0x7fc9b4100e90 2023-10-06T22:12:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:47 smithi156 ceph-mon[134358]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T22:12:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:47 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-06T22:12:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:47 smithi156 ceph-mon[134358]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T22:12:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:47 smithi156 ceph-mon[134358]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:47 smithi156 ceph-mon[134358]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-10-06T22:12:48.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:47 smithi156 ceph-mon[134358]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-10-06T22:12:48.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:47 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1408782181' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T22:12:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:47 smithi190 ceph-mon[134891]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T22:12:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:47 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-06T22:12:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:47 smithi190 ceph-mon[134891]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T22:12:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:47 smithi190 ceph-mon[134891]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:47 smithi190 ceph-mon[134891]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-10-06T22:12:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:47 smithi190 ceph-mon[134891]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-10-06T22:12:48.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1408782181' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T22:12:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:47 smithi057 ceph-mon[98492]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T22:12:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:47 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-06T22:12:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:47 smithi057 ceph-mon[98492]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T22:12:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:47 smithi057 ceph-mon[98492]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-10-06T22:12:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:47 smithi057 ceph-mon[98492]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-10-06T22:12:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:47 smithi057 ceph-mon[98492]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-10-06T22:12:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:47 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1408782181' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T22:12:48.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.244+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc9b4107f10 con 0x7fc9b4100e90 2023-10-06T22:12:48.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.747+0000 7fc9aaffd700 1 -- 172.21.15.57:0/49967289 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc9b009ae60 con 0x7fc9b4100e90 2023-10-06T22:12:48.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc9a00a1480 msgr2=0x7fc9a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc9a00a1480 0x7fc9a00a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4100bb0 tx=0x7fc9a4009310 comp rx=0 tx=0).stop 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4100e90 msgr2=0x7fc9b4196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4100e90 0x7fc9b4196c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000c00 tx=0x7fc9b000b790 comp rx=0 tx=0).stop 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 shutdown_connections 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc9a00a1480 0x7fc9a00a3940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4102a30 0x7fc9b419e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9b4100e90 0x7fc9b4196c80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 --2- 172.21.15.57:0/49967289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b4102070 0x7fc9b41971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 >> 172.21.15.57:0/49967289 conn(0x7fc9b40fc430 msgr2=0x7fc9b40fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 shutdown_connections 2023-10-06T22:12:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.750+0000 7fc9bc1ff700 1 -- 172.21.15.57:0/49967289 wait complete. 2023-10-06T22:12:48.779 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-10-06T22:12:48.780 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-06T22:12:48.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.989+0000 7f769abff700 1 -- 172.21.15.57:0/1485660381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694074ef0 msgr2=0x7f7694073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:48.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.989+0000 7f769abff700 1 --2- 172.21.15.57:0/1485660381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694074ef0 0x7f7694073340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7688009990 tx=0x7f7688009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.989+0000 7f769abff700 1 -- 172.21.15.57:0/1485660381 shutdown_connections 2023-10-06T22:12:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.989+0000 7f769abff700 1 --2- 172.21.15.57:0/1485660381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7694108da0 0x7f769410b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.989+0000 7f769abff700 1 --2- 172.21.15.57:0/1485660381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694074ef0 0x7f7694073340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.989+0000 7f769abff700 1 --2- 172.21.15.57:0/1485660381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7694073910 0x7f7694073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.989+0000 7f769abff700 1 -- 172.21.15.57:0/1485660381 >> 172.21.15.57:0/1485660381 conn(0x7f76940fc430 msgr2=0x7f76940fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.990+0000 7f769abff700 1 -- 172.21.15.57:0/1485660381 shutdown_connections 2023-10-06T22:12:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.990+0000 7f769abff700 1 -- 172.21.15.57:0/1485660381 wait complete. 2023-10-06T22:12:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 Processor -- start 2023-10-06T22:12:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 -- start start 2023-10-06T22:12:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7694073910 0x7f7694198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7694074ef0 0x7f7694199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694108da0 0x7f76941a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941a09a0 con 0x7f7694108da0 2023-10-06T22:12:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941a0ae0 con 0x7f7694074ef0 2023-10-06T22:12:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941a0c70 con 0x7f7694073910 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769919c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694108da0 0x7f76941a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769919c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694108da0 0x7f76941a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59248/0 (socket says 172.21.15.57:59248) 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769919c700 1 -- 172.21.15.57:0/2747547081 learned_addr learned my addr 172.21.15.57:0/2747547081 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.991+0000 7f769899b700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7694073910 0x7f7694198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f7693fff700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7694074ef0 0x7f7694199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769919c700 1 -- 172.21.15.57:0/2747547081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7694073910 msgr2=0x7f7694198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769919c700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7694073910 0x7f7694198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769919c700 1 -- 172.21.15.57:0/2747547081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7694074ef0 msgr2=0x7f7694199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769919c700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7694074ef0 0x7f7694199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769919c700 1 -- 172.21.15.57:0/2747547081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880095f0 con 0x7f7694108da0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769919c700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694108da0 0x7f76941a03d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f768c007ee0 tx=0x7f768c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f7691ffb700 1 -- 172.21.15.57:0/2747547081 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c017070 con 0x7f7694108da0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f7691ffb700 1 -- 172.21.15.57:0/2747547081 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f768c00f040 con 0x7f7694108da0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76941a0f50 con 0x7f7694108da0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f7691ffb700 1 -- 172.21.15.57:0/2747547081 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c005740 con 0x7f7694108da0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.992+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76941a13d0 con 0x7f7694108da0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.993+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7694107450 con 0x7f7694108da0 2023-10-06T22:12:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.996+0000 7f7691ffb700 1 -- 172.21.15.57:0/2747547081 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f768c00dc60 con 0x7f7694108da0 2023-10-06T22:12:48.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.997+0000 7f7691ffb700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76840a1500 0x7f76840a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:48.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.997+0000 7f7691ffb700 1 -- 172.21.15.57:0/2747547081 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f768c0cca20 con 0x7f7694108da0 2023-10-06T22:12:48.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.997+0000 7f769899b700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76840a1500 0x7f76840a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:49.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.998+0000 7f769899b700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76840a1500 0x7f76840a39c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f768800b2e0 tx=0x7f7688005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:49.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:48.999+0000 7f7691ffb700 1 -- 172.21.15.57:0/2747547081 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f768c096130 con 0x7f7694108da0 2023-10-06T22:12:49.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: pgmap v274: 97 pgs: 4 creating+activating, 10 creating+peering, 71 active+clean, 12 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-10-06T22:12:49.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1408782181' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T22:12:49.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] up:active 2023-10-06T22:12:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T22:12:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-10-06T22:12:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-10-06T22:12:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-06T22:12:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:48 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/49967289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T22:12:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: pgmap v274: 97 pgs: 4 creating+activating, 10 creating+peering, 71 active+clean, 12 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-10-06T22:12:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1408782181' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T22:12:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] up:active 2023-10-06T22:12:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T22:12:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-10-06T22:12:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-10-06T22:12:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-06T22:12:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:48 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/49967289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T22:12:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: pgmap v274: 97 pgs: 4 creating+activating, 10 creating+peering, 71 active+clean, 12 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-10-06T22:12:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1408782181' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T22:12:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] up:active 2023-10-06T22:12:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T22:12:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-10-06T22:12:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-10-06T22:12:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-06T22:12:49.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:48 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/49967289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T22:12:49.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.241+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f769402d2e0 con 0x7f76840a1500 2023-10-06T22:12:49.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:12:49.248+0000 7fbcc5c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:12:49.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:12:49.248+0000 7fbcc5c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:12:49.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:12:49.248+0000 7fbcc5c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:12:49.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:12:49.248+0000 7fbcc5c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:12:49.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:12:49.248+0000 7fbcc5c0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:12:49.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.808+0000 7f7691ffb700 1 -- 172.21.15.57:0/2747547081 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f769402d2e0 con 0x7f76840a1500 2023-10-06T22:12:49.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76840a1500 msgr2=0x7f76840a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76840a1500 0x7f76840a39c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f768800b2e0 tx=0x7f7688005fb0 comp rx=0 tx=0).stop 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694108da0 msgr2=0x7f76941a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694108da0 0x7f76941a03d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f768c007ee0 tx=0x7f768c00d3b0 comp rx=0 tx=0).stop 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 shutdown_connections 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76840a1500 0x7f76840a39c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7694108da0 0x7f76941a03d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7694073910 0x7f7694198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 --2- 172.21.15.57:0/2747547081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7694074ef0 0x7f7694199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 >> 172.21.15.57:0/2747547081 conn(0x7f76940fc430 msgr2=0x7f76940fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 shutdown_connections 2023-10-06T22:12:49.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:49.810+0000 7f769abff700 1 -- 172.21.15.57:0/2747547081 wait complete. 2023-10-06T22:12:49.832 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-06T22:12:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:49 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/49967289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T22:12:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:49 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] up:active 2023-10-06T22:12:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:49 smithi156 ceph-mon[134358]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-06T22:12:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:49 smithi156 ceph-mon[134358]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-10-06T22:12:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:49 smithi156 ceph-mon[134358]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-10-06T22:12:50.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:49 smithi156 ceph-mon[134358]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-06T22:12:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.036+0000 7fcc8adff700 1 -- 172.21.15.57:0/1001344014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84102080 msgr2=0x7fcc84102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.036+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1001344014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84102080 0x7fcc84102500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009990 tx=0x7fcc70009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.037+0000 7fcc8adff700 1 -- 172.21.15.57:0/1001344014 shutdown_connections 2023-10-06T22:12:50.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.037+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1001344014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc84102a40 0x7fcc84109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.037+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1001344014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc84100ea0 0x7fcc841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.037+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1001344014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84102080 0x7fcc84102500 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.037+0000 7fcc8adff700 1 -- 172.21.15.57:0/1001344014 >> 172.21.15.57:0/1001344014 conn(0x7fcc840fc460 msgr2=0x7fcc840fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.037+0000 7fcc8adff700 1 -- 172.21.15.57:0/1001344014 shutdown_connections 2023-10-06T22:12:50.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.037+0000 7fcc8adff700 1 -- 172.21.15.57:0/1001344014 wait complete. 2023-10-06T22:12:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 Processor -- start 2023-10-06T22:12:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 -- start start 2023-10-06T22:12:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84100ea0 0x7fcc84196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc84102080 0x7fcc841971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc84102a40 0x7fcc8419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8419e800 con 0x7fcc84100ea0 2023-10-06T22:12:50.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8419e970 con 0x7fcc84102080 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8adff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8419eb30 con 0x7fcc84102a40 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc88b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84100ea0 0x7fcc84196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc83fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc84102080 0x7fcc841971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc8939c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc84102a40 0x7fcc8419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc88b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84100ea0 0x7fcc84196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59292/0 (socket says 172.21.15.57:59292) 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.038+0000 7fcc88b9b700 1 -- 172.21.15.57:0/1606453826 learned_addr learned my addr 172.21.15.57:0/1606453826 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc8939c700 1 -- 172.21.15.57:0/1606453826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc84102080 msgr2=0x7fcc841971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc8939c700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc84102080 0x7fcc841971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc8939c700 1 -- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84100ea0 msgr2=0x7fcc84196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc8939c700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84100ea0 0x7fcc84196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc8939c700 1 -- 172.21.15.57:0/1606453826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc700095f0 con 0x7fcc84102a40 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc88b9b700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84100ea0 0x7fcc84196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc8939c700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc84102a40 0x7fcc8419e230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00efc0 tx=0x7fcc7c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.039+0000 7fcc81ffb700 1 -- 172.21.15.57:0/1606453826 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c018070 con 0x7fcc84102a40 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.040+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8419edb0 con 0x7fcc84102a40 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.040+0000 7fcc81ffb700 1 -- 172.21.15.57:0/1606453826 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc7c00f040 con 0x7fcc84102a40 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.040+0000 7fcc81ffb700 1 -- 172.21.15.57:0/1606453826 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c00cd40 con 0x7fcc84102a40 2023-10-06T22:12:50.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.040+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8419f1e0 con 0x7fcc84102a40 2023-10-06T22:12:50.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.041+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc84108500 con 0x7fcc84102a40 2023-10-06T22:12:50.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.043+0000 7fcc81ffb700 1 -- 172.21.15.57:0/1606453826 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcc7c003a00 con 0x7fcc84102a40 2023-10-06T22:12:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.044+0000 7fcc81ffb700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc740a1480 0x7fcc740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.044+0000 7fcc81ffb700 1 -- 172.21.15.57:0/1606453826 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc7c0cd110 con 0x7fcc84102a40 2023-10-06T22:12:50.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.044+0000 7fcc88b9b700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc740a1480 0x7fcc740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.045+0000 7fcc88b9b700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc740a1480 0x7fcc740a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc84197b60 tx=0x7fcc78009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:50.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.046+0000 7fcc81ffb700 1 -- 172.21.15.57:0/1606453826 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcc7c0968b0 con 0x7fcc84102a40 2023-10-06T22:12:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:49 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/49967289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T22:12:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:49 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] up:active 2023-10-06T22:12:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:49 smithi190 ceph-mon[134891]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-06T22:12:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:49 smithi190 ceph-mon[134891]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-10-06T22:12:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:49 smithi190 ceph-mon[134891]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-10-06T22:12:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:49 smithi190 ceph-mon[134891]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-06T22:12:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/49967289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T22:12:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] up:active 2023-10-06T22:12:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-mon[98492]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-06T22:12:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-mon[98492]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-10-06T22:12:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-mon[98492]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-10-06T22:12:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:49 smithi057 ceph-mon[98492]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-06T22:12:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.294+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc8402d250 con 0x7fcc740a1480 2023-10-06T22:12:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.298+0000 7fcc81ffb700 1 -- 172.21.15.57:0/1606453826 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fcc8402d250 con 0x7fcc740a1480 2023-10-06T22:12:50.300 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/_nogroup/sv_0/eb571871-f6ea-4499-920e-b491b52a6267 2023-10-06T22:12:50.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc740a1480 msgr2=0x7fcc740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc740a1480 0x7fcc740a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc84197b60 tx=0x7fcc78009260 comp rx=0 tx=0).stop 2023-10-06T22:12:50.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc84102a40 msgr2=0x7fcc8419e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc84102a40 0x7fcc8419e230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00efc0 tx=0x7fcc7c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 shutdown_connections 2023-10-06T22:12:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcc740a1480 0x7fcc740a3940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc84102a40 0x7fcc8419e230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84100ea0 0x7fcc84196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 --2- 172.21.15.57:0/1606453826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc84102080 0x7fcc841971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 >> 172.21.15.57:0/1606453826 conn(0x7fcc840fc460 msgr2=0x7fcc84105640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 shutdown_connections 2023-10-06T22:12:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.301+0000 7fcc8adff700 1 -- 172.21.15.57:0/1606453826 wait complete. 2023-10-06T22:12:50.323 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-06T22:12:50.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 -- 172.21.15.57:0/2121312260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 msgr2=0x7faaec1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 --2- 172.21.15.57:0/2121312260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 0x7faaec1012b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faadc009990 tx=0x7faadc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 -- 172.21.15.57:0/2121312260 shutdown_connections 2023-10-06T22:12:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 --2- 172.21.15.57:0/2121312260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec102a30 0x7faaec109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 --2- 172.21.15.57:0/2121312260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 0x7faaec1012b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 --2- 172.21.15.57:0/2121312260 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 0x7faaec1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 -- 172.21.15.57:0/2121312260 >> 172.21.15.57:0/2121312260 conn(0x7faaec0fc430 msgr2=0x7faaec0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 -- 172.21.15.57:0/2121312260 shutdown_connections 2023-10-06T22:12:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.530+0000 7faaf13ff700 1 -- 172.21.15.57:0/2121312260 wait complete. 2023-10-06T22:12:50.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.531+0000 7faaf13ff700 1 Processor -- start 2023-10-06T22:12:50.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaf13ff700 1 -- start start 2023-10-06T22:12:50.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaf13ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 0x7faaec196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaf13ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 0x7faaec197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaf13ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec102a30 0x7faaec19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaf13ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec19e770 con 0x7faaec102070 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaf13ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec19e8b0 con 0x7faaec100e90 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaf13ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec19ea40 con 0x7faaec102a30 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faae3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 0x7faaec197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaeaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 0x7faaec196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faae3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 0x7faaec197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59298/0 (socket says 172.21.15.57:59298) 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faae3fff700 1 -- 172.21.15.57:0/247174478 learned_addr learned my addr 172.21.15.57:0/247174478 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.532+0000 7faaeb7fe700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec102a30 0x7faaec19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae3fff700 1 -- 172.21.15.57:0/247174478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec102a30 msgr2=0x7faaec19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae3fff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec102a30 0x7faaec19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae3fff700 1 -- 172.21.15.57:0/247174478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 msgr2=0x7faaec196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae3fff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 0x7faaec196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae3fff700 1 -- 172.21.15.57:0/247174478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0095f0 con 0x7faaec102070 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae3fff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 0x7faaec197130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faad400e970 tx=0x7faad400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae8ff9700 1 -- 172.21.15.57:0/247174478 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4009790 con 0x7faaec102070 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec19ed20 con 0x7faaec102070 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae8ff9700 1 -- 172.21.15.57:0/247174478 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad400c7e0 con 0x7faaec102070 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faae8ff9700 1 -- 172.21.15.57:0/247174478 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4003c70 con 0x7faaec102070 2023-10-06T22:12:50.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.533+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec19f220 con 0x7faaec102070 2023-10-06T22:12:50.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.534+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec107e90 con 0x7faaec102070 2023-10-06T22:12:50.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.538+0000 7faae8ff9700 1 -- 172.21.15.57:0/247174478 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faad400cc50 con 0x7faaec102070 2023-10-06T22:12:50.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.540+0000 7faae8ff9700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faad80a1480 0x7faad80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:50.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.540+0000 7faae8ff9700 1 -- 172.21.15.57:0/247174478 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faad4014070 con 0x7faaec102070 2023-10-06T22:12:50.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.540+0000 7faaeaffd700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faad80a1480 0x7faad80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:50.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.540+0000 7faaeaffd700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faad80a1480 0x7faad80a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faadc00b680 tx=0x7faadc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:50.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.541+0000 7faae8ff9700 1 -- 172.21.15.57:0/247174478 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faad4096690 con 0x7faaec102070 2023-10-06T22:12:50.766 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: 2023-10-06T22:12:50.441+0000 7eff0ff7e700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-10-06T22:12:50.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.797+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7faaec02d250 con 0x7faad80a1480 2023-10-06T22:12:50.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.823+0000 7faae8ff9700 1 -- 172.21.15.57:0/247174478 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faaec02d250 con 0x7faad80a1480 2023-10-06T22:12:50.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faad80a1480 msgr2=0x7faad80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faad80a1480 0x7faad80a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faadc00b680 tx=0x7faadc005fd0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 msgr2=0x7faaec197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:50.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 0x7faaec197130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faad400e970 tx=0x7faad400ec80 comp rx=0 tx=0).stop 2023-10-06T22:12:50.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 shutdown_connections 2023-10-06T22:12:50.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faad80a1480 0x7faad80a3940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaec102a30 0x7faaec19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaec100e90 0x7faaec196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 --2- 172.21.15.57:0/247174478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faaec102070 0x7faaec197130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:50.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.825+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 >> 172.21.15.57:0/247174478 conn(0x7faaec0fc430 msgr2=0x7faaec0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:50.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.826+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 shutdown_connections 2023-10-06T22:12:50.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:50.826+0000 7faaf13ff700 1 -- 172.21.15.57:0/247174478 wait complete. 2023-10-06T22:12:50.847 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-06T22:12:51.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 -- 172.21.15.57:0/4264626437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102070 msgr2=0x7f77601024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 --2- 172.21.15.57:0/4264626437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102070 0x7f77601024f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7748009990 tx=0x7f7748009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 -- 172.21.15.57:0/4264626437 shutdown_connections 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 --2- 172.21.15.57:0/4264626437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7760102a30 0x7f7760109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 --2- 172.21.15.57:0/4264626437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7760100e90 0x7f77601012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 --2- 172.21.15.57:0/4264626437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102070 0x7f77601024f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 -- 172.21.15.57:0/4264626437 >> 172.21.15.57:0/4264626437 conn(0x7f77600fc430 msgr2=0x7f77600fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 -- 172.21.15.57:0/4264626437 shutdown_connections 2023-10-06T22:12:51.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.051+0000 7f7764fff700 1 -- 172.21.15.57:0/4264626437 wait complete. 2023-10-06T22:12:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 Processor -- start 2023-10-06T22:12:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 -- start start 2023-10-06T22:12:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7760100e90 0x7f7760198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7760102070 0x7f77601993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102a30 0x7f77601a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77601a09f0 con 0x7f7760102a30 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77601a0b30 con 0x7f7760100e90 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.052+0000 7f7764fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77601a0cc0 con 0x7f7760102070 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102a30 0x7f77601a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7760100e90 0x7f7760198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7760102070 0x7f77601993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102a30 0x7f77601a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59314/0 (socket says 172.21.15.57:59314) 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 -- 172.21.15.57:0/1940418359 learned_addr learned my addr 172.21.15.57:0/1940418359 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 -- 172.21.15.57:0/1940418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7760102070 msgr2=0x7f77601993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7760102070 0x7f77601993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 -- 172.21.15.57:0/1940418359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7760100e90 msgr2=0x7f7760198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7760100e90 0x7f7760198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 -- 172.21.15.57:0/1940418359 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480095f0 con 0x7f7760102a30 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775e59c700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7760102070 0x7f77601993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775659c700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7760100e90 0x7f7760198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.053+0000 7f775ed9d700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102a30 0x7f77601a0420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f775800ef30 tx=0x7f775800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.054+0000 7f7757fff700 1 -- 172.21.15.57:0/1940418359 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758018070 con 0x7f7760102a30 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.054+0000 7f7757fff700 1 -- 172.21.15.57:0/1940418359 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f775800f040 con 0x7f7760102a30 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.054+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77601a0f40 con 0x7f7760102a30 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.054+0000 7f7757fff700 1 -- 172.21.15.57:0/1940418359 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800cd00 con 0x7f7760102a30 2023-10-06T22:12:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.054+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77601a1370 con 0x7f7760102a30 2023-10-06T22:12:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.055+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7760193540 con 0x7f7760102a30 2023-10-06T22:12:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.057+0000 7f7757fff700 1 -- 172.21.15.57:0/1940418359 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7758003a00 con 0x7f7760102a30 2023-10-06T22:12:51.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.058+0000 7f7757fff700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f774c0a1480 0x7f774c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.059+0000 7f7757fff700 1 -- 172.21.15.57:0/1940418359 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f77580cc6c0 con 0x7f7760102a30 2023-10-06T22:12:51.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.059+0000 7f775659c700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f774c0a1480 0x7f774c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.059+0000 7f775659c700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f774c0a1480 0x7f774c0a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7760101be0 tx=0x7f7750009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:51.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.063+0000 7f7757fff700 1 -- 172.21.15.57:0/1940418359 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7758095f00 con 0x7f7760102a30 2023-10-06T22:12:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: pgmap v275: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s wr, 12 op/s 2023-10-06T22:12:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:active 2023-10-06T22:12:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-06T22:12:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T22:12:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: Cluster is now healthy 2023-10-06T22:12:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2023-10-06T22:12:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-06T22:12:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: daemon mds.h is now active in filesystem cephfs as rank 4 2023-10-06T22:12:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:active 2023-10-06T22:12:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:50 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: pgmap v275: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s wr, 12 op/s 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:active 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: Cluster is now healthy 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2023-10-06T22:12:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-06T22:12:51.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: daemon mds.h is now active in filesystem cephfs as rank 4 2023-10-06T22:12:51.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:active 2023-10-06T22:12:51.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:50 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: pgmap v275: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s wr, 12 op/s 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:active 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: Cluster is now healthy 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-06T22:12:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: daemon mds.h is now active in filesystem cephfs as rank 4 2023-10-06T22:12:51.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:active 2023-10-06T22:12:51.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:50 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:12:51.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.311+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f776002d2e0 con 0x7f774c0a1480 2023-10-06T22:12:51.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.316+0000 7f7757fff700 1 -- 172.21.15.57:0/1940418359 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f776002d2e0 con 0x7f774c0a1480 2023-10-06T22:12:51.318 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc 2023-10-06T22:12:51.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f774c0a1480 msgr2=0x7f774c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f774c0a1480 0x7f774c0a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7760101be0 tx=0x7f7750009260 comp rx=0 tx=0).stop 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102a30 msgr2=0x7f77601a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102a30 0x7f77601a0420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f775800ef30 tx=0x7f775800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 shutdown_connections 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f774c0a1480 0x7f774c0a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7760102a30 0x7f77601a0420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7760100e90 0x7f7760198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 --2- 172.21.15.57:0/1940418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7760102070 0x7f77601993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 >> 172.21.15.57:0/1940418359 conn(0x7f77600fc430 msgr2=0x7f7760107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:51.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 shutdown_connections 2023-10-06T22:12:51.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.318+0000 7f7764fff700 1 -- 172.21.15.57:0/1940418359 wait complete. 2023-10-06T22:12:51.339 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:12:51.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 -- 172.21.15.57:0/4116595333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8100ea0 msgr2=0x7fa7a81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/4116595333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8100ea0 0x7fa7a81012c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40098f0 tx=0x7fa7a4009c00 comp rx=0 tx=0).stop 2023-10-06T22:12:51.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 -- 172.21.15.57:0/4116595333 shutdown_connections 2023-10-06T22:12:51.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/4116595333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102a40 0x7fa7a8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/4116595333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8100ea0 0x7fa7a81012c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/4116595333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8102080 0x7fa7a8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 -- 172.21.15.57:0/4116595333 >> 172.21.15.57:0/4116595333 conn(0x7fa7a80fc420 msgr2=0x7fa7a80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:51.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 -- 172.21.15.57:0/4116595333 shutdown_connections 2023-10-06T22:12:51.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.540+0000 7fa7b07ff700 1 -- 172.21.15.57:0/4116595333 wait complete. 2023-10-06T22:12:51.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.541+0000 7fa7b07ff700 1 Processor -- start 2023-10-06T22:12:51.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.541+0000 7fa7b07ff700 1 -- start start 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7b07ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8100ea0 0x7fa7a8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7b07ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8102080 0x7fa7a80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7b07ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102a40 0x7fa7a80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7b07ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8073f80 con 0x7fa7a8100ea0 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7b07ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a80740f0 con 0x7fa7a8102080 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7b07ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81a0cb0 con 0x7fa7a8102a40 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8100ea0 0x7fa7a8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8100ea0 0x7fa7a8074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59324/0 (socket says 172.21.15.57:59324) 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 -- 172.21.15.57:0/1531224492 learned_addr learned my addr 172.21.15.57:0/1531224492 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7add9a700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8102080 0x7fa7a80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7aed9c700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102a40 0x7fa7a80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 -- 172.21.15.57:0/1531224492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102a40 msgr2=0x7fa7a80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102a40 0x7fa7a80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 -- 172.21.15.57:0/1531224492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8102080 msgr2=0x7fa7a80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8102080 0x7fa7a80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7ae59b700 1 -- 172.21.15.57:0/1531224492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4009520 con 0x7fa7a8100ea0 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.542+0000 7fa7add9a700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8102080 0x7fa7a80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.543+0000 7fa7ae59b700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8100ea0 0x7fa7a8074c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fd0 tx=0x7fa7a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.543+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a401c070 con 0x7fa7a8100ea0 2023-10-06T22:12:51.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.543+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7a400f820 con 0x7fa7a8100ea0 2023-10-06T22:12:51.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.543+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a81a0f30 con 0x7fa7a8100ea0 2023-10-06T22:12:51.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.543+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4003be0 con 0x7fa7a8100ea0 2023-10-06T22:12:51.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.543+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81a12f0 con 0x7fa7a8100ea0 2023-10-06T22:12:51.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.544+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a810c760 con 0x7fa7a8100ea0 2023-10-06T22:12:51.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.546+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa7a4020030 con 0x7fa7a8100ea0 2023-10-06T22:12:51.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.547+0000 7fa79f7fe700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7940a1550 0x7fa7940a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:51.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.547+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7a40cdd50 con 0x7fa7a8100ea0 2023-10-06T22:12:51.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.547+0000 7fa7add9a700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7940a1550 0x7fa7940a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:51.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.548+0000 7fa7add9a700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7940a1550 0x7fa7940a3a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8100bc0 tx=0x7fa798009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:51.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.551+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa7a4101100 con 0x7fa7a8100ea0 2023-10-06T22:12:51.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.789+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa7a4003d40 con 0x7fa7a8100ea0 2023-10-06T22:12:51.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.823+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7a8107e60 con 0x7fa7a8100ea0 2023-10-06T22:12:51.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.824+0000 7fa79f7fe700 1 -- 172.21.15.57:0/1531224492 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fa7a4095490 con 0x7fa7a8100ea0 2023-10-06T22:12:51.826 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:51.826 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:12:51.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7940a1550 msgr2=0x7fa7940a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7940a1550 0x7fa7940a3a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8100bc0 tx=0x7fa798009310 comp rx=0 tx=0).stop 2023-10-06T22:12:51.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8100ea0 msgr2=0x7fa7a8074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:51.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8100ea0 0x7fa7a8074c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009fd0 tx=0x7fa7a400b840 comp rx=0 tx=0).stop 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 shutdown_connections 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7940a1550 0x7fa7940a3a10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7a8102a40 0x7fa7a80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7a8100ea0 0x7fa7a8074c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 --2- 172.21.15.57:0/1531224492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7a8102080 0x7fa7a80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.826+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 >> 172.21.15.57:0/1531224492 conn(0x7fa7a80fc420 msgr2=0x7fa7a80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.827+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 shutdown_connections 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:51.827+0000 7fa7b07ff700 1 -- 172.21.15.57:0/1531224492 wait complete. 2023-10-06T22:12:51.830 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:12:51.850 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 -- 172.21.15.57:0/907974228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 msgr2=0x7f2c00102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 --2- 172.21.15.57:0/907974228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00102500 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009990 tx=0x7f2bf0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 -- 172.21.15.57:0/907974228 shutdown_connections 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 --2- 172.21.15.57:0/907974228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c00102a40 0x7f2c00109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 --2- 172.21.15.57:0/907974228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c00100ea0 0x7f2c001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 --2- 172.21.15.57:0/907974228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00102500 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 -- 172.21.15.57:0/907974228 >> 172.21.15.57:0/907974228 conn(0x7f2c000fc420 msgr2=0x7f2c000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 -- 172.21.15.57:0/907974228 shutdown_connections 2023-10-06T22:12:52.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.051+0000 7f2c081ff700 1 -- 172.21.15.57:0/907974228 wait complete. 2023-10-06T22:12:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.052+0000 7f2c081ff700 1 Processor -- start 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.052+0000 7f2c081ff700 1 -- start start 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c081ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c00100ea0 0x7f2c00196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c081ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c081ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c00102a40 0x7f2c0019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c081ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0019e740 con 0x7f2c00102080 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c081ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0019e880 con 0x7f2c00102a40 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c081ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0019ea10 con 0x7f2c00100ea0 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54494/0 (socket says 172.21.15.57:54494) 2023-10-06T22:12:52.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 -- 172.21.15.57:0/1200646579 learned_addr learned my addr 172.21.15.57:0/1200646579 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c05f9b700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c00100ea0 0x7f2c00196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0679c700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c00102a40 0x7f2c0019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 -- 172.21.15.57:0/1200646579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c00100ea0 msgr2=0x7f2c00196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c00100ea0 0x7f2c00196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 -- 172.21.15.57:0/1200646579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c00102a40 msgr2=0x7f2c0019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c00102a40 0x7f2c0019e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.053+0000 7f2c0579a700 1 -- 172.21.15.57:0/1200646579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf00095f0 con 0x7f2c00102080 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.054+0000 7f2c0579a700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00197100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0005720 tx=0x7f2bf0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.054+0000 7f2bf6ffd700 1 -- 172.21.15.57:0/1200646579 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf001d070 con 0x7f2c00102080 2023-10-06T22:12:52.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.054+0000 7f2bf6ffd700 1 -- 172.21.15.57:0/1200646579 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bf0021410 con 0x7f2c00102080 2023-10-06T22:12:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.054+0000 7f2bf6ffd700 1 -- 172.21.15.57:0/1200646579 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0017650 con 0x7f2c00102080 2023-10-06T22:12:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.054+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0019ec90 con 0x7f2c00102080 2023-10-06T22:12:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.054+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0019f060 con 0x7f2c00102080 2023-10-06T22:12:52.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.055+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0010c760 con 0x7f2c00102080 2023-10-06T22:12:52.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.057+0000 7f2bf6ffd700 1 -- 172.21.15.57:0/1200646579 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2bf0021580 con 0x7f2c00102080 2023-10-06T22:12:52.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.059+0000 7f2bf6ffd700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bec0a15d0 0x7f2bec0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.059+0000 7f2bf6ffd700 1 -- 172.21.15.57:0/1200646579 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2bf00cdf40 con 0x7f2c00102080 2023-10-06T22:12:52.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.060+0000 7f2c05f9b700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bec0a15d0 0x7f2bec0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.060+0000 7f2c05f9b700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bec0a15d0 0x7f2bec0a3a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c00101bf0 tx=0x7f2bfc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:52.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.064+0000 7f2bf6ffd700 1 -- 172.21.15.57:0/1200646579 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2bf0097580 con 0x7f2c00102080 2023-10-06T22:12:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:51 smithi190 ceph-mon[134891]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:51 smithi190 ceph-mon[134891]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:51 smithi190 ceph-mon[134891]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:51 smithi190 ceph-mon[134891]: pgmap v276: 97 pgs: 97 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s wr, 13 op/s 2023-10-06T22:12:52.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:51 smithi057 ceph-mon[98492]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:51 smithi057 ceph-mon[98492]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:51 smithi057 ceph-mon[98492]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:51 smithi057 ceph-mon[98492]: pgmap v276: 97 pgs: 97 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s wr, 13 op/s 2023-10-06T22:12:52.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:51 smithi156 ceph-mon[134358]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:51 smithi156 ceph-mon[134358]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:51 smithi156 ceph-mon[134358]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:12:52.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:51 smithi156 ceph-mon[134358]: pgmap v276: 97 pgs: 97 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s wr, 13 op/s 2023-10-06T22:12:52.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.277+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c0010c970 con 0x7f2c00102080 2023-10-06T22:12:52.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.278+0000 7f2bf6ffd700 1 -- 172.21.15.57:0/1200646579 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22109 (secure 0 0 0) 0x7f2bf009aba0 con 0x7f2c00102080 2023-10-06T22:12:52.281 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:52.281 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":83,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:12:44.676689+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:12:52.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bec0a15d0 msgr2=0x7f2bec0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bec0a15d0 0x7f2bec0a3a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c00101bf0 tx=0x7f2bfc009260 comp rx=0 tx=0).stop 2023-10-06T22:12:52.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 msgr2=0x7f2c00197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00197100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0005720 tx=0x7f2bf0009580 comp rx=0 tx=0).stop 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 shutdown_connections 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bec0a15d0 0x7f2bec0a3a90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c00102a40 0x7f2c0019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c00100ea0 0x7f2c00196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 --2- 172.21.15.57:0/1200646579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c00102080 0x7f2c00197100 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 >> 172.21.15.57:0/1200646579 conn(0x7f2c000fc420 msgr2=0x7f2c000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 shutdown_connections 2023-10-06T22:12:52.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.281+0000 7f2c081ff700 1 -- 172.21.15.57:0/1200646579 wait complete. 2023-10-06T22:12:52.302 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T22:12:52.303 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 -- 172.21.15.57:0/1929763829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 msgr2=0x7f5db4102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 --2- 172.21.15.57:0/1929763829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4102500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009990 tx=0x7f5da8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 -- 172.21.15.57:0/1929763829 shutdown_connections 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 --2- 172.21.15.57:0/1929763829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db4102a40 0x7f5db4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 --2- 172.21.15.57:0/1929763829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5db4100ea0 0x7f5db41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 --2- 172.21.15.57:0/1929763829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4102500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 -- 172.21.15.57:0/1929763829 >> 172.21.15.57:0/1929763829 conn(0x7f5db40fc420 msgr2=0x7f5db40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 -- 172.21.15.57:0/1929763829 shutdown_connections 2023-10-06T22:12:52.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.504+0000 7f5db99ff700 1 -- 172.21.15.57:0/1929763829 wait complete. 2023-10-06T22:12:52.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.505+0000 7f5db99ff700 1 Processor -- start 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.505+0000 7f5db99ff700 1 -- start start 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.505+0000 7f5db99ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db4100ea0 0x7f5db4196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db99ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db99ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5db4102a40 0x7f5db419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db99ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db419e770 con 0x7f5db4102080 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db99ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db419e8b0 con 0x7f5db4100ea0 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db99ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db419ea40 con 0x7f5db4102a40 2023-10-06T22:12:52.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54514/0 (socket says 172.21.15.57:54514) 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 -- 172.21.15.57:0/940138652 learned_addr learned my addr 172.21.15.57:0/940138652 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db2ffd700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db4100ea0 0x7f5db4196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db37fe700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5db4102a40 0x7f5db419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 -- 172.21.15.57:0/940138652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5db4102a40 msgr2=0x7f5db419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5db4102a40 0x7f5db419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 -- 172.21.15.57:0/940138652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db4100ea0 msgr2=0x7f5db4196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db4100ea0 0x7f5db4196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.506+0000 7f5db27fc700 1 -- 172.21.15.57:0/940138652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da80095f0 con 0x7f5db4102080 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.507+0000 7f5db27fc700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4197130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005720 tx=0x7f5da8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:52.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.507+0000 7f5db89fd700 1 -- 172.21.15.57:0/940138652 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da801d070 con 0x7f5db4102080 2023-10-06T22:12:52.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.507+0000 7f5db89fd700 1 -- 172.21.15.57:0/940138652 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da8021410 con 0x7f5db4102080 2023-10-06T22:12:52.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.507+0000 7f5db89fd700 1 -- 172.21.15.57:0/940138652 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8017650 con 0x7f5db4102080 2023-10-06T22:12:52.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.507+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db419ecc0 con 0x7f5db4102080 2023-10-06T22:12:52.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.507+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db419f0e0 con 0x7f5db4102080 2023-10-06T22:12:52.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.508+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db41084b0 con 0x7f5db4102080 2023-10-06T22:12:52.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.510+0000 7f5db89fd700 1 -- 172.21.15.57:0/940138652 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5da8021580 con 0x7f5db4102080 2023-10-06T22:12:52.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.511+0000 7f5db89fd700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d9c0a15d0 0x7f5d9c0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:52.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.511+0000 7f5db89fd700 1 -- 172.21.15.57:0/940138652 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5da80cdee0 con 0x7f5db4102080 2023-10-06T22:12:52.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.511+0000 7f5db2ffd700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d9c0a15d0 0x7f5d9c0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:52.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.512+0000 7f5db2ffd700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d9c0a15d0 0x7f5d9c0a3a90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5db4101bf0 tx=0x7f5da4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:52.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.515+0000 7f5db89fd700 1 -- 172.21.15.57:0/940138652 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5da8097520 con 0x7f5db4102080 2023-10-06T22:12:52.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.723+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5db402d2f0 con 0x7f5d9c0a15d0 2023-10-06T22:12:52.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.736+0000 7f5db89fd700 1 -- 172.21.15.57:0/940138652 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336191 (secure 0 0 0) 0x7f5db402d2f0 con 0x7f5d9c0a15d0 2023-10-06T22:12:52.743 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d9c0a15d0 msgr2=0x7f5d9c0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d9c0a15d0 0x7f5d9c0a3a90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5db4101bf0 tx=0x7f5da4009260 comp rx=0 tx=0).stop 2023-10-06T22:12:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 msgr2=0x7f5db4197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4197130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005720 tx=0x7f5da8009580 comp rx=0 tx=0).stop 2023-10-06T22:12:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 shutdown_connections 2023-10-06T22:12:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d9c0a15d0 0x7f5d9c0a3a90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5db4102a40 0x7f5db419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5db4100ea0 0x7f5db4196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 --2- 172.21.15.57:0/940138652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5db4102080 0x7f5db4197130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:52.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.740+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 >> 172.21.15.57:0/940138652 conn(0x7f5db40fc420 msgr2=0x7f5db41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:52.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.741+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 shutdown_connections 2023-10-06T22:12:52.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:52.741+0000 7f5db99ff700 1 -- 172.21.15.57:0/940138652 wait complete. 2023-10-06T22:12:52.747 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-06T22:12:52.762 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2023-10-06T22:12:51.614019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597094,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":292,"num_write_kb":2985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336556,"kb_used_data":12332,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605268,"statfs":{"total":1151940427776,"available":1151595794432,"internally_reserved":0,"allocated":12627968,"data_stored":6894038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002060"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200666+0000","last_change":"2023-10-06T22:12:43.704484+0000","last_active":"2023-10-06T22:12:45.200666+0000","last_peered":"2023-10-06T22:12:45.200666+0000","last_clean":"2023-10-06T22:12:45.200666+0000","last_became_active":"2023-10-06T22:12:43.703740+0000","last_became_peered":"2023-10-06T22:12:43.703740+0000","last_unstale":"2023-10-06T22:12:45.200666+0000","last_undegraded":"2023-10-06T22:12:45.200666+0000","last_fullsized":"2023-10-06T22:12:45.200666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:25.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061532+0000","last_change":"2023-10-06T22:12:43.696968+0000","last_active":"2023-10-06T22:12:45.061532+0000","last_peered":"2023-10-06T22:12:45.061532+0000","last_clean":"2023-10-06T22:12:45.061532+0000","last_became_active":"2023-10-06T22:12:43.696532+0000","last_became_peered":"2023-10-06T22:12:43.696532+0000","last_unstale":"2023-10-06T22:12:45.061532+0000","last_undegraded":"2023-10-06T22:12:45.061532+0000","last_fullsized":"2023-10-06T22:12:45.061532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:59:50.754606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061851+0000","last_change":"2023-10-06T22:12:43.700845+0000","last_active":"2023-10-06T22:12:45.061851+0000","last_peered":"2023-10-06T22:12:45.061851+0000","last_clean":"2023-10-06T22:12:45.061851+0000","last_became_active":"2023-10-06T22:12:43.700531+0000","last_became_peered":"2023-10-06T22:12:43.700531+0000","last_unstale":"2023-10-06T22:12:45.061851+0000","last_undegraded":"2023-10-06T22:12:45.061851+0000","last_fullsized":"2023-10-06T22:12:45.061851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:18.518772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061912+0000","last_change":"2023-10-06T22:12:43.700806+0000","last_active":"2023-10-06T22:12:45.061912+0000","last_peered":"2023-10-06T22:12:45.061912+0000","last_clean":"2023-10-06T22:12:45.061912+0000","last_became_active":"2023-10-06T22:12:43.699901+0000","last_became_peered":"2023-10-06T22:12:43.699901+0000","last_unstale":"2023-10-06T22:12:45.061912+0000","last_undegraded":"2023-10-06T22:12:45.061912+0000","last_fullsized":"2023-10-06T22:12:45.061912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:41:35.906415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060439+0000","last_change":"2023-10-06T22:12:43.694776+0000","last_active":"2023-10-06T22:12:45.060439+0000","last_peered":"2023-10-06T22:12:45.060439+0000","last_clean":"2023-10-06T22:12:45.060439+0000","last_became_active":"2023-10-06T22:12:43.694397+0000","last_became_peered":"2023-10-06T22:12:43.694397+0000","last_unstale":"2023-10-06T22:12:45.060439+0000","last_undegraded":"2023-10-06T22:12:45.060439+0000","last_fullsized":"2023-10-06T22:12:45.060439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:47:43.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060558+0000","last_change":"2023-10-06T22:12:43.703515+0000","last_active":"2023-10-06T22:12:45.060558+0000","last_peered":"2023-10-06T22:12:45.060558+0000","last_clean":"2023-10-06T22:12:45.060558+0000","last_became_active":"2023-10-06T22:12:43.703094+0000","last_became_peered":"2023-10-06T22:12:43.703094+0000","last_unstale":"2023-10-06T22:12:45.060558+0000","last_undegraded":"2023-10-06T22:12:45.060558+0000","last_fullsized":"2023-10-06T22:12:45.060558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:54:32.130131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060949+0000","last_change":"2023-10-06T22:12:43.697894+0000","last_active":"2023-10-06T22:12:45.060949+0000","last_peered":"2023-10-06T22:12:45.060949+0000","last_clean":"2023-10-06T22:12:45.060949+0000","last_became_active":"2023-10-06T22:12:43.697591+0000","last_became_peered":"2023-10-06T22:12:43.697591+0000","last_unstale":"2023-10-06T22:12:45.060949+0000","last_undegraded":"2023-10-06T22:12:45.060949+0000","last_fullsized":"2023-10-06T22:12:45.060949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:30.313447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060532+0000","last_change":"2023-10-06T22:12:43.696555+0000","last_active":"2023-10-06T22:12:45.060532+0000","last_peered":"2023-10-06T22:12:45.060532+0000","last_clean":"2023-10-06T22:12:45.060532+0000","last_became_active":"2023-10-06T22:12:43.696185+0000","last_became_peered":"2023-10-06T22:12:43.696185+0000","last_unstale":"2023-10-06T22:12:45.060532+0000","last_undegraded":"2023-10-06T22:12:45.060532+0000","last_fullsized":"2023-10-06T22:12:45.060532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:35:59.584557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684897+0000","last_change":"2023-10-06T22:12:43.696995+0000","last_active":"2023-10-06T22:12:44.684897+0000","last_peered":"2023-10-06T22:12:44.684897+0000","last_clean":"2023-10-06T22:12:44.684897+0000","last_became_active":"2023-10-06T22:12:43.696731+0000","last_became_peered":"2023-10-06T22:12:43.696731+0000","last_unstale":"2023-10-06T22:12:44.684897+0000","last_undegraded":"2023-10-06T22:12:44.684897+0000","last_fullsized":"2023-10-06T22:12:44.684897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:43:31.261922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060958+0000","last_change":"2023-10-06T22:12:43.704391+0000","last_active":"2023-10-06T22:12:45.060958+0000","last_peered":"2023-10-06T22:12:45.060958+0000","last_clean":"2023-10-06T22:12:45.060958+0000","last_became_active":"2023-10-06T22:12:43.702954+0000","last_became_peered":"2023-10-06T22:12:43.702954+0000","last_unstale":"2023-10-06T22:12:45.060958+0000","last_undegraded":"2023-10-06T22:12:45.060958+0000","last_fullsized":"2023-10-06T22:12:45.060958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:38.973781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061588+0000","last_change":"2023-10-06T22:12:43.696265+0000","last_active":"2023-10-06T22:12:45.061588+0000","last_peered":"2023-10-06T22:12:45.061588+0000","last_clean":"2023-10-06T22:12:45.061588+0000","last_became_active":"2023-10-06T22:12:43.695939+0000","last_became_peered":"2023-10-06T22:12:43.695939+0000","last_unstale":"2023-10-06T22:12:45.061588+0000","last_undegraded":"2023-10-06T22:12:45.061588+0000","last_fullsized":"2023-10-06T22:12:45.061588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:32.882083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061835+0000","last_change":"2023-10-06T22:12:43.698527+0000","last_active":"2023-10-06T22:12:45.061835+0000","last_peered":"2023-10-06T22:12:45.061835+0000","last_clean":"2023-10-06T22:12:45.061835+0000","last_became_active":"2023-10-06T22:12:43.697867+0000","last_became_peered":"2023-10-06T22:12:43.697867+0000","last_unstale":"2023-10-06T22:12:45.061835+0000","last_undegraded":"2023-10-06T22:12:45.061835+0000","last_fullsized":"2023-10-06T22:12:45.061835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:08:03.617390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834916+0000","last_change":"2023-10-06T22:12:43.694717+0000","last_active":"2023-10-06T22:12:44.834916+0000","last_peered":"2023-10-06T22:12:44.834916+0000","last_clean":"2023-10-06T22:12:44.834916+0000","last_became_active":"2023-10-06T22:12:43.694321+0000","last_became_peered":"2023-10-06T22:12:43.694321+0000","last_unstale":"2023-10-06T22:12:44.834916+0000","last_undegraded":"2023-10-06T22:12:44.834916+0000","last_fullsized":"2023-10-06T22:12:44.834916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:06.334726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061102+0000","last_change":"2023-10-06T22:12:43.703770+0000","last_active":"2023-10-06T22:12:45.061102+0000","last_peered":"2023-10-06T22:12:45.061102+0000","last_clean":"2023-10-06T22:12:45.061102+0000","last_became_active":"2023-10-06T22:12:43.703434+0000","last_became_peered":"2023-10-06T22:12:43.703434+0000","last_unstale":"2023-10-06T22:12:45.061102+0000","last_undegraded":"2023-10-06T22:12:45.061102+0000","last_fullsized":"2023-10-06T22:12:45.061102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:25.894280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060975+0000","last_change":"2023-10-06T22:12:43.700994+0000","last_active":"2023-10-06T22:12:45.060975+0000","last_peered":"2023-10-06T22:12:45.060975+0000","last_clean":"2023-10-06T22:12:45.060975+0000","last_became_active":"2023-10-06T22:12:43.700652+0000","last_became_peered":"2023-10-06T22:12:43.700652+0000","last_unstale":"2023-10-06T22:12:45.060975+0000","last_undegraded":"2023-10-06T22:12:45.060975+0000","last_fullsized":"2023-10-06T22:12:45.060975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:54.855597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684430+0000","last_change":"2023-10-06T22:12:43.702099+0000","last_active":"2023-10-06T22:12:44.684430+0000","last_peered":"2023-10-06T22:12:44.684430+0000","last_clean":"2023-10-06T22:12:44.684430+0000","last_became_active":"2023-10-06T22:12:43.701802+0000","last_became_peered":"2023-10-06T22:12:43.701802+0000","last_unstale":"2023-10-06T22:12:44.684430+0000","last_undegraded":"2023-10-06T22:12:44.684430+0000","last_fullsized":"2023-10-06T22:12:44.684430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:00.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200796+0000","last_change":"2023-10-06T22:12:43.704679+0000","last_active":"2023-10-06T22:12:45.200796+0000","last_peered":"2023-10-06T22:12:45.200796+0000","last_clean":"2023-10-06T22:12:45.200796+0000","last_became_active":"2023-10-06T22:12:43.704186+0000","last_became_peered":"2023-10-06T22:12:43.704186+0000","last_unstale":"2023-10-06T22:12:45.200796+0000","last_undegraded":"2023-10-06T22:12:45.200796+0000","last_fullsized":"2023-10-06T22:12:45.200796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:28.665217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.686808+0000","last_change":"2023-10-06T22:12:43.695699+0000","last_active":"2023-10-06T22:12:44.686808+0000","last_peered":"2023-10-06T22:12:44.686808+0000","last_clean":"2023-10-06T22:12:44.686808+0000","last_became_active":"2023-10-06T22:12:43.695359+0000","last_became_peered":"2023-10-06T22:12:43.695359+0000","last_unstale":"2023-10-06T22:12:44.686808+0000","last_undegraded":"2023-10-06T22:12:44.686808+0000","last_fullsized":"2023-10-06T22:12:44.686808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:46.296768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.685297+0000","last_change":"2023-10-06T22:12:43.707170+0000","last_active":"2023-10-06T22:12:44.685297+0000","last_peered":"2023-10-06T22:12:44.685297+0000","last_clean":"2023-10-06T22:12:44.685297+0000","last_became_active":"2023-10-06T22:12:43.706875+0000","last_became_peered":"2023-10-06T22:12:43.706875+0000","last_unstale":"2023-10-06T22:12:44.685297+0000","last_undegraded":"2023-10-06T22:12:44.685297+0000","last_fullsized":"2023-10-06T22:12:44.685297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:46.548458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.726181+0000","last_change":"2023-10-06T22:12:42.693280+0000","last_active":"2023-10-06T22:12:43.726181+0000","last_peered":"2023-10-06T22:12:43.726181+0000","last_clean":"2023-10-06T22:12:43.726181+0000","last_became_active":"2023-10-06T22:12:42.692871+0000","last_became_peered":"2023-10-06T22:12:42.692871+0000","last_unstale":"2023-10-06T22:12:43.726181+0000","last_undegraded":"2023-10-06T22:12:43.726181+0000","last_fullsized":"2023-10-06T22:12:43.726181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.021829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061673+0000","last_change":"2023-10-06T22:12:43.702574+0000","last_active":"2023-10-06T22:12:45.061673+0000","last_peered":"2023-10-06T22:12:45.061673+0000","last_clean":"2023-10-06T22:12:45.061673+0000","last_became_active":"2023-10-06T22:12:43.702238+0000","last_became_peered":"2023-10-06T22:12:43.702238+0000","last_unstale":"2023-10-06T22:12:45.061673+0000","last_undegraded":"2023-10-06T22:12:45.061673+0000","last_fullsized":"2023-10-06T22:12:45.061673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:34:09.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.723314+0000","last_change":"2023-10-06T22:12:42.697256+0000","last_active":"2023-10-06T22:12:43.723314+0000","last_peered":"2023-10-06T22:12:43.723314+0000","last_clean":"2023-10-06T22:12:43.723314+0000","last_became_active":"2023-10-06T22:12:42.696724+0000","last_became_peered":"2023-10-06T22:12:42.696724+0000","last_unstale":"2023-10-06T22:12:43.723314+0000","last_undegraded":"2023-10-06T22:12:43.723314+0000","last_fullsized":"2023-10-06T22:12:43.723314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:50:01.378292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061995+0000","last_change":"2023-10-06T22:12:43.702050+0000","last_active":"2023-10-06T22:12:45.061995+0000","last_peered":"2023-10-06T22:12:45.061995+0000","last_clean":"2023-10-06T22:12:45.061995+0000","last_became_active":"2023-10-06T22:12:43.701432+0000","last_became_peered":"2023-10-06T22:12:43.701432+0000","last_unstale":"2023-10-06T22:12:45.061995+0000","last_undegraded":"2023-10-06T22:12:45.061995+0000","last_fullsized":"2023-10-06T22:12:45.061995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:30:37.909775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.751756+0000","last_change":"2023-10-06T22:12:42.693051+0000","last_active":"2023-10-06T22:12:47.751756+0000","last_peered":"2023-10-06T22:12:47.751756+0000","last_clean":"2023-10-06T22:12:47.751756+0000","last_became_active":"2023-10-06T22:12:42.692750+0000","last_became_peered":"2023-10-06T22:12:42.692750+0000","last_unstale":"2023-10-06T22:12:47.751756+0000","last_undegraded":"2023-10-06T22:12:47.751756+0000","last_fullsized":"2023-10-06T22:12:47.751756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:49:58.053826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684522+0000","last_change":"2023-10-06T22:12:43.696511+0000","last_active":"2023-10-06T22:12:44.684522+0000","last_peered":"2023-10-06T22:12:44.684522+0000","last_clean":"2023-10-06T22:12:44.684522+0000","last_became_active":"2023-10-06T22:12:43.696087+0000","last_became_peered":"2023-10-06T22:12:43.696087+0000","last_unstale":"2023-10-06T22:12:44.684522+0000","last_undegraded":"2023-10-06T22:12:44.684522+0000","last_fullsized":"2023-10-06T22:12:44.684522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:12:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.725457+0000","last_change":"2023-10-06T22:12:42.694963+0000","last_active":"2023-10-06T22:12:43.725457+0000","last_peered":"2023-10-06T22:12:43.725457+0000","last_clean":"2023-10-06T22:12:43.725457+0000","last_became_active":"2023-10-06T22:12:42.694517+0000","last_became_peered":"2023-10-06T22:12:42.694517+0000","last_unstale":"2023-10-06T22:12:43.725457+0000","last_undegraded":"2023-10-06T22:12:43.725457+0000","last_fullsized":"2023-10-06T22:12:43.725457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:07:33.404328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061501+0000","last_change":"2023-10-06T22:12:43.698077+0000","last_active":"2023-10-06T22:12:45.061501+0000","last_peered":"2023-10-06T22:12:45.061501+0000","last_clean":"2023-10-06T22:12:45.061501+0000","last_became_active":"2023-10-06T22:12:43.697676+0000","last_became_peered":"2023-10-06T22:12:43.697676+0000","last_unstale":"2023-10-06T22:12:45.061501+0000","last_undegraded":"2023-10-06T22:12:45.061501+0000","last_fullsized":"2023-10-06T22:12:45.061501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:59:56.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.744398+0000","last_change":"2023-10-06T22:12:42.698067+0000","last_active":"2023-10-06T22:12:46.744398+0000","last_peered":"2023-10-06T22:12:46.744398+0000","last_clean":"2023-10-06T22:12:46.744398+0000","last_became_active":"2023-10-06T22:12:42.697671+0000","last_became_peered":"2023-10-06T22:12:42.697671+0000","last_unstale":"2023-10-06T22:12:46.744398+0000","last_undegraded":"2023-10-06T22:12:46.744398+0000","last_fullsized":"2023-10-06T22:12:46.744398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:33.607513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060516+0000","last_change":"2023-10-06T22:12:43.696213+0000","last_active":"2023-10-06T22:12:45.060516+0000","last_peered":"2023-10-06T22:12:45.060516+0000","last_clean":"2023-10-06T22:12:45.060516+0000","last_became_active":"2023-10-06T22:12:43.695925+0000","last_became_peered":"2023-10-06T22:12:43.695925+0000","last_unstale":"2023-10-06T22:12:45.060516+0000","last_undegraded":"2023-10-06T22:12:45.060516+0000","last_fullsized":"2023-10-06T22:12:45.060516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:24.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.751367+0000","last_change":"2023-10-06T22:12:42.687188+0000","last_active":"2023-10-06T22:12:47.751367+0000","last_peered":"2023-10-06T22:12:47.751367+0000","last_clean":"2023-10-06T22:12:47.751367+0000","last_became_active":"2023-10-06T22:12:42.686747+0000","last_became_peered":"2023-10-06T22:12:42.686747+0000","last_unstale":"2023-10-06T22:12:47.751367+0000","last_undegraded":"2023-10-06T22:12:47.751367+0000","last_fullsized":"2023-10-06T22:12:47.751367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:21.326911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061242+0000","last_change":"2023-10-06T22:12:43.697988+0000","last_active":"2023-10-06T22:12:45.061242+0000","last_peered":"2023-10-06T22:12:45.061242+0000","last_clean":"2023-10-06T22:12:45.061242+0000","last_became_active":"2023-10-06T22:12:43.697641+0000","last_became_peered":"2023-10-06T22:12:43.697641+0000","last_unstale":"2023-10-06T22:12:45.061242+0000","last_undegraded":"2023-10-06T22:12:45.061242+0000","last_fullsized":"2023-10-06T22:12:45.061242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:33.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.689932+0000","last_change":"2023-10-06T22:12:42.688882+0000","last_active":"2023-10-06T22:12:43.689932+0000","last_peered":"2023-10-06T22:12:43.689932+0000","last_clean":"2023-10-06T22:12:43.689932+0000","last_became_active":"2023-10-06T22:12:42.687653+0000","last_became_peered":"2023-10-06T22:12:42.687653+0000","last_unstale":"2023-10-06T22:12:43.689932+0000","last_undegraded":"2023-10-06T22:12:43.689932+0000","last_fullsized":"2023-10-06T22:12:43.689932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:18.146081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060491+0000","last_change":"2023-10-06T22:12:43.696559+0000","last_active":"2023-10-06T22:12:45.060491+0000","last_peered":"2023-10-06T22:12:45.060491+0000","last_clean":"2023-10-06T22:12:45.060491+0000","last_became_active":"2023-10-06T22:12:43.696252+0000","last_became_peered":"2023-10-06T22:12:43.696252+0000","last_unstale":"2023-10-06T22:12:45.060491+0000","last_undegraded":"2023-10-06T22:12:45.060491+0000","last_fullsized":"2023-10-06T22:12:45.060491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:31:36.097000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.767500+0000","last_change":"2023-10-06T22:12:42.694299+0000","last_active":"2023-10-06T22:12:48.767500+0000","last_peered":"2023-10-06T22:12:48.767500+0000","last_clean":"2023-10-06T22:12:48.767500+0000","last_became_active":"2023-10-06T22:12:42.693820+0000","last_became_peered":"2023-10-06T22:12:42.693820+0000","last_unstale":"2023-10-06T22:12:48.767500+0000","last_undegraded":"2023-10-06T22:12:48.767500+0000","last_fullsized":"2023-10-06T22:12:48.767500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:13.837379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.686381+0000","last_change":"2023-10-06T22:12:43.695234+0000","last_active":"2023-10-06T22:12:44.686381+0000","last_peered":"2023-10-06T22:12:44.686381+0000","last_clean":"2023-10-06T22:12:44.686381+0000","last_became_active":"2023-10-06T22:12:43.694889+0000","last_became_peered":"2023-10-06T22:12:43.694889+0000","last_unstale":"2023-10-06T22:12:44.686381+0000","last_undegraded":"2023-10-06T22:12:44.686381+0000","last_fullsized":"2023-10-06T22:12:44.686381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:51.039264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.741266+0000","last_change":"2023-10-06T22:12:42.695438+0000","last_active":"2023-10-06T22:12:46.741266+0000","last_peered":"2023-10-06T22:12:46.741266+0000","last_clean":"2023-10-06T22:12:46.741266+0000","last_became_active":"2023-10-06T22:12:42.695090+0000","last_became_peered":"2023-10-06T22:12:42.695090+0000","last_unstale":"2023-10-06T22:12:46.741266+0000","last_undegraded":"2023-10-06T22:12:46.741266+0000","last_fullsized":"2023-10-06T22:12:46.741266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:25:34.385961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684842+0000","last_change":"2023-10-06T22:12:43.697459+0000","last_active":"2023-10-06T22:12:44.684842+0000","last_peered":"2023-10-06T22:12:44.684842+0000","last_clean":"2023-10-06T22:12:44.684842+0000","last_became_active":"2023-10-06T22:12:43.696405+0000","last_became_peered":"2023-10-06T22:12:43.696405+0000","last_unstale":"2023-10-06T22:12:44.684842+0000","last_undegraded":"2023-10-06T22:12:44.684842+0000","last_fullsized":"2023-10-06T22:12:44.684842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:30.928083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.738473+0000","last_change":"2023-10-06T22:12:42.692151+0000","last_active":"2023-10-06T22:12:46.738473+0000","last_peered":"2023-10-06T22:12:46.738473+0000","last_clean":"2023-10-06T22:12:46.738473+0000","last_became_active":"2023-10-06T22:12:42.691780+0000","last_became_peered":"2023-10-06T22:12:42.691780+0000","last_unstale":"2023-10-06T22:12:46.738473+0000","last_undegraded":"2023-10-06T22:12:46.738473+0000","last_fullsized":"2023-10-06T22:12:46.738473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:49:35.697733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061969+0000","last_change":"2023-10-06T22:12:43.701754+0000","last_active":"2023-10-06T22:12:45.061969+0000","last_peered":"2023-10-06T22:12:45.061969+0000","last_clean":"2023-10-06T22:12:45.061969+0000","last_became_active":"2023-10-06T22:12:43.700881+0000","last_became_peered":"2023-10-06T22:12:43.700881+0000","last_unstale":"2023-10-06T22:12:45.061969+0000","last_undegraded":"2023-10-06T22:12:45.061969+0000","last_fullsized":"2023-10-06T22:12:45.061969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:35:21.988307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.742232+0000","last_change":"2023-10-06T22:12:42.699751+0000","last_active":"2023-10-06T22:12:46.742232+0000","last_peered":"2023-10-06T22:12:46.742232+0000","last_clean":"2023-10-06T22:12:46.742232+0000","last_became_active":"2023-10-06T22:12:42.699321+0000","last_became_peered":"2023-10-06T22:12:42.699321+0000","last_unstale":"2023-10-06T22:12:46.742232+0000","last_undegraded":"2023-10-06T22:12:46.742232+0000","last_fullsized":"2023-10-06T22:12:46.742232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:35:25.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684708+0000","last_change":"2023-10-06T22:12:43.702426+0000","last_active":"2023-10-06T22:12:44.684708+0000","last_peered":"2023-10-06T22:12:44.684708+0000","last_clean":"2023-10-06T22:12:44.684708+0000","last_became_active":"2023-10-06T22:12:43.702101+0000","last_became_peered":"2023-10-06T22:12:43.702101+0000","last_unstale":"2023-10-06T22:12:44.684708+0000","last_undegraded":"2023-10-06T22:12:44.684708+0000","last_fullsized":"2023-10-06T22:12:44.684708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:31:09.133941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.740145+0000","last_change":"2023-10-06T22:12:42.688815+0000","last_active":"2023-10-06T22:12:46.740145+0000","last_peered":"2023-10-06T22:12:46.740145+0000","last_clean":"2023-10-06T22:12:46.740145+0000","last_became_active":"2023-10-06T22:12:42.688447+0000","last_became_peered":"2023-10-06T22:12:42.688447+0000","last_unstale":"2023-10-06T22:12:46.740145+0000","last_undegraded":"2023-10-06T22:12:46.740145+0000","last_fullsized":"2023-10-06T22:12:46.740145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:26:25.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060840+0000","last_change":"2023-10-06T22:12:43.698406+0000","last_active":"2023-10-06T22:12:45.060840+0000","last_peered":"2023-10-06T22:12:45.060840+0000","last_clean":"2023-10-06T22:12:45.060840+0000","last_became_active":"2023-10-06T22:12:43.697204+0000","last_became_peered":"2023-10-06T22:12:43.697204+0000","last_unstale":"2023-10-06T22:12:45.060840+0000","last_undegraded":"2023-10-06T22:12:45.060840+0000","last_fullsized":"2023-10-06T22:12:45.060840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:59.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.753271+0000","last_change":"2023-10-06T22:12:42.691007+0000","last_active":"2023-10-06T22:12:47.753271+0000","last_peered":"2023-10-06T22:12:47.753271+0000","last_clean":"2023-10-06T22:12:47.753271+0000","last_became_active":"2023-10-06T22:12:42.690693+0000","last_became_peered":"2023-10-06T22:12:42.690693+0000","last_unstale":"2023-10-06T22:12:47.753271+0000","last_undegraded":"2023-10-06T22:12:47.753271+0000","last_fullsized":"2023-10-06T22:12:47.753271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:47.670091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834687+0000","last_change":"2023-10-06T22:12:43.705131+0000","last_active":"2023-10-06T22:12:44.834687+0000","last_peered":"2023-10-06T22:12:44.834687+0000","last_clean":"2023-10-06T22:12:44.834687+0000","last_became_active":"2023-10-06T22:12:43.704469+0000","last_became_peered":"2023-10-06T22:12:43.704469+0000","last_unstale":"2023-10-06T22:12:44.834687+0000","last_undegraded":"2023-10-06T22:12:44.834687+0000","last_fullsized":"2023-10-06T22:12:44.834687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:36:39.392274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.768075+0000","last_change":"2023-10-06T22:12:42.698392+0000","last_active":"2023-10-06T22:12:48.768075+0000","last_peered":"2023-10-06T22:12:48.768075+0000","last_clean":"2023-10-06T22:12:48.768075+0000","last_became_active":"2023-10-06T22:12:42.698019+0000","last_became_peered":"2023-10-06T22:12:42.698019+0000","last_unstale":"2023-10-06T22:12:48.768075+0000","last_undegraded":"2023-10-06T22:12:48.768075+0000","last_fullsized":"2023-10-06T22:12:48.768075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:00.345300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061318+0000","last_change":"2023-10-06T22:12:43.693238+0000","last_active":"2023-10-06T22:12:45.061318+0000","last_peered":"2023-10-06T22:12:45.061318+0000","last_clean":"2023-10-06T22:12:45.061318+0000","last_became_active":"2023-10-06T22:12:43.692948+0000","last_became_peered":"2023-10-06T22:12:43.692948+0000","last_unstale":"2023-10-06T22:12:45.061318+0000","last_undegraded":"2023-10-06T22:12:45.061318+0000","last_fullsized":"2023-10-06T22:12:45.061318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:24:30.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.749906+0000","last_change":"2023-10-06T22:12:42.693213+0000","last_active":"2023-10-06T22:12:47.749906+0000","last_peered":"2023-10-06T22:12:47.749906+0000","last_clean":"2023-10-06T22:12:47.749906+0000","last_became_active":"2023-10-06T22:12:42.692894+0000","last_became_peered":"2023-10-06T22:12:42.692894+0000","last_unstale":"2023-10-06T22:12:47.749906+0000","last_undegraded":"2023-10-06T22:12:47.749906+0000","last_fullsized":"2023-10-06T22:12:47.749906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:40:21.159102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060258+0000","last_change":"2023-10-06T22:12:43.695527+0000","last_active":"2023-10-06T22:12:45.060258+0000","last_peered":"2023-10-06T22:12:45.060258+0000","last_clean":"2023-10-06T22:12:45.060258+0000","last_became_active":"2023-10-06T22:12:43.694364+0000","last_became_peered":"2023-10-06T22:12:43.694364+0000","last_unstale":"2023-10-06T22:12:45.060258+0000","last_undegraded":"2023-10-06T22:12:45.060258+0000","last_fullsized":"2023-10-06T22:12:45.060258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.754346+0000","last_change":"2023-10-06T22:12:42.697854+0000","last_active":"2023-10-06T22:12:47.754346+0000","last_peered":"2023-10-06T22:12:47.754346+0000","last_clean":"2023-10-06T22:12:47.754346+0000","last_became_active":"2023-10-06T22:12:42.697542+0000","last_became_peered":"2023-10-06T22:12:42.697542+0000","last_unstale":"2023-10-06T22:12:47.754346+0000","last_undegraded":"2023-10-06T22:12:47.754346+0000","last_fullsized":"2023-10-06T22:12:47.754346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:12:11.181398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060021+0000","last_change":"2023-10-06T22:12:43.698070+0000","last_active":"2023-10-06T22:12:45.060021+0000","last_peered":"2023-10-06T22:12:45.060021+0000","last_clean":"2023-10-06T22:12:45.060021+0000","last_became_active":"2023-10-06T22:12:43.697682+0000","last_became_peered":"2023-10-06T22:12:43.697682+0000","last_unstale":"2023-10-06T22:12:45.060021+0000","last_undegraded":"2023-10-06T22:12:45.060021+0000","last_fullsized":"2023-10-06T22:12:45.060021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:43.808866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.728852+0000","last_change":"2023-10-06T22:12:42.691691+0000","last_active":"2023-10-06T22:12:43.728852+0000","last_peered":"2023-10-06T22:12:43.728852+0000","last_clean":"2023-10-06T22:12:43.728852+0000","last_became_active":"2023-10-06T22:12:42.691001+0000","last_became_peered":"2023-10-06T22:12:42.691001+0000","last_unstale":"2023-10-06T22:12:43.728852+0000","last_undegraded":"2023-10-06T22:12:43.728852+0000","last_fullsized":"2023-10-06T22:12:43.728852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:11:18.559570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200656+0000","last_change":"2023-10-06T22:12:43.699240+0000","last_active":"2023-10-06T22:12:45.200656+0000","last_peered":"2023-10-06T22:12:45.200656+0000","last_clean":"2023-10-06T22:12:45.200656+0000","last_became_active":"2023-10-06T22:12:43.698933+0000","last_became_peered":"2023-10-06T22:12:43.698933+0000","last_unstale":"2023-10-06T22:12:45.200656+0000","last_undegraded":"2023-10-06T22:12:45.200656+0000","last_fullsized":"2023-10-06T22:12:45.200656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:18:20.928529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.682435+0000","last_change":"2023-10-06T22:12:42.695650+0000","last_active":"2023-10-06T22:12:43.682435+0000","last_peered":"2023-10-06T22:12:43.682435+0000","last_clean":"2023-10-06T22:12:43.682435+0000","last_became_active":"2023-10-06T22:12:42.695283+0000","last_became_peered":"2023-10-06T22:12:42.695283+0000","last_unstale":"2023-10-06T22:12:43.682435+0000","last_undegraded":"2023-10-06T22:12:43.682435+0000","last_fullsized":"2023-10-06T22:12:43.682435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:17:04.351934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061090+0000","last_change":"2023-10-06T22:12:43.702870+0000","last_active":"2023-10-06T22:12:45.061090+0000","last_peered":"2023-10-06T22:12:45.061090+0000","last_clean":"2023-10-06T22:12:45.061090+0000","last_became_active":"2023-10-06T22:12:43.702564+0000","last_became_peered":"2023-10-06T22:12:43.702564+0000","last_unstale":"2023-10-06T22:12:45.061090+0000","last_undegraded":"2023-10-06T22:12:45.061090+0000","last_fullsized":"2023-10-06T22:12:45.061090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:23:32.238268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061245+0000","last_change":"2023-10-06T22:12:42.696603+0000","last_active":"2023-10-06T22:12:45.061245+0000","last_peered":"2023-10-06T22:12:45.061245+0000","last_clean":"2023-10-06T22:12:45.061245+0000","last_became_active":"2023-10-06T22:12:42.696313+0000","last_became_peered":"2023-10-06T22:12:42.696313+0000","last_unstale":"2023-10-06T22:12:45.061245+0000","last_undegraded":"2023-10-06T22:12:45.061245+0000","last_fullsized":"2023-10-06T22:12:45.061245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:15.719749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060674+0000","last_change":"2023-10-06T22:12:43.703356+0000","last_active":"2023-10-06T22:12:45.060674+0000","last_peered":"2023-10-06T22:12:45.060674+0000","last_clean":"2023-10-06T22:12:45.060674+0000","last_became_active":"2023-10-06T22:12:43.702670+0000","last_became_peered":"2023-10-06T22:12:43.702670+0000","last_unstale":"2023-10-06T22:12:45.060674+0000","last_undegraded":"2023-10-06T22:12:45.060674+0000","last_fullsized":"2023-10-06T22:12:45.060674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:15:05.681491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060063+0000","last_change":"2023-10-06T22:12:43.699083+0000","last_active":"2023-10-06T22:12:45.060063+0000","last_peered":"2023-10-06T22:12:45.060063+0000","last_clean":"2023-10-06T22:12:45.060063+0000","last_became_active":"2023-10-06T22:12:43.698826+0000","last_became_peered":"2023-10-06T22:12:43.698826+0000","last_unstale":"2023-10-06T22:12:45.060063+0000","last_undegraded":"2023-10-06T22:12:45.060063+0000","last_fullsized":"2023-10-06T22:12:45.060063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:04.225060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834521+0000","last_change":"2023-10-06T22:12:42.699873+0000","last_active":"2023-10-06T22:12:44.834521+0000","last_peered":"2023-10-06T22:12:44.834521+0000","last_clean":"2023-10-06T22:12:44.834521+0000","last_became_active":"2023-10-06T22:12:42.699476+0000","last_became_peered":"2023-10-06T22:12:42.699476+0000","last_unstale":"2023-10-06T22:12:44.834521+0000","last_undegraded":"2023-10-06T22:12:44.834521+0000","last_fullsized":"2023-10-06T22:12:44.834521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:13:27.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060891+0000","last_change":"2023-10-06T22:12:43.697720+0000","last_active":"2023-10-06T22:12:45.060891+0000","last_peered":"2023-10-06T22:12:45.060891+0000","last_clean":"2023-10-06T22:12:45.060891+0000","last_became_active":"2023-10-06T22:12:43.697430+0000","last_became_peered":"2023-10-06T22:12:43.697430+0000","last_unstale":"2023-10-06T22:12:45.060891+0000","last_undegraded":"2023-10-06T22:12:45.060891+0000","last_fullsized":"2023-10-06T22:12:45.060891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:29.633012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.740844+0000","last_change":"2023-10-06T22:12:42.690781+0000","last_active":"2023-10-06T22:12:46.740844+0000","last_peered":"2023-10-06T22:12:46.740844+0000","last_clean":"2023-10-06T22:12:46.740844+0000","last_became_active":"2023-10-06T22:12:42.690274+0000","last_became_peered":"2023-10-06T22:12:42.690274+0000","last_unstale":"2023-10-06T22:12:46.740844+0000","last_undegraded":"2023-10-06T22:12:46.740844+0000","last_fullsized":"2023-10-06T22:12:46.740844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:26.653182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.693551+0000","last_change":"2023-10-06T22:11:36.409669+0000","last_active":"2023-10-06T22:12:43.693551+0000","last_peered":"2023-10-06T22:12:43.693551+0000","last_clean":"2023-10-06T22:12:43.693551+0000","last_became_active":"2023-10-06T22:11:36.409197+0000","last_became_peered":"2023-10-06T22:11:36.409197+0000","last_unstale":"2023-10-06T22:12:43.693551+0000","last_undegraded":"2023-10-06T22:12:43.693551+0000","last_fullsized":"2023-10-06T22:12:43.693551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_clean_scrub_stamp":"2023-10-06T22:08:24.598743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:50:55.982571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834557+0000","last_change":"2023-10-06T22:12:43.698617+0000","last_active":"2023-10-06T22:12:44.834557+0000","last_peered":"2023-10-06T22:12:44.834557+0000","last_clean":"2023-10-06T22:12:44.834557+0000","last_became_active":"2023-10-06T22:12:43.697942+0000","last_became_peered":"2023-10-06T22:12:43.697942+0000","last_unstale":"2023-10-06T22:12:44.834557+0000","last_undegraded":"2023-10-06T22:12:44.834557+0000","last_fullsized":"2023-10-06T22:12:44.834557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:38.075730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.750582+0000","last_change":"2023-10-06T22:12:42.695553+0000","last_active":"2023-10-06T22:12:47.750582+0000","last_peered":"2023-10-06T22:12:47.750582+0000","last_clean":"2023-10-06T22:12:47.750582+0000","last_became_active":"2023-10-06T22:12:42.695209+0000","last_became_peered":"2023-10-06T22:12:42.695209+0000","last_unstale":"2023-10-06T22:12:47.750582+0000","last_undegraded":"2023-10-06T22:12:47.750582+0000","last_fullsized":"2023-10-06T22:12:47.750582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:10.387710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060959+0000","last_change":"2023-10-06T22:12:42.699606+0000","last_active":"2023-10-06T22:12:45.060959+0000","last_peered":"2023-10-06T22:12:45.060959+0000","last_clean":"2023-10-06T22:12:45.060959+0000","last_became_active":"2023-10-06T22:12:42.699222+0000","last_became_peered":"2023-10-06T22:12:42.699222+0000","last_unstale":"2023-10-06T22:12:45.060959+0000","last_undegraded":"2023-10-06T22:12:45.060959+0000","last_fullsized":"2023-10-06T22:12:45.060959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:52.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834957+0000","last_change":"2023-10-06T22:12:43.697783+0000","last_active":"2023-10-06T22:12:44.834957+0000","last_peered":"2023-10-06T22:12:44.834957+0000","last_clean":"2023-10-06T22:12:44.834957+0000","last_became_active":"2023-10-06T22:12:43.697388+0000","last_became_peered":"2023-10-06T22:12:43.697388+0000","last_unstale":"2023-10-06T22:12:44.834957+0000","last_undegraded":"2023-10-06T22:12:44.834957+0000","last_fullsized":"2023-10-06T22:12:44.834957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:50.113458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.682676+0000","last_change":"2023-10-06T22:12:42.694180+0000","last_active":"2023-10-06T22:12:43.682676+0000","last_peered":"2023-10-06T22:12:43.682676+0000","last_clean":"2023-10-06T22:12:43.682676+0000","last_became_active":"2023-10-06T22:12:42.693861+0000","last_became_peered":"2023-10-06T22:12:42.693861+0000","last_unstale":"2023-10-06T22:12:43.682676+0000","last_undegraded":"2023-10-06T22:12:43.682676+0000","last_fullsized":"2023-10-06T22:12:43.682676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:28:02.511117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061448+0000","last_change":"2023-10-06T22:12:43.699118+0000","last_active":"2023-10-06T22:12:45.061448+0000","last_peered":"2023-10-06T22:12:45.061448+0000","last_clean":"2023-10-06T22:12:45.061448+0000","last_became_active":"2023-10-06T22:12:43.698540+0000","last_became_peered":"2023-10-06T22:12:43.698540+0000","last_unstale":"2023-10-06T22:12:45.061448+0000","last_undegraded":"2023-10-06T22:12:45.061448+0000","last_fullsized":"2023-10-06T22:12:45.061448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:31.192762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.739401+0000","last_change":"2023-10-06T22:12:42.690617+0000","last_active":"2023-10-06T22:12:46.739401+0000","last_peered":"2023-10-06T22:12:46.739401+0000","last_clean":"2023-10-06T22:12:46.739401+0000","last_became_active":"2023-10-06T22:12:42.690238+0000","last_became_peered":"2023-10-06T22:12:42.690238+0000","last_unstale":"2023-10-06T22:12:46.739401+0000","last_undegraded":"2023-10-06T22:12:46.739401+0000","last_fullsized":"2023-10-06T22:12:46.739401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:09.819570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061674+0000","last_change":"2023-10-06T22:12:43.699064+0000","last_active":"2023-10-06T22:12:45.061674+0000","last_peered":"2023-10-06T22:12:45.061674+0000","last_clean":"2023-10-06T22:12:45.061674+0000","last_became_active":"2023-10-06T22:12:43.698727+0000","last_became_peered":"2023-10-06T22:12:43.698727+0000","last_unstale":"2023-10-06T22:12:45.061674+0000","last_undegraded":"2023-10-06T22:12:45.061674+0000","last_fullsized":"2023-10-06T22:12:45.061674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:48:14.493376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.739139+0000","last_change":"2023-10-06T22:12:42.693779+0000","last_active":"2023-10-06T22:12:46.739139+0000","last_peered":"2023-10-06T22:12:46.739139+0000","last_clean":"2023-10-06T22:12:46.739139+0000","last_became_active":"2023-10-06T22:12:42.693257+0000","last_became_peered":"2023-10-06T22:12:42.693257+0000","last_unstale":"2023-10-06T22:12:46.739139+0000","last_undegraded":"2023-10-06T22:12:46.739139+0000","last_fullsized":"2023-10-06T22:12:46.739139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:56:48.945078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060316+0000","last_change":"2023-10-06T22:12:43.700620+0000","last_active":"2023-10-06T22:12:45.060316+0000","last_peered":"2023-10-06T22:12:45.060316+0000","last_clean":"2023-10-06T22:12:45.060316+0000","last_became_active":"2023-10-06T22:12:43.700093+0000","last_became_peered":"2023-10-06T22:12:43.700093+0000","last_unstale":"2023-10-06T22:12:45.060316+0000","last_undegraded":"2023-10-06T22:12:45.060316+0000","last_fullsized":"2023-10-06T22:12:45.060316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:29.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.684106+0000","last_change":"2023-10-06T22:12:42.689640+0000","last_active":"2023-10-06T22:12:43.684106+0000","last_peered":"2023-10-06T22:12:43.684106+0000","last_clean":"2023-10-06T22:12:43.684106+0000","last_became_active":"2023-10-06T22:12:42.688584+0000","last_became_peered":"2023-10-06T22:12:42.688584+0000","last_unstale":"2023-10-06T22:12:43.684106+0000","last_undegraded":"2023-10-06T22:12:43.684106+0000","last_fullsized":"2023-10-06T22:12:43.684106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:43.066396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200483+0000","last_change":"2023-10-06T22:12:43.700084+0000","last_active":"2023-10-06T22:12:45.200483+0000","last_peered":"2023-10-06T22:12:45.200483+0000","last_clean":"2023-10-06T22:12:45.200483+0000","last_became_active":"2023-10-06T22:12:43.699787+0000","last_became_peered":"2023-10-06T22:12:43.699787+0000","last_unstale":"2023-10-06T22:12:45.200483+0000","last_undegraded":"2023-10-06T22:12:45.200483+0000","last_fullsized":"2023-10-06T22:12:45.200483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:07.219800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.357090+0000","last_change":"2023-10-06T22:12:42.688750+0000","last_active":"2023-10-06T22:12:48.357090+0000","last_peered":"2023-10-06T22:12:48.357090+0000","last_clean":"2023-10-06T22:12:48.357090+0000","last_became_active":"2023-10-06T22:12:42.688387+0000","last_became_peered":"2023-10-06T22:12:42.688387+0000","last_unstale":"2023-10-06T22:12:48.357090+0000","last_undegraded":"2023-10-06T22:12:48.357090+0000","last_fullsized":"2023-10-06T22:12:48.357090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:32.980631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061298+0000","last_change":"2023-10-06T22:12:43.706741+0000","last_active":"2023-10-06T22:12:45.061298+0000","last_peered":"2023-10-06T22:12:45.061298+0000","last_clean":"2023-10-06T22:12:45.061298+0000","last_became_active":"2023-10-06T22:12:43.706416+0000","last_became_peered":"2023-10-06T22:12:43.706416+0000","last_unstale":"2023-10-06T22:12:45.061298+0000","last_undegraded":"2023-10-06T22:12:45.061298+0000","last_fullsized":"2023-10-06T22:12:45.061298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:55:16.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.739590+0000","last_change":"2023-10-06T22:12:42.692850+0000","last_active":"2023-10-06T22:12:46.739590+0000","last_peered":"2023-10-06T22:12:46.739590+0000","last_clean":"2023-10-06T22:12:46.739590+0000","last_became_active":"2023-10-06T22:12:42.692287+0000","last_became_peered":"2023-10-06T22:12:42.692287+0000","last_unstale":"2023-10-06T22:12:46.739590+0000","last_undegraded":"2023-10-06T22:12:46.739590+0000","last_fullsized":"2023-10-06T22:12:46.739590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:37:59.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061394+0000","last_change":"2023-10-06T22:12:43.700667+0000","last_active":"2023-10-06T22:12:45.061394+0000","last_peered":"2023-10-06T22:12:45.061394+0000","last_clean":"2023-10-06T22:12:45.061394+0000","last_became_active":"2023-10-06T22:12:43.698758+0000","last_became_peered":"2023-10-06T22:12:43.698758+0000","last_unstale":"2023-10-06T22:12:45.061394+0000","last_undegraded":"2023-10-06T22:12:45.061394+0000","last_fullsized":"2023-10-06T22:12:45.061394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:49:53.091578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.768637+0000","last_change":"2023-10-06T22:12:42.695964+0000","last_active":"2023-10-06T22:12:48.768637+0000","last_peered":"2023-10-06T22:12:48.768637+0000","last_clean":"2023-10-06T22:12:48.768637+0000","last_became_active":"2023-10-06T22:12:42.695431+0000","last_became_peered":"2023-10-06T22:12:42.695431+0000","last_unstale":"2023-10-06T22:12:48.768637+0000","last_undegraded":"2023-10-06T22:12:48.768637+0000","last_fullsized":"2023-10-06T22:12:48.768637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:21:42.761421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061634+0000","last_change":"2023-10-06T22:12:43.698088+0000","last_active":"2023-10-06T22:12:45.061634+0000","last_peered":"2023-10-06T22:12:45.061634+0000","last_clean":"2023-10-06T22:12:45.061634+0000","last_became_active":"2023-10-06T22:12:43.697765+0000","last_became_peered":"2023-10-06T22:12:43.697765+0000","last_unstale":"2023-10-06T22:12:45.061634+0000","last_undegraded":"2023-10-06T22:12:45.061634+0000","last_fullsized":"2023-10-06T22:12:45.061634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:26:22.958348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.767937+0000","last_change":"2023-10-06T22:12:42.698586+0000","last_active":"2023-10-06T22:12:48.767937+0000","last_peered":"2023-10-06T22:12:48.767937+0000","last_clean":"2023-10-06T22:12:48.767937+0000","last_became_active":"2023-10-06T22:12:42.698198+0000","last_became_peered":"2023-10-06T22:12:42.698198+0000","last_unstale":"2023-10-06T22:12:48.767937+0000","last_undegraded":"2023-10-06T22:12:48.767937+0000","last_fullsized":"2023-10-06T22:12:48.767937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:03:43.898987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060873+0000","last_change":"2023-10-06T22:12:43.698615+0000","last_active":"2023-10-06T22:12:45.060873+0000","last_peered":"2023-10-06T22:12:45.060873+0000","last_clean":"2023-10-06T22:12:45.060873+0000","last_became_active":"2023-10-06T22:12:43.698302+0000","last_became_peered":"2023-10-06T22:12:43.698302+0000","last_unstale":"2023-10-06T22:12:45.060873+0000","last_undegraded":"2023-10-06T22:12:45.060873+0000","last_fullsized":"2023-10-06T22:12:45.060873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:35.973119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.725187+0000","last_change":"2023-10-06T22:12:42.700325+0000","last_active":"2023-10-06T22:12:43.725187+0000","last_peered":"2023-10-06T22:12:43.725187+0000","last_clean":"2023-10-06T22:12:43.725187+0000","last_became_active":"2023-10-06T22:12:42.699426+0000","last_became_peered":"2023-10-06T22:12:42.699426+0000","last_unstale":"2023-10-06T22:12:43.725187+0000","last_undegraded":"2023-10-06T22:12:43.725187+0000","last_fullsized":"2023-10-06T22:12:43.725187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:09.649737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061763+0000","last_change":"2023-10-06T22:12:43.698620+0000","last_active":"2023-10-06T22:12:45.061763+0000","last_peered":"2023-10-06T22:12:45.061763+0000","last_clean":"2023-10-06T22:12:45.061763+0000","last_became_active":"2023-10-06T22:12:43.698335+0000","last_became_peered":"2023-10-06T22:12:43.698335+0000","last_unstale":"2023-10-06T22:12:45.061763+0000","last_undegraded":"2023-10-06T22:12:45.061763+0000","last_fullsized":"2023-10-06T22:12:45.061763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:49.222848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200485+0000","last_change":"2023-10-06T22:12:43.700477+0000","last_active":"2023-10-06T22:12:45.200485+0000","last_peered":"2023-10-06T22:12:45.200485+0000","last_clean":"2023-10-06T22:12:45.200485+0000","last_became_active":"2023-10-06T22:12:43.700177+0000","last_became_peered":"2023-10-06T22:12:43.700177+0000","last_unstale":"2023-10-06T22:12:45.200485+0000","last_undegraded":"2023-10-06T22:12:45.200485+0000","last_fullsized":"2023-10-06T22:12:45.200485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:11.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684636+0000","last_change":"2023-10-06T22:12:43.701792+0000","last_active":"2023-10-06T22:12:44.684636+0000","last_peered":"2023-10-06T22:12:44.684636+0000","last_clean":"2023-10-06T22:12:44.684636+0000","last_became_active":"2023-10-06T22:12:43.701316+0000","last_became_peered":"2023-10-06T22:12:43.701316+0000","last_unstale":"2023-10-06T22:12:44.684636+0000","last_undegraded":"2023-10-06T22:12:44.684636+0000","last_fullsized":"2023-10-06T22:12:44.684636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:03.085219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684558+0000","last_change":"2023-10-06T22:12:43.707871+0000","last_active":"2023-10-06T22:12:44.684558+0000","last_peered":"2023-10-06T22:12:44.684558+0000","last_clean":"2023-10-06T22:12:44.684558+0000","last_became_active":"2023-10-06T22:12:43.707659+0000","last_became_peered":"2023-10-06T22:12:43.707659+0000","last_unstale":"2023-10-06T22:12:44.684558+0000","last_undegraded":"2023-10-06T22:12:44.684558+0000","last_fullsized":"2023-10-06T22:12:44.684558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:43.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061483+0000","last_change":"2023-10-06T22:12:43.697701+0000","last_active":"2023-10-06T22:12:45.061483+0000","last_peered":"2023-10-06T22:12:45.061483+0000","last_clean":"2023-10-06T22:12:45.061483+0000","last_became_active":"2023-10-06T22:12:43.697361+0000","last_became_peered":"2023-10-06T22:12:43.697361+0000","last_unstale":"2023-10-06T22:12:45.061483+0000","last_undegraded":"2023-10-06T22:12:45.061483+0000","last_fullsized":"2023-10-06T22:12:45.061483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:14:53.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834822+0000","last_change":"2023-10-06T22:12:43.700058+0000","last_active":"2023-10-06T22:12:44.834822+0000","last_peered":"2023-10-06T22:12:44.834822+0000","last_clean":"2023-10-06T22:12:44.834822+0000","last_became_active":"2023-10-06T22:12:43.697893+0000","last_became_peered":"2023-10-06T22:12:43.697893+0000","last_unstale":"2023-10-06T22:12:44.834822+0000","last_undegraded":"2023-10-06T22:12:44.834822+0000","last_fullsized":"2023-10-06T22:12:44.834822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:01:37.518096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200632+0000","last_change":"2023-10-06T22:12:43.702946+0000","last_active":"2023-10-06T22:12:45.200632+0000","last_peered":"2023-10-06T22:12:45.200632+0000","last_clean":"2023-10-06T22:12:45.200632+0000","last_became_active":"2023-10-06T22:12:43.702628+0000","last_became_peered":"2023-10-06T22:12:43.702628+0000","last_unstale":"2023-10-06T22:12:45.200632+0000","last_undegraded":"2023-10-06T22:12:45.200632+0000","last_fullsized":"2023-10-06T22:12:45.200632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:17.040874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.685426+0000","last_change":"2023-10-06T22:12:43.700668+0000","last_active":"2023-10-06T22:12:44.685426+0000","last_peered":"2023-10-06T22:12:44.685426+0000","last_clean":"2023-10-06T22:12:44.685426+0000","last_became_active":"2023-10-06T22:12:43.700387+0000","last_became_peered":"2023-10-06T22:12:43.700387+0000","last_unstale":"2023-10-06T22:12:44.685426+0000","last_undegraded":"2023-10-06T22:12:44.685426+0000","last_fullsized":"2023-10-06T22:12:44.685426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:36:00.053496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061280+0000","last_change":"2023-10-06T22:12:43.702658+0000","last_active":"2023-10-06T22:12:45.061280+0000","last_peered":"2023-10-06T22:12:45.061280+0000","last_clean":"2023-10-06T22:12:45.061280+0000","last_became_active":"2023-10-06T22:12:43.702264+0000","last_became_peered":"2023-10-06T22:12:43.702264+0000","last_unstale":"2023-10-06T22:12:45.061280+0000","last_undegraded":"2023-10-06T22:12:45.061280+0000","last_fullsized":"2023-10-06T22:12:45.061280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:56.472029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.059936+0000","last_change":"2023-10-06T22:12:43.705718+0000","last_active":"2023-10-06T22:12:45.059936+0000","last_peered":"2023-10-06T22:12:45.059936+0000","last_clean":"2023-10-06T22:12:45.059936+0000","last_became_active":"2023-10-06T22:12:43.705479+0000","last_became_peered":"2023-10-06T22:12:43.705479+0000","last_unstale":"2023-10-06T22:12:45.059936+0000","last_undegraded":"2023-10-06T22:12:45.059936+0000","last_fullsized":"2023-10-06T22:12:45.059936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:55:36.393577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060870+0000","last_change":"2023-10-06T22:12:43.701891+0000","last_active":"2023-10-06T22:12:45.060870+0000","last_peered":"2023-10-06T22:12:45.060870+0000","last_clean":"2023-10-06T22:12:45.060870+0000","last_became_active":"2023-10-06T22:12:43.701535+0000","last_became_peered":"2023-10-06T22:12:43.701535+0000","last_unstale":"2023-10-06T22:12:45.060870+0000","last_undegraded":"2023-10-06T22:12:45.060870+0000","last_fullsized":"2023-10-06T22:12:45.060870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:17:40.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684533+0000","last_change":"2023-10-06T22:12:43.707075+0000","last_active":"2023-10-06T22:12:44.684533+0000","last_peered":"2023-10-06T22:12:44.684533+0000","last_clean":"2023-10-06T22:12:44.684533+0000","last_became_active":"2023-10-06T22:12:43.706723+0000","last_became_peered":"2023-10-06T22:12:43.706723+0000","last_unstale":"2023-10-06T22:12:44.684533+0000","last_undegraded":"2023-10-06T22:12:44.684533+0000","last_fullsized":"2023-10-06T22:12:44.684533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:18.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834964+0000","last_change":"2023-10-06T22:12:43.704656+0000","last_active":"2023-10-06T22:12:44.834964+0000","last_peered":"2023-10-06T22:12:44.834964+0000","last_clean":"2023-10-06T22:12:44.834964+0000","last_became_active":"2023-10-06T22:12:43.704312+0000","last_became_peered":"2023-10-06T22:12:43.704312+0000","last_unstale":"2023-10-06T22:12:44.834964+0000","last_undegraded":"2023-10-06T22:12:44.834964+0000","last_fullsized":"2023-10-06T22:12:44.834964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:21:37.951180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200727+0000","last_change":"2023-10-06T22:12:43.702239+0000","last_active":"2023-10-06T22:12:45.200727+0000","last_peered":"2023-10-06T22:12:45.200727+0000","last_clean":"2023-10-06T22:12:45.200727+0000","last_became_active":"2023-10-06T22:12:43.701851+0000","last_became_peered":"2023-10-06T22:12:43.701851+0000","last_unstale":"2023-10-06T22:12:45.200727+0000","last_undegraded":"2023-10-06T22:12:45.200727+0000","last_fullsized":"2023-10-06T22:12:45.200727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:40:53.661587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":18404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":427400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1019045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.80400000000000005}]},{"osd":6,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.376},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.505}]},{"osd":8,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":426400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.497},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.81799999999999995}]},{"osd":8,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.91700000000000004}]},{"osd":10,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.097},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.96799999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103195,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":426225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.30199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40699999999999997}]},{"osd":3,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.623,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.629},{"interface":"front","average":{"1min":0.755,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.50900000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66200000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.75600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1017705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.42599999999999999},"min":{"1min":0.27300000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.437,"15min":0.435},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.441},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68999999999999995,"5min":0.875,"15min":0.875},"last":0.34200000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80600000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60799999999999998}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.625}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70899999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":425545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46999999999999997,"15min":0.45800000000000002},"min":{"1min":0.38200000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82199999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.55100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.54600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.52900000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.379},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.54700000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.55900000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.075,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.006,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.60499999999999998,"5min":0.499,"15min":0.499},"max":{"1min":1.0740000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.63700000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0840000000000001,"5min":1.173,"15min":1.173},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84399999999999997,"15min":0.84299999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0269999999999999,"5min":1.145,"15min":1.145},"last":0.74199999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.873},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.94299999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":427430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.627},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86099999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.39700000000000002,"15min":0.38100000000000001},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.434,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.753}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77600000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89100000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.90600000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":425304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57299999999999995,"15min":0.52300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.55100000000000005,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Oct 6 22:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 22:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82999999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.46899999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.55000000000000004,"5min":0.376,"15min":0.376},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.86899999999999999},"min":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.121,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.103,"15min":1.103},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.749},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.90900000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.875},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":429071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.57099999999999995},"min":{"1min":0.42899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52600000000000002,"15min":0.498},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.53900000000000003,"15min":0.495},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81899999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Oct 6 22:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.433,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.46200000000000002,"15min":0.45000000000000001},"min":{"1min":0.32200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72399999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":1.002,"15min":1.002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60199999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.754}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":425649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41299999999999998,"15min":0.38400000000000001},"min":{"1min":0.35499999999999998,"5min":0.221,"15min":0.221},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.433,"15min":0.39900000000000002},"min":{"1min":0.38700000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.63700000000000001,"15min":0.59099999999999997},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.63300000000000001,"15min":0.59799999999999998},"min":{"1min":0.495,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.92700000000000005}]},{"osd":2,"last update":"Fri Oct 6 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94699999999999995,"5min":1.046,"15min":1.046},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.67700000000000005,"15min":0.65000000000000002},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0700000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.67300000000000004,"15min":0.63300000000000001},"min":{"1min":0.64900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65000000000000002,"15min":0.60799999999999998},"min":{"1min":0.61699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.64200000000000002},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.95699999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.746},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95499999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.54500000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.60799999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.1180000000000001,"15min":1.155},"min":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0649999999999999,"5min":7.0529999999999999,"15min":7.0529999999999999},"last":0.93300000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.83899999999999997}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.87},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.96599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":425840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.443},{"interface":"front","average":{"1min":0.432,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80500000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.45500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.628}]},{"osd":5,"last update":"Fri Oct 6 22:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.54500000000000004},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.53700000000000003},"min":{"1min":0.49399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57599999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.627,"15min":0.61199999999999999},"min":{"1min":0.52700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1,"5min":1,"15min":1},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Oct 6 22:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.52800000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.58899999999999997}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.74099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757290,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1018424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.39800000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.56399999999999995}]},{"osd":3,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.60199999999999998,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 22:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.755},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.50800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.437,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.7440000000000002,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.81100000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89900000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.77600000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.87}]},{"osd":9,"last update":"Fri Oct 6 22:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T22:12:52.771 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T22:12:53.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 -- 172.21.15.57:0/1374902635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 msgr2=0x7f290c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:53.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 --2- 172.21.15.57:0/1374902635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 0x7f290c1012c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0098f0 tx=0x7f28fc009c00 comp rx=0 tx=0).stop 2023-10-06T22:12:53.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 -- 172.21.15.57:0/1374902635 shutdown_connections 2023-10-06T22:12:53.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 --2- 172.21.15.57:0/1374902635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f290c102a40 0x7f290c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 --2- 172.21.15.57:0/1374902635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 0x7f290c1012c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 --2- 172.21.15.57:0/1374902635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102080 0x7f290c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 -- 172.21.15.57:0/1374902635 >> 172.21.15.57:0/1374902635 conn(0x7f290c0fc420 msgr2=0x7f290c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:53.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 -- 172.21.15.57:0/1374902635 shutdown_connections 2023-10-06T22:12:53.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.003+0000 7f29121ff700 1 -- 172.21.15.57:0/1374902635 wait complete. 2023-10-06T22:12:53.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.004+0000 7f29121ff700 1 Processor -- start 2023-10-06T22:12:53.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f29121ff700 1 -- start start 2023-10-06T22:12:53.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f29121ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 0x7f290c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:53.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f29121ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f290c102080 0x7f290c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:53.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f29121ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102a40 0x7f290c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:53.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f29121ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c1a0a10 con 0x7f290c102a40 2023-10-06T22:12:53.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f29121ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c1a0b50 con 0x7f290c100ea0 2023-10-06T22:12:53.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f29121ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c1a0ce0 con 0x7f290c102080 2023-10-06T22:12:53.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102a40 0x7f290c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102a40 0x7f290c1a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54542/0 (socket says 172.21.15.57:54542) 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 -- 172.21.15.57:0/3789786603 learned_addr learned my addr 172.21.15.57:0/3789786603 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290b7fe700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 0x7f290c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290affd700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f290c102080 0x7f290c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 -- 172.21.15.57:0/3789786603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f290c102080 msgr2=0x7f290c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f290c102080 0x7f290c1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 -- 172.21.15.57:0/3789786603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 msgr2=0x7f290c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:53.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 0x7f290c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290bfff700 1 -- 172.21.15.57:0/3789786603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc009520 con 0x7f290c102a40 2023-10-06T22:12:53.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.005+0000 7f290affd700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f290c102080 0x7f290c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:12:53.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.006+0000 7f290bfff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102a40 0x7f290c1a0440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f290400ef30 tx=0x7f290400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:53.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.006+0000 7f2908ff9700 1 -- 172.21.15.57:0/3789786603 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904018070 con 0x7f290c102a40 2023-10-06T22:12:53.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.006+0000 7f2908ff9700 1 -- 172.21.15.57:0/3789786603 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f290400f040 con 0x7f290c102a40 2023-10-06T22:12:53.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.006+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c1a0f60 con 0x7f290c102a40 2023-10-06T22:12:53.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.006+0000 7f2908ff9700 1 -- 172.21.15.57:0/3789786603 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400cdb0 con 0x7f290c102a40 2023-10-06T22:12:53.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.006+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c1a1390 con 0x7f290c102a40 2023-10-06T22:12:53.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.007+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c10c760 con 0x7f290c102a40 2023-10-06T22:12:53.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.009+0000 7f2908ff9700 1 -- 172.21.15.57:0/3789786603 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2904003a00 con 0x7f290c102a40 2023-10-06T22:12:53.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.010+0000 7f2908ff9700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28f40a1550 0x7f28f40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:53.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.011+0000 7f2908ff9700 1 -- 172.21.15.57:0/3789786603 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f29040cc9b0 con 0x7f290c102a40 2023-10-06T22:12:53.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.011+0000 7f290b7fe700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28f40a1550 0x7f28f40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:53.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.011+0000 7f290b7fe700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28f40a1550 0x7f28f40a3a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009fd0 tx=0x7f28fc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:53.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.015+0000 7f2908ff9700 1 -- 172.21.15.57:0/3789786603 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2904095ff0 con 0x7f290c102a40 2023-10-06T22:12:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:52 smithi190 ceph-mon[134891]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-06T22:12:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:52 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1531224492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:52 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1200646579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:53.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:52 smithi190 ceph-mon[134891]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:12:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:52 smithi057 ceph-mon[98492]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-06T22:12:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:52 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1531224492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:52 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1200646579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:52 smithi057 ceph-mon[98492]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:12:53.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.224+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f290c108b00 con 0x7f28f40a1550 2023-10-06T22:12:53.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.235+0000 7f2908ff9700 1 -- 172.21.15.57:0/3789786603 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336191 (secure 0 0 0) 0x7f290c108b00 con 0x7f28f40a1550 2023-10-06T22:12:53.242 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.239+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28f40a1550 msgr2=0x7f28f40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.239+0000 7f29121ff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28f40a1550 0x7f28f40a3a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009fd0 tx=0x7f28fc019040 comp rx=0 tx=0).stop 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.239+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102a40 msgr2=0x7f290c1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.239+0000 7f29121ff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102a40 0x7f290c1a0440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f290400ef30 tx=0x7f290400c3a0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 shutdown_connections 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28f40a1550 0x7f28f40a3a10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f290c102a40 0x7f290c1a0440 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f290c100ea0 0x7f290c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 --2- 172.21.15.57:0/3789786603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f290c102080 0x7f290c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 >> 172.21.15.57:0/3789786603 conn(0x7f290c0fc420 msgr2=0x7f290c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 shutdown_connections 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:53.240+0000 7f29121ff700 1 -- 172.21.15.57:0/3789786603 wait complete. 2023-10-06T22:12:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-06T22:12:53.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:52 smithi156 ceph-mon[134358]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-06T22:12:53.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:52 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1531224492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:12:53.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:52 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1200646579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:12:53.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:52 smithi156 ceph-mon[134358]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:12:53.261 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2023-10-06T22:12:51.614019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597094,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":292,"num_write_kb":2985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336556,"kb_used_data":12332,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605268,"statfs":{"total":1151940427776,"available":1151595794432,"internally_reserved":0,"allocated":12627968,"data_stored":6894038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002060"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200666+0000","last_change":"2023-10-06T22:12:43.704484+0000","last_active":"2023-10-06T22:12:45.200666+0000","last_peered":"2023-10-06T22:12:45.200666+0000","last_clean":"2023-10-06T22:12:45.200666+0000","last_became_active":"2023-10-06T22:12:43.703740+0000","last_became_peered":"2023-10-06T22:12:43.703740+0000","last_unstale":"2023-10-06T22:12:45.200666+0000","last_undegraded":"2023-10-06T22:12:45.200666+0000","last_fullsized":"2023-10-06T22:12:45.200666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:25.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061532+0000","last_change":"2023-10-06T22:12:43.696968+0000","last_active":"2023-10-06T22:12:45.061532+0000","last_peered":"2023-10-06T22:12:45.061532+0000","last_clean":"2023-10-06T22:12:45.061532+0000","last_became_active":"2023-10-06T22:12:43.696532+0000","last_became_peered":"2023-10-06T22:12:43.696532+0000","last_unstale":"2023-10-06T22:12:45.061532+0000","last_undegraded":"2023-10-06T22:12:45.061532+0000","last_fullsized":"2023-10-06T22:12:45.061532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:59:50.754606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061851+0000","last_change":"2023-10-06T22:12:43.700845+0000","last_active":"2023-10-06T22:12:45.061851+0000","last_peered":"2023-10-06T22:12:45.061851+0000","last_clean":"2023-10-06T22:12:45.061851+0000","last_became_active":"2023-10-06T22:12:43.700531+0000","last_became_peered":"2023-10-06T22:12:43.700531+0000","last_unstale":"2023-10-06T22:12:45.061851+0000","last_undegraded":"2023-10-06T22:12:45.061851+0000","last_fullsized":"2023-10-06T22:12:45.061851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:18.518772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061912+0000","last_change":"2023-10-06T22:12:43.700806+0000","last_active":"2023-10-06T22:12:45.061912+0000","last_peered":"2023-10-06T22:12:45.061912+0000","last_clean":"2023-10-06T22:12:45.061912+0000","last_became_active":"2023-10-06T22:12:43.699901+0000","last_became_peered":"2023-10-06T22:12:43.699901+0000","last_unstale":"2023-10-06T22:12:45.061912+0000","last_undegraded":"2023-10-06T22:12:45.061912+0000","last_fullsized":"2023-10-06T22:12:45.061912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:41:35.906415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060439+0000","last_change":"2023-10-06T22:12:43.694776+0000","last_active":"2023-10-06T22:12:45.060439+0000","last_peered":"2023-10-06T22:12:45.060439+0000","last_clean":"2023-10-06T22:12:45.060439+0000","last_became_active":"2023-10-06T22:12:43.694397+0000","last_became_peered":"2023-10-06T22:12:43.694397+0000","last_unstale":"2023-10-06T22:12:45.060439+0000","last_undegraded":"2023-10-06T22:12:45.060439+0000","last_fullsized":"2023-10-06T22:12:45.060439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:47:43.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060558+0000","last_change":"2023-10-06T22:12:43.703515+0000","last_active":"2023-10-06T22:12:45.060558+0000","last_peered":"2023-10-06T22:12:45.060558+0000","last_clean":"2023-10-06T22:12:45.060558+0000","last_became_active":"2023-10-06T22:12:43.703094+0000","last_became_peered":"2023-10-06T22:12:43.703094+0000","last_unstale":"2023-10-06T22:12:45.060558+0000","last_undegraded":"2023-10-06T22:12:45.060558+0000","last_fullsized":"2023-10-06T22:12:45.060558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:54:32.130131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060949+0000","last_change":"2023-10-06T22:12:43.697894+0000","last_active":"2023-10-06T22:12:45.060949+0000","last_peered":"2023-10-06T22:12:45.060949+0000","last_clean":"2023-10-06T22:12:45.060949+0000","last_became_active":"2023-10-06T22:12:43.697591+0000","last_became_peered":"2023-10-06T22:12:43.697591+0000","last_unstale":"2023-10-06T22:12:45.060949+0000","last_undegraded":"2023-10-06T22:12:45.060949+0000","last_fullsized":"2023-10-06T22:12:45.060949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:30.313447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060532+0000","last_change":"2023-10-06T22:12:43.696555+0000","last_active":"2023-10-06T22:12:45.060532+0000","last_peered":"2023-10-06T22:12:45.060532+0000","last_clean":"2023-10-06T22:12:45.060532+0000","last_became_active":"2023-10-06T22:12:43.696185+0000","last_became_peered":"2023-10-06T22:12:43.696185+0000","last_unstale":"2023-10-06T22:12:45.060532+0000","last_undegraded":"2023-10-06T22:12:45.060532+0000","last_fullsized":"2023-10-06T22:12:45.060532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:35:59.584557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684897+0000","last_change":"2023-10-06T22:12:43.696995+0000","last_active":"2023-10-06T22:12:44.684897+0000","last_peered":"2023-10-06T22:12:44.684897+0000","last_clean":"2023-10-06T22:12:44.684897+0000","last_became_active":"2023-10-06T22:12:43.696731+0000","last_became_peered":"2023-10-06T22:12:43.696731+0000","last_unstale":"2023-10-06T22:12:44.684897+0000","last_undegraded":"2023-10-06T22:12:44.684897+0000","last_fullsized":"2023-10-06T22:12:44.684897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:43:31.261922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060958+0000","last_change":"2023-10-06T22:12:43.704391+0000","last_active":"2023-10-06T22:12:45.060958+0000","last_peered":"2023-10-06T22:12:45.060958+0000","last_clean":"2023-10-06T22:12:45.060958+0000","last_became_active":"2023-10-06T22:12:43.702954+0000","last_became_peered":"2023-10-06T22:12:43.702954+0000","last_unstale":"2023-10-06T22:12:45.060958+0000","last_undegraded":"2023-10-06T22:12:45.060958+0000","last_fullsized":"2023-10-06T22:12:45.060958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:38.973781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061588+0000","last_change":"2023-10-06T22:12:43.696265+0000","last_active":"2023-10-06T22:12:45.061588+0000","last_peered":"2023-10-06T22:12:45.061588+0000","last_clean":"2023-10-06T22:12:45.061588+0000","last_became_active":"2023-10-06T22:12:43.695939+0000","last_became_peered":"2023-10-06T22:12:43.695939+0000","last_unstale":"2023-10-06T22:12:45.061588+0000","last_undegraded":"2023-10-06T22:12:45.061588+0000","last_fullsized":"2023-10-06T22:12:45.061588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:32.882083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061835+0000","last_change":"2023-10-06T22:12:43.698527+0000","last_active":"2023-10-06T22:12:45.061835+0000","last_peered":"2023-10-06T22:12:45.061835+0000","last_clean":"2023-10-06T22:12:45.061835+0000","last_became_active":"2023-10-06T22:12:43.697867+0000","last_became_peered":"2023-10-06T22:12:43.697867+0000","last_unstale":"2023-10-06T22:12:45.061835+0000","last_undegraded":"2023-10-06T22:12:45.061835+0000","last_fullsized":"2023-10-06T22:12:45.061835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:08:03.617390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834916+0000","last_change":"2023-10-06T22:12:43.694717+0000","last_active":"2023-10-06T22:12:44.834916+0000","last_peered":"2023-10-06T22:12:44.834916+0000","last_clean":"2023-10-06T22:12:44.834916+0000","last_became_active":"2023-10-06T22:12:43.694321+0000","last_became_peered":"2023-10-06T22:12:43.694321+0000","last_unstale":"2023-10-06T22:12:44.834916+0000","last_undegraded":"2023-10-06T22:12:44.834916+0000","last_fullsized":"2023-10-06T22:12:44.834916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:06.334726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061102+0000","last_change":"2023-10-06T22:12:43.703770+0000","last_active":"2023-10-06T22:12:45.061102+0000","last_peered":"2023-10-06T22:12:45.061102+0000","last_clean":"2023-10-06T22:12:45.061102+0000","last_became_active":"2023-10-06T22:12:43.703434+0000","last_became_peered":"2023-10-06T22:12:43.703434+0000","last_unstale":"2023-10-06T22:12:45.061102+0000","last_undegraded":"2023-10-06T22:12:45.061102+0000","last_fullsized":"2023-10-06T22:12:45.061102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:25.894280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060975+0000","last_change":"2023-10-06T22:12:43.700994+0000","last_active":"2023-10-06T22:12:45.060975+0000","last_peered":"2023-10-06T22:12:45.060975+0000","last_clean":"2023-10-06T22:12:45.060975+0000","last_became_active":"2023-10-06T22:12:43.700652+0000","last_became_peered":"2023-10-06T22:12:43.700652+0000","last_unstale":"2023-10-06T22:12:45.060975+0000","last_undegraded":"2023-10-06T22:12:45.060975+0000","last_fullsized":"2023-10-06T22:12:45.060975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:54.855597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684430+0000","last_change":"2023-10-06T22:12:43.702099+0000","last_active":"2023-10-06T22:12:44.684430+0000","last_peered":"2023-10-06T22:12:44.684430+0000","last_clean":"2023-10-06T22:12:44.684430+0000","last_became_active":"2023-10-06T22:12:43.701802+0000","last_became_peered":"2023-10-06T22:12:43.701802+0000","last_unstale":"2023-10-06T22:12:44.684430+0000","last_undegraded":"2023-10-06T22:12:44.684430+0000","last_fullsized":"2023-10-06T22:12:44.684430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:00.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200796+0000","last_change":"2023-10-06T22:12:43.704679+0000","last_active":"2023-10-06T22:12:45.200796+0000","last_peered":"2023-10-06T22:12:45.200796+0000","last_clean":"2023-10-06T22:12:45.200796+0000","last_became_active":"2023-10-06T22:12:43.704186+0000","last_became_peered":"2023-10-06T22:12:43.704186+0000","last_unstale":"2023-10-06T22:12:45.200796+0000","last_undegraded":"2023-10-06T22:12:45.200796+0000","last_fullsized":"2023-10-06T22:12:45.200796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:28.665217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.686808+0000","last_change":"2023-10-06T22:12:43.695699+0000","last_active":"2023-10-06T22:12:44.686808+0000","last_peered":"2023-10-06T22:12:44.686808+0000","last_clean":"2023-10-06T22:12:44.686808+0000","last_became_active":"2023-10-06T22:12:43.695359+0000","last_became_peered":"2023-10-06T22:12:43.695359+0000","last_unstale":"2023-10-06T22:12:44.686808+0000","last_undegraded":"2023-10-06T22:12:44.686808+0000","last_fullsized":"2023-10-06T22:12:44.686808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:46.296768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.685297+0000","last_change":"2023-10-06T22:12:43.707170+0000","last_active":"2023-10-06T22:12:44.685297+0000","last_peered":"2023-10-06T22:12:44.685297+0000","last_clean":"2023-10-06T22:12:44.685297+0000","last_became_active":"2023-10-06T22:12:43.706875+0000","last_became_peered":"2023-10-06T22:12:43.706875+0000","last_unstale":"2023-10-06T22:12:44.685297+0000","last_undegraded":"2023-10-06T22:12:44.685297+0000","last_fullsized":"2023-10-06T22:12:44.685297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:46.548458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.726181+0000","last_change":"2023-10-06T22:12:42.693280+0000","last_active":"2023-10-06T22:12:43.726181+0000","last_peered":"2023-10-06T22:12:43.726181+0000","last_clean":"2023-10-06T22:12:43.726181+0000","last_became_active":"2023-10-06T22:12:42.692871+0000","last_became_peered":"2023-10-06T22:12:42.692871+0000","last_unstale":"2023-10-06T22:12:43.726181+0000","last_undegraded":"2023-10-06T22:12:43.726181+0000","last_fullsized":"2023-10-06T22:12:43.726181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:42.021829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061673+0000","last_change":"2023-10-06T22:12:43.702574+0000","last_active":"2023-10-06T22:12:45.061673+0000","last_peered":"2023-10-06T22:12:45.061673+0000","last_clean":"2023-10-06T22:12:45.061673+0000","last_became_active":"2023-10-06T22:12:43.702238+0000","last_became_peered":"2023-10-06T22:12:43.702238+0000","last_unstale":"2023-10-06T22:12:45.061673+0000","last_undegraded":"2023-10-06T22:12:45.061673+0000","last_fullsized":"2023-10-06T22:12:45.061673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:34:09.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.723314+0000","last_change":"2023-10-06T22:12:42.697256+0000","last_active":"2023-10-06T22:12:43.723314+0000","last_peered":"2023-10-06T22:12:43.723314+0000","last_clean":"2023-10-06T22:12:43.723314+0000","last_became_active":"2023-10-06T22:12:42.696724+0000","last_became_peered":"2023-10-06T22:12:42.696724+0000","last_unstale":"2023-10-06T22:12:43.723314+0000","last_undegraded":"2023-10-06T22:12:43.723314+0000","last_fullsized":"2023-10-06T22:12:43.723314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:50:01.378292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061995+0000","last_change":"2023-10-06T22:12:43.702050+0000","last_active":"2023-10-06T22:12:45.061995+0000","last_peered":"2023-10-06T22:12:45.061995+0000","last_clean":"2023-10-06T22:12:45.061995+0000","last_became_active":"2023-10-06T22:12:43.701432+0000","last_became_peered":"2023-10-06T22:12:43.701432+0000","last_unstale":"2023-10-06T22:12:45.061995+0000","last_undegraded":"2023-10-06T22:12:45.061995+0000","last_fullsized":"2023-10-06T22:12:45.061995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:30:37.909775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.751756+0000","last_change":"2023-10-06T22:12:42.693051+0000","last_active":"2023-10-06T22:12:47.751756+0000","last_peered":"2023-10-06T22:12:47.751756+0000","last_clean":"2023-10-06T22:12:47.751756+0000","last_became_active":"2023-10-06T22:12:42.692750+0000","last_became_peered":"2023-10-06T22:12:42.692750+0000","last_unstale":"2023-10-06T22:12:47.751756+0000","last_undegraded":"2023-10-06T22:12:47.751756+0000","last_fullsized":"2023-10-06T22:12:47.751756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:49:58.053826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684522+0000","last_change":"2023-10-06T22:12:43.696511+0000","last_active":"2023-10-06T22:12:44.684522+0000","last_peered":"2023-10-06T22:12:44.684522+0000","last_clean":"2023-10-06T22:12:44.684522+0000","last_became_active":"2023-10-06T22:12:43.696087+0000","last_became_peered":"2023-10-06T22:12:43.696087+0000","last_unstale":"2023-10-06T22:12:44.684522+0000","last_undegraded":"2023-10-06T22:12:44.684522+0000","last_fullsized":"2023-10-06T22:12:44.684522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:12:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.725457+0000","last_change":"2023-10-06T22:12:42.694963+0000","last_active":"2023-10-06T22:12:43.725457+0000","last_peered":"2023-10-06T22:12:43.725457+0000","last_clean":"2023-10-06T22:12:43.725457+0000","last_became_active":"2023-10-06T22:12:42.694517+0000","last_became_peered":"2023-10-06T22:12:42.694517+0000","last_unstale":"2023-10-06T22:12:43.725457+0000","last_undegraded":"2023-10-06T22:12:43.725457+0000","last_fullsized":"2023-10-06T22:12:43.725457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:07:33.404328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061501+0000","last_change":"2023-10-06T22:12:43.698077+0000","last_active":"2023-10-06T22:12:45.061501+0000","last_peered":"2023-10-06T22:12:45.061501+0000","last_clean":"2023-10-06T22:12:45.061501+0000","last_became_active":"2023-10-06T22:12:43.697676+0000","last_became_peered":"2023-10-06T22:12:43.697676+0000","last_unstale":"2023-10-06T22:12:45.061501+0000","last_undegraded":"2023-10-06T22:12:45.061501+0000","last_fullsized":"2023-10-06T22:12:45.061501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:59:56.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.744398+0000","last_change":"2023-10-06T22:12:42.698067+0000","last_active":"2023-10-06T22:12:46.744398+0000","last_peered":"2023-10-06T22:12:46.744398+0000","last_clean":"2023-10-06T22:12:46.744398+0000","last_became_active":"2023-10-06T22:12:42.697671+0000","last_became_peered":"2023-10-06T22:12:42.697671+0000","last_unstale":"2023-10-06T22:12:46.744398+0000","last_undegraded":"2023-10-06T22:12:46.744398+0000","last_fullsized":"2023-10-06T22:12:46.744398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:33.607513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060516+0000","last_change":"2023-10-06T22:12:43.696213+0000","last_active":"2023-10-06T22:12:45.060516+0000","last_peered":"2023-10-06T22:12:45.060516+0000","last_clean":"2023-10-06T22:12:45.060516+0000","last_became_active":"2023-10-06T22:12:43.695925+0000","last_became_peered":"2023-10-06T22:12:43.695925+0000","last_unstale":"2023-10-06T22:12:45.060516+0000","last_undegraded":"2023-10-06T22:12:45.060516+0000","last_fullsized":"2023-10-06T22:12:45.060516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:24.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.751367+0000","last_change":"2023-10-06T22:12:42.687188+0000","last_active":"2023-10-06T22:12:47.751367+0000","last_peered":"2023-10-06T22:12:47.751367+0000","last_clean":"2023-10-06T22:12:47.751367+0000","last_became_active":"2023-10-06T22:12:42.686747+0000","last_became_peered":"2023-10-06T22:12:42.686747+0000","last_unstale":"2023-10-06T22:12:47.751367+0000","last_undegraded":"2023-10-06T22:12:47.751367+0000","last_fullsized":"2023-10-06T22:12:47.751367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:21.326911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061242+0000","last_change":"2023-10-06T22:12:43.697988+0000","last_active":"2023-10-06T22:12:45.061242+0000","last_peered":"2023-10-06T22:12:45.061242+0000","last_clean":"2023-10-06T22:12:45.061242+0000","last_became_active":"2023-10-06T22:12:43.697641+0000","last_became_peered":"2023-10-06T22:12:43.697641+0000","last_unstale":"2023-10-06T22:12:45.061242+0000","last_undegraded":"2023-10-06T22:12:45.061242+0000","last_fullsized":"2023-10-06T22:12:45.061242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:33.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.689932+0000","last_change":"2023-10-06T22:12:42.688882+0000","last_active":"2023-10-06T22:12:43.689932+0000","last_peered":"2023-10-06T22:12:43.689932+0000","last_clean":"2023-10-06T22:12:43.689932+0000","last_became_active":"2023-10-06T22:12:42.687653+0000","last_became_peered":"2023-10-06T22:12:42.687653+0000","last_unstale":"2023-10-06T22:12:43.689932+0000","last_undegraded":"2023-10-06T22:12:43.689932+0000","last_fullsized":"2023-10-06T22:12:43.689932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:18.146081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060491+0000","last_change":"2023-10-06T22:12:43.696559+0000","last_active":"2023-10-06T22:12:45.060491+0000","last_peered":"2023-10-06T22:12:45.060491+0000","last_clean":"2023-10-06T22:12:45.060491+0000","last_became_active":"2023-10-06T22:12:43.696252+0000","last_became_peered":"2023-10-06T22:12:43.696252+0000","last_unstale":"2023-10-06T22:12:45.060491+0000","last_undegraded":"2023-10-06T22:12:45.060491+0000","last_fullsized":"2023-10-06T22:12:45.060491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:31:36.097000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.767500+0000","last_change":"2023-10-06T22:12:42.694299+0000","last_active":"2023-10-06T22:12:48.767500+0000","last_peered":"2023-10-06T22:12:48.767500+0000","last_clean":"2023-10-06T22:12:48.767500+0000","last_became_active":"2023-10-06T22:12:42.693820+0000","last_became_peered":"2023-10-06T22:12:42.693820+0000","last_unstale":"2023-10-06T22:12:48.767500+0000","last_undegraded":"2023-10-06T22:12:48.767500+0000","last_fullsized":"2023-10-06T22:12:48.767500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:13.837379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.686381+0000","last_change":"2023-10-06T22:12:43.695234+0000","last_active":"2023-10-06T22:12:44.686381+0000","last_peered":"2023-10-06T22:12:44.686381+0000","last_clean":"2023-10-06T22:12:44.686381+0000","last_became_active":"2023-10-06T22:12:43.694889+0000","last_became_peered":"2023-10-06T22:12:43.694889+0000","last_unstale":"2023-10-06T22:12:44.686381+0000","last_undegraded":"2023-10-06T22:12:44.686381+0000","last_fullsized":"2023-10-06T22:12:44.686381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:51.039264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.741266+0000","last_change":"2023-10-06T22:12:42.695438+0000","last_active":"2023-10-06T22:12:46.741266+0000","last_peered":"2023-10-06T22:12:46.741266+0000","last_clean":"2023-10-06T22:12:46.741266+0000","last_became_active":"2023-10-06T22:12:42.695090+0000","last_became_peered":"2023-10-06T22:12:42.695090+0000","last_unstale":"2023-10-06T22:12:46.741266+0000","last_undegraded":"2023-10-06T22:12:46.741266+0000","last_fullsized":"2023-10-06T22:12:46.741266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:25:34.385961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684842+0000","last_change":"2023-10-06T22:12:43.697459+0000","last_active":"2023-10-06T22:12:44.684842+0000","last_peered":"2023-10-06T22:12:44.684842+0000","last_clean":"2023-10-06T22:12:44.684842+0000","last_became_active":"2023-10-06T22:12:43.696405+0000","last_became_peered":"2023-10-06T22:12:43.696405+0000","last_unstale":"2023-10-06T22:12:44.684842+0000","last_undegraded":"2023-10-06T22:12:44.684842+0000","last_fullsized":"2023-10-06T22:12:44.684842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:30.928083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.738473+0000","last_change":"2023-10-06T22:12:42.692151+0000","last_active":"2023-10-06T22:12:46.738473+0000","last_peered":"2023-10-06T22:12:46.738473+0000","last_clean":"2023-10-06T22:12:46.738473+0000","last_became_active":"2023-10-06T22:12:42.691780+0000","last_became_peered":"2023-10-06T22:12:42.691780+0000","last_unstale":"2023-10-06T22:12:46.738473+0000","last_undegraded":"2023-10-06T22:12:46.738473+0000","last_fullsized":"2023-10-06T22:12:46.738473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:49:35.697733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061969+0000","last_change":"2023-10-06T22:12:43.701754+0000","last_active":"2023-10-06T22:12:45.061969+0000","last_peered":"2023-10-06T22:12:45.061969+0000","last_clean":"2023-10-06T22:12:45.061969+0000","last_became_active":"2023-10-06T22:12:43.700881+0000","last_became_peered":"2023-10-06T22:12:43.700881+0000","last_unstale":"2023-10-06T22:12:45.061969+0000","last_undegraded":"2023-10-06T22:12:45.061969+0000","last_fullsized":"2023-10-06T22:12:45.061969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:35:21.988307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.742232+0000","last_change":"2023-10-06T22:12:42.699751+0000","last_active":"2023-10-06T22:12:46.742232+0000","last_peered":"2023-10-06T22:12:46.742232+0000","last_clean":"2023-10-06T22:12:46.742232+0000","last_became_active":"2023-10-06T22:12:42.699321+0000","last_became_peered":"2023-10-06T22:12:42.699321+0000","last_unstale":"2023-10-06T22:12:46.742232+0000","last_undegraded":"2023-10-06T22:12:46.742232+0000","last_fullsized":"2023-10-06T22:12:46.742232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:35:25.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684708+0000","last_change":"2023-10-06T22:12:43.702426+0000","last_active":"2023-10-06T22:12:44.684708+0000","last_peered":"2023-10-06T22:12:44.684708+0000","last_clean":"2023-10-06T22:12:44.684708+0000","last_became_active":"2023-10-06T22:12:43.702101+0000","last_became_peered":"2023-10-06T22:12:43.702101+0000","last_unstale":"2023-10-06T22:12:44.684708+0000","last_undegraded":"2023-10-06T22:12:44.684708+0000","last_fullsized":"2023-10-06T22:12:44.684708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:31:09.133941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.740145+0000","last_change":"2023-10-06T22:12:42.688815+0000","last_active":"2023-10-06T22:12:46.740145+0000","last_peered":"2023-10-06T22:12:46.740145+0000","last_clean":"2023-10-06T22:12:46.740145+0000","last_became_active":"2023-10-06T22:12:42.688447+0000","last_became_peered":"2023-10-06T22:12:42.688447+0000","last_unstale":"2023-10-06T22:12:46.740145+0000","last_undegraded":"2023-10-06T22:12:46.740145+0000","last_fullsized":"2023-10-06T22:12:46.740145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:26:25.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060840+0000","last_change":"2023-10-06T22:12:43.698406+0000","last_active":"2023-10-06T22:12:45.060840+0000","last_peered":"2023-10-06T22:12:45.060840+0000","last_clean":"2023-10-06T22:12:45.060840+0000","last_became_active":"2023-10-06T22:12:43.697204+0000","last_became_peered":"2023-10-06T22:12:43.697204+0000","last_unstale":"2023-10-06T22:12:45.060840+0000","last_undegraded":"2023-10-06T22:12:45.060840+0000","last_fullsized":"2023-10-06T22:12:45.060840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:59.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.753271+0000","last_change":"2023-10-06T22:12:42.691007+0000","last_active":"2023-10-06T22:12:47.753271+0000","last_peered":"2023-10-06T22:12:47.753271+0000","last_clean":"2023-10-06T22:12:47.753271+0000","last_became_active":"2023-10-06T22:12:42.690693+0000","last_became_peered":"2023-10-06T22:12:42.690693+0000","last_unstale":"2023-10-06T22:12:47.753271+0000","last_undegraded":"2023-10-06T22:12:47.753271+0000","last_fullsized":"2023-10-06T22:12:47.753271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:47.670091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834687+0000","last_change":"2023-10-06T22:12:43.705131+0000","last_active":"2023-10-06T22:12:44.834687+0000","last_peered":"2023-10-06T22:12:44.834687+0000","last_clean":"2023-10-06T22:12:44.834687+0000","last_became_active":"2023-10-06T22:12:43.704469+0000","last_became_peered":"2023-10-06T22:12:43.704469+0000","last_unstale":"2023-10-06T22:12:44.834687+0000","last_undegraded":"2023-10-06T22:12:44.834687+0000","last_fullsized":"2023-10-06T22:12:44.834687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:36:39.392274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.768075+0000","last_change":"2023-10-06T22:12:42.698392+0000","last_active":"2023-10-06T22:12:48.768075+0000","last_peered":"2023-10-06T22:12:48.768075+0000","last_clean":"2023-10-06T22:12:48.768075+0000","last_became_active":"2023-10-06T22:12:42.698019+0000","last_became_peered":"2023-10-06T22:12:42.698019+0000","last_unstale":"2023-10-06T22:12:48.768075+0000","last_undegraded":"2023-10-06T22:12:48.768075+0000","last_fullsized":"2023-10-06T22:12:48.768075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:00.345300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061318+0000","last_change":"2023-10-06T22:12:43.693238+0000","last_active":"2023-10-06T22:12:45.061318+0000","last_peered":"2023-10-06T22:12:45.061318+0000","last_clean":"2023-10-06T22:12:45.061318+0000","last_became_active":"2023-10-06T22:12:43.692948+0000","last_became_peered":"2023-10-06T22:12:43.692948+0000","last_unstale":"2023-10-06T22:12:45.061318+0000","last_undegraded":"2023-10-06T22:12:45.061318+0000","last_fullsized":"2023-10-06T22:12:45.061318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:24:30.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.749906+0000","last_change":"2023-10-06T22:12:42.693213+0000","last_active":"2023-10-06T22:12:47.749906+0000","last_peered":"2023-10-06T22:12:47.749906+0000","last_clean":"2023-10-06T22:12:47.749906+0000","last_became_active":"2023-10-06T22:12:42.692894+0000","last_became_peered":"2023-10-06T22:12:42.692894+0000","last_unstale":"2023-10-06T22:12:47.749906+0000","last_undegraded":"2023-10-06T22:12:47.749906+0000","last_fullsized":"2023-10-06T22:12:47.749906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:40:21.159102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060258+0000","last_change":"2023-10-06T22:12:43.695527+0000","last_active":"2023-10-06T22:12:45.060258+0000","last_peered":"2023-10-06T22:12:45.060258+0000","last_clean":"2023-10-06T22:12:45.060258+0000","last_became_active":"2023-10-06T22:12:43.694364+0000","last_became_peered":"2023-10-06T22:12:43.694364+0000","last_unstale":"2023-10-06T22:12:45.060258+0000","last_undegraded":"2023-10-06T22:12:45.060258+0000","last_fullsized":"2023-10-06T22:12:45.060258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.754346+0000","last_change":"2023-10-06T22:12:42.697854+0000","last_active":"2023-10-06T22:12:47.754346+0000","last_peered":"2023-10-06T22:12:47.754346+0000","last_clean":"2023-10-06T22:12:47.754346+0000","last_became_active":"2023-10-06T22:12:42.697542+0000","last_became_peered":"2023-10-06T22:12:42.697542+0000","last_unstale":"2023-10-06T22:12:47.754346+0000","last_undegraded":"2023-10-06T22:12:47.754346+0000","last_fullsized":"2023-10-06T22:12:47.754346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:12:11.181398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060021+0000","last_change":"2023-10-06T22:12:43.698070+0000","last_active":"2023-10-06T22:12:45.060021+0000","last_peered":"2023-10-06T22:12:45.060021+0000","last_clean":"2023-10-06T22:12:45.060021+0000","last_became_active":"2023-10-06T22:12:43.697682+0000","last_became_peered":"2023-10-06T22:12:43.697682+0000","last_unstale":"2023-10-06T22:12:45.060021+0000","last_undegraded":"2023-10-06T22:12:45.060021+0000","last_fullsized":"2023-10-06T22:12:45.060021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:43.808866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.728852+0000","last_change":"2023-10-06T22:12:42.691691+0000","last_active":"2023-10-06T22:12:43.728852+0000","last_peered":"2023-10-06T22:12:43.728852+0000","last_clean":"2023-10-06T22:12:43.728852+0000","last_became_active":"2023-10-06T22:12:42.691001+0000","last_became_peered":"2023-10-06T22:12:42.691001+0000","last_unstale":"2023-10-06T22:12:43.728852+0000","last_undegraded":"2023-10-06T22:12:43.728852+0000","last_fullsized":"2023-10-06T22:12:43.728852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:11:18.559570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200656+0000","last_change":"2023-10-06T22:12:43.699240+0000","last_active":"2023-10-06T22:12:45.200656+0000","last_peered":"2023-10-06T22:12:45.200656+0000","last_clean":"2023-10-06T22:12:45.200656+0000","last_became_active":"2023-10-06T22:12:43.698933+0000","last_became_peered":"2023-10-06T22:12:43.698933+0000","last_unstale":"2023-10-06T22:12:45.200656+0000","last_undegraded":"2023-10-06T22:12:45.200656+0000","last_fullsized":"2023-10-06T22:12:45.200656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:18:20.928529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.682435+0000","last_change":"2023-10-06T22:12:42.695650+0000","last_active":"2023-10-06T22:12:43.682435+0000","last_peered":"2023-10-06T22:12:43.682435+0000","last_clean":"2023-10-06T22:12:43.682435+0000","last_became_active":"2023-10-06T22:12:42.695283+0000","last_became_peered":"2023-10-06T22:12:42.695283+0000","last_unstale":"2023-10-06T22:12:43.682435+0000","last_undegraded":"2023-10-06T22:12:43.682435+0000","last_fullsized":"2023-10-06T22:12:43.682435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:17:04.351934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061090+0000","last_change":"2023-10-06T22:12:43.702870+0000","last_active":"2023-10-06T22:12:45.061090+0000","last_peered":"2023-10-06T22:12:45.061090+0000","last_clean":"2023-10-06T22:12:45.061090+0000","last_became_active":"2023-10-06T22:12:43.702564+0000","last_became_peered":"2023-10-06T22:12:43.702564+0000","last_unstale":"2023-10-06T22:12:45.061090+0000","last_undegraded":"2023-10-06T22:12:45.061090+0000","last_fullsized":"2023-10-06T22:12:45.061090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:23:32.238268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061245+0000","last_change":"2023-10-06T22:12:42.696603+0000","last_active":"2023-10-06T22:12:45.061245+0000","last_peered":"2023-10-06T22:12:45.061245+0000","last_clean":"2023-10-06T22:12:45.061245+0000","last_became_active":"2023-10-06T22:12:42.696313+0000","last_became_peered":"2023-10-06T22:12:42.696313+0000","last_unstale":"2023-10-06T22:12:45.061245+0000","last_undegraded":"2023-10-06T22:12:45.061245+0000","last_fullsized":"2023-10-06T22:12:45.061245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:15.719749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060674+0000","last_change":"2023-10-06T22:12:43.703356+0000","last_active":"2023-10-06T22:12:45.060674+0000","last_peered":"2023-10-06T22:12:45.060674+0000","last_clean":"2023-10-06T22:12:45.060674+0000","last_became_active":"2023-10-06T22:12:43.702670+0000","last_became_peered":"2023-10-06T22:12:43.702670+0000","last_unstale":"2023-10-06T22:12:45.060674+0000","last_undegraded":"2023-10-06T22:12:45.060674+0000","last_fullsized":"2023-10-06T22:12:45.060674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:15:05.681491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060063+0000","last_change":"2023-10-06T22:12:43.699083+0000","last_active":"2023-10-06T22:12:45.060063+0000","last_peered":"2023-10-06T22:12:45.060063+0000","last_clean":"2023-10-06T22:12:45.060063+0000","last_became_active":"2023-10-06T22:12:43.698826+0000","last_became_peered":"2023-10-06T22:12:43.698826+0000","last_unstale":"2023-10-06T22:12:45.060063+0000","last_undegraded":"2023-10-06T22:12:45.060063+0000","last_fullsized":"2023-10-06T22:12:45.060063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:04.225060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834521+0000","last_change":"2023-10-06T22:12:42.699873+0000","last_active":"2023-10-06T22:12:44.834521+0000","last_peered":"2023-10-06T22:12:44.834521+0000","last_clean":"2023-10-06T22:12:44.834521+0000","last_became_active":"2023-10-06T22:12:42.699476+0000","last_became_peered":"2023-10-06T22:12:42.699476+0000","last_unstale":"2023-10-06T22:12:44.834521+0000","last_undegraded":"2023-10-06T22:12:44.834521+0000","last_fullsized":"2023-10-06T22:12:44.834521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:13:27.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060891+0000","last_change":"2023-10-06T22:12:43.697720+0000","last_active":"2023-10-06T22:12:45.060891+0000","last_peered":"2023-10-06T22:12:45.060891+0000","last_clean":"2023-10-06T22:12:45.060891+0000","last_became_active":"2023-10-06T22:12:43.697430+0000","last_became_peered":"2023-10-06T22:12:43.697430+0000","last_unstale":"2023-10-06T22:12:45.060891+0000","last_undegraded":"2023-10-06T22:12:45.060891+0000","last_fullsized":"2023-10-06T22:12:45.060891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:29.633012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.740844+0000","last_change":"2023-10-06T22:12:42.690781+0000","last_active":"2023-10-06T22:12:46.740844+0000","last_peered":"2023-10-06T22:12:46.740844+0000","last_clean":"2023-10-06T22:12:46.740844+0000","last_became_active":"2023-10-06T22:12:42.690274+0000","last_became_peered":"2023-10-06T22:12:42.690274+0000","last_unstale":"2023-10-06T22:12:46.740844+0000","last_undegraded":"2023-10-06T22:12:46.740844+0000","last_fullsized":"2023-10-06T22:12:46.740844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:26.653182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.693551+0000","last_change":"2023-10-06T22:11:36.409669+0000","last_active":"2023-10-06T22:12:43.693551+0000","last_peered":"2023-10-06T22:12:43.693551+0000","last_clean":"2023-10-06T22:12:43.693551+0000","last_became_active":"2023-10-06T22:11:36.409197+0000","last_became_peered":"2023-10-06T22:11:36.409197+0000","last_unstale":"2023-10-06T22:12:43.693551+0000","last_undegraded":"2023-10-06T22:12:43.693551+0000","last_fullsized":"2023-10-06T22:12:43.693551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_clean_scrub_stamp":"2023-10-06T22:08:24.598743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:50:55.982571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834557+0000","last_change":"2023-10-06T22:12:43.698617+0000","last_active":"2023-10-06T22:12:44.834557+0000","last_peered":"2023-10-06T22:12:44.834557+0000","last_clean":"2023-10-06T22:12:44.834557+0000","last_became_active":"2023-10-06T22:12:43.697942+0000","last_became_peered":"2023-10-06T22:12:43.697942+0000","last_unstale":"2023-10-06T22:12:44.834557+0000","last_undegraded":"2023-10-06T22:12:44.834557+0000","last_fullsized":"2023-10-06T22:12:44.834557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:38.075730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:47.750582+0000","last_change":"2023-10-06T22:12:42.695553+0000","last_active":"2023-10-06T22:12:47.750582+0000","last_peered":"2023-10-06T22:12:47.750582+0000","last_clean":"2023-10-06T22:12:47.750582+0000","last_became_active":"2023-10-06T22:12:42.695209+0000","last_became_peered":"2023-10-06T22:12:42.695209+0000","last_unstale":"2023-10-06T22:12:47.750582+0000","last_undegraded":"2023-10-06T22:12:47.750582+0000","last_fullsized":"2023-10-06T22:12:47.750582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:10.387710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060959+0000","last_change":"2023-10-06T22:12:42.699606+0000","last_active":"2023-10-06T22:12:45.060959+0000","last_peered":"2023-10-06T22:12:45.060959+0000","last_clean":"2023-10-06T22:12:45.060959+0000","last_became_active":"2023-10-06T22:12:42.699222+0000","last_became_peered":"2023-10-06T22:12:42.699222+0000","last_unstale":"2023-10-06T22:12:45.060959+0000","last_undegraded":"2023-10-06T22:12:45.060959+0000","last_fullsized":"2023-10-06T22:12:45.060959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:52.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834957+0000","last_change":"2023-10-06T22:12:43.697783+0000","last_active":"2023-10-06T22:12:44.834957+0000","last_peered":"2023-10-06T22:12:44.834957+0000","last_clean":"2023-10-06T22:12:44.834957+0000","last_became_active":"2023-10-06T22:12:43.697388+0000","last_became_peered":"2023-10-06T22:12:43.697388+0000","last_unstale":"2023-10-06T22:12:44.834957+0000","last_undegraded":"2023-10-06T22:12:44.834957+0000","last_fullsized":"2023-10-06T22:12:44.834957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:50.113458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.682676+0000","last_change":"2023-10-06T22:12:42.694180+0000","last_active":"2023-10-06T22:12:43.682676+0000","last_peered":"2023-10-06T22:12:43.682676+0000","last_clean":"2023-10-06T22:12:43.682676+0000","last_became_active":"2023-10-06T22:12:42.693861+0000","last_became_peered":"2023-10-06T22:12:42.693861+0000","last_unstale":"2023-10-06T22:12:43.682676+0000","last_undegraded":"2023-10-06T22:12:43.682676+0000","last_fullsized":"2023-10-06T22:12:43.682676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:28:02.511117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061448+0000","last_change":"2023-10-06T22:12:43.699118+0000","last_active":"2023-10-06T22:12:45.061448+0000","last_peered":"2023-10-06T22:12:45.061448+0000","last_clean":"2023-10-06T22:12:45.061448+0000","last_became_active":"2023-10-06T22:12:43.698540+0000","last_became_peered":"2023-10-06T22:12:43.698540+0000","last_unstale":"2023-10-06T22:12:45.061448+0000","last_undegraded":"2023-10-06T22:12:45.061448+0000","last_fullsized":"2023-10-06T22:12:45.061448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:31.192762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.739401+0000","last_change":"2023-10-06T22:12:42.690617+0000","last_active":"2023-10-06T22:12:46.739401+0000","last_peered":"2023-10-06T22:12:46.739401+0000","last_clean":"2023-10-06T22:12:46.739401+0000","last_became_active":"2023-10-06T22:12:42.690238+0000","last_became_peered":"2023-10-06T22:12:42.690238+0000","last_unstale":"2023-10-06T22:12:46.739401+0000","last_undegraded":"2023-10-06T22:12:46.739401+0000","last_fullsized":"2023-10-06T22:12:46.739401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:09.819570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061674+0000","last_change":"2023-10-06T22:12:43.699064+0000","last_active":"2023-10-06T22:12:45.061674+0000","last_peered":"2023-10-06T22:12:45.061674+0000","last_clean":"2023-10-06T22:12:45.061674+0000","last_became_active":"2023-10-06T22:12:43.698727+0000","last_became_peered":"2023-10-06T22:12:43.698727+0000","last_unstale":"2023-10-06T22:12:45.061674+0000","last_undegraded":"2023-10-06T22:12:45.061674+0000","last_fullsized":"2023-10-06T22:12:45.061674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:48:14.493376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.739139+0000","last_change":"2023-10-06T22:12:42.693779+0000","last_active":"2023-10-06T22:12:46.739139+0000","last_peered":"2023-10-06T22:12:46.739139+0000","last_clean":"2023-10-06T22:12:46.739139+0000","last_became_active":"2023-10-06T22:12:42.693257+0000","last_became_peered":"2023-10-06T22:12:42.693257+0000","last_unstale":"2023-10-06T22:12:46.739139+0000","last_undegraded":"2023-10-06T22:12:46.739139+0000","last_fullsized":"2023-10-06T22:12:46.739139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:56:48.945078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060316+0000","last_change":"2023-10-06T22:12:43.700620+0000","last_active":"2023-10-06T22:12:45.060316+0000","last_peered":"2023-10-06T22:12:45.060316+0000","last_clean":"2023-10-06T22:12:45.060316+0000","last_became_active":"2023-10-06T22:12:43.700093+0000","last_became_peered":"2023-10-06T22:12:43.700093+0000","last_unstale":"2023-10-06T22:12:45.060316+0000","last_undegraded":"2023-10-06T22:12:45.060316+0000","last_fullsized":"2023-10-06T22:12:45.060316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:29.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.684106+0000","last_change":"2023-10-06T22:12:42.689640+0000","last_active":"2023-10-06T22:12:43.684106+0000","last_peered":"2023-10-06T22:12:43.684106+0000","last_clean":"2023-10-06T22:12:43.684106+0000","last_became_active":"2023-10-06T22:12:42.688584+0000","last_became_peered":"2023-10-06T22:12:42.688584+0000","last_unstale":"2023-10-06T22:12:43.684106+0000","last_undegraded":"2023-10-06T22:12:43.684106+0000","last_fullsized":"2023-10-06T22:12:43.684106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:43.066396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200483+0000","last_change":"2023-10-06T22:12:43.700084+0000","last_active":"2023-10-06T22:12:45.200483+0000","last_peered":"2023-10-06T22:12:45.200483+0000","last_clean":"2023-10-06T22:12:45.200483+0000","last_became_active":"2023-10-06T22:12:43.699787+0000","last_became_peered":"2023-10-06T22:12:43.699787+0000","last_unstale":"2023-10-06T22:12:45.200483+0000","last_undegraded":"2023-10-06T22:12:45.200483+0000","last_fullsized":"2023-10-06T22:12:45.200483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:07.219800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.357090+0000","last_change":"2023-10-06T22:12:42.688750+0000","last_active":"2023-10-06T22:12:48.357090+0000","last_peered":"2023-10-06T22:12:48.357090+0000","last_clean":"2023-10-06T22:12:48.357090+0000","last_became_active":"2023-10-06T22:12:42.688387+0000","last_became_peered":"2023-10-06T22:12:42.688387+0000","last_unstale":"2023-10-06T22:12:48.357090+0000","last_undegraded":"2023-10-06T22:12:48.357090+0000","last_fullsized":"2023-10-06T22:12:48.357090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:32.980631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061298+0000","last_change":"2023-10-06T22:12:43.706741+0000","last_active":"2023-10-06T22:12:45.061298+0000","last_peered":"2023-10-06T22:12:45.061298+0000","last_clean":"2023-10-06T22:12:45.061298+0000","last_became_active":"2023-10-06T22:12:43.706416+0000","last_became_peered":"2023-10-06T22:12:43.706416+0000","last_unstale":"2023-10-06T22:12:45.061298+0000","last_undegraded":"2023-10-06T22:12:45.061298+0000","last_fullsized":"2023-10-06T22:12:45.061298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:55:16.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:46.739590+0000","last_change":"2023-10-06T22:12:42.692850+0000","last_active":"2023-10-06T22:12:46.739590+0000","last_peered":"2023-10-06T22:12:46.739590+0000","last_clean":"2023-10-06T22:12:46.739590+0000","last_became_active":"2023-10-06T22:12:42.692287+0000","last_became_peered":"2023-10-06T22:12:42.692287+0000","last_unstale":"2023-10-06T22:12:46.739590+0000","last_undegraded":"2023-10-06T22:12:46.739590+0000","last_fullsized":"2023-10-06T22:12:46.739590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:37:59.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061394+0000","last_change":"2023-10-06T22:12:43.700667+0000","last_active":"2023-10-06T22:12:45.061394+0000","last_peered":"2023-10-06T22:12:45.061394+0000","last_clean":"2023-10-06T22:12:45.061394+0000","last_became_active":"2023-10-06T22:12:43.698758+0000","last_became_peered":"2023-10-06T22:12:43.698758+0000","last_unstale":"2023-10-06T22:12:45.061394+0000","last_undegraded":"2023-10-06T22:12:45.061394+0000","last_fullsized":"2023-10-06T22:12:45.061394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:49:53.091578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.768637+0000","last_change":"2023-10-06T22:12:42.695964+0000","last_active":"2023-10-06T22:12:48.768637+0000","last_peered":"2023-10-06T22:12:48.768637+0000","last_clean":"2023-10-06T22:12:48.768637+0000","last_became_active":"2023-10-06T22:12:42.695431+0000","last_became_peered":"2023-10-06T22:12:42.695431+0000","last_unstale":"2023-10-06T22:12:48.768637+0000","last_undegraded":"2023-10-06T22:12:48.768637+0000","last_fullsized":"2023-10-06T22:12:48.768637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:21:42.761421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061634+0000","last_change":"2023-10-06T22:12:43.698088+0000","last_active":"2023-10-06T22:12:45.061634+0000","last_peered":"2023-10-06T22:12:45.061634+0000","last_clean":"2023-10-06T22:12:45.061634+0000","last_became_active":"2023-10-06T22:12:43.697765+0000","last_became_peered":"2023-10-06T22:12:43.697765+0000","last_unstale":"2023-10-06T22:12:45.061634+0000","last_undegraded":"2023-10-06T22:12:45.061634+0000","last_fullsized":"2023-10-06T22:12:45.061634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:26:22.958348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:48.767937+0000","last_change":"2023-10-06T22:12:42.698586+0000","last_active":"2023-10-06T22:12:48.767937+0000","last_peered":"2023-10-06T22:12:48.767937+0000","last_clean":"2023-10-06T22:12:48.767937+0000","last_became_active":"2023-10-06T22:12:42.698198+0000","last_became_peered":"2023-10-06T22:12:42.698198+0000","last_unstale":"2023-10-06T22:12:48.767937+0000","last_undegraded":"2023-10-06T22:12:48.767937+0000","last_fullsized":"2023-10-06T22:12:48.767937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:03:43.898987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060873+0000","last_change":"2023-10-06T22:12:43.698615+0000","last_active":"2023-10-06T22:12:45.060873+0000","last_peered":"2023-10-06T22:12:45.060873+0000","last_clean":"2023-10-06T22:12:45.060873+0000","last_became_active":"2023-10-06T22:12:43.698302+0000","last_became_peered":"2023-10-06T22:12:43.698302+0000","last_unstale":"2023-10-06T22:12:45.060873+0000","last_undegraded":"2023-10-06T22:12:45.060873+0000","last_fullsized":"2023-10-06T22:12:45.060873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:35.973119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T22:12:43.725187+0000","last_change":"2023-10-06T22:12:42.700325+0000","last_active":"2023-10-06T22:12:43.725187+0000","last_peered":"2023-10-06T22:12:43.725187+0000","last_clean":"2023-10-06T22:12:43.725187+0000","last_became_active":"2023-10-06T22:12:42.699426+0000","last_became_peered":"2023-10-06T22:12:42.699426+0000","last_unstale":"2023-10-06T22:12:43.725187+0000","last_undegraded":"2023-10-06T22:12:43.725187+0000","last_fullsized":"2023-10-06T22:12:43.725187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:09.649737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061763+0000","last_change":"2023-10-06T22:12:43.698620+0000","last_active":"2023-10-06T22:12:45.061763+0000","last_peered":"2023-10-06T22:12:45.061763+0000","last_clean":"2023-10-06T22:12:45.061763+0000","last_became_active":"2023-10-06T22:12:43.698335+0000","last_became_peered":"2023-10-06T22:12:43.698335+0000","last_unstale":"2023-10-06T22:12:45.061763+0000","last_undegraded":"2023-10-06T22:12:45.061763+0000","last_fullsized":"2023-10-06T22:12:45.061763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:49.222848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200485+0000","last_change":"2023-10-06T22:12:43.700477+0000","last_active":"2023-10-06T22:12:45.200485+0000","last_peered":"2023-10-06T22:12:45.200485+0000","last_clean":"2023-10-06T22:12:45.200485+0000","last_became_active":"2023-10-06T22:12:43.700177+0000","last_became_peered":"2023-10-06T22:12:43.700177+0000","last_unstale":"2023-10-06T22:12:45.200485+0000","last_undegraded":"2023-10-06T22:12:45.200485+0000","last_fullsized":"2023-10-06T22:12:45.200485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:11.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684636+0000","last_change":"2023-10-06T22:12:43.701792+0000","last_active":"2023-10-06T22:12:44.684636+0000","last_peered":"2023-10-06T22:12:44.684636+0000","last_clean":"2023-10-06T22:12:44.684636+0000","last_became_active":"2023-10-06T22:12:43.701316+0000","last_became_peered":"2023-10-06T22:12:43.701316+0000","last_unstale":"2023-10-06T22:12:44.684636+0000","last_undegraded":"2023-10-06T22:12:44.684636+0000","last_fullsized":"2023-10-06T22:12:44.684636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:03.085219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684558+0000","last_change":"2023-10-06T22:12:43.707871+0000","last_active":"2023-10-06T22:12:44.684558+0000","last_peered":"2023-10-06T22:12:44.684558+0000","last_clean":"2023-10-06T22:12:44.684558+0000","last_became_active":"2023-10-06T22:12:43.707659+0000","last_became_peered":"2023-10-06T22:12:43.707659+0000","last_unstale":"2023-10-06T22:12:44.684558+0000","last_undegraded":"2023-10-06T22:12:44.684558+0000","last_fullsized":"2023-10-06T22:12:44.684558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:43.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061483+0000","last_change":"2023-10-06T22:12:43.697701+0000","last_active":"2023-10-06T22:12:45.061483+0000","last_peered":"2023-10-06T22:12:45.061483+0000","last_clean":"2023-10-06T22:12:45.061483+0000","last_became_active":"2023-10-06T22:12:43.697361+0000","last_became_peered":"2023-10-06T22:12:43.697361+0000","last_unstale":"2023-10-06T22:12:45.061483+0000","last_undegraded":"2023-10-06T22:12:45.061483+0000","last_fullsized":"2023-10-06T22:12:45.061483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:14:53.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834822+0000","last_change":"2023-10-06T22:12:43.700058+0000","last_active":"2023-10-06T22:12:44.834822+0000","last_peered":"2023-10-06T22:12:44.834822+0000","last_clean":"2023-10-06T22:12:44.834822+0000","last_became_active":"2023-10-06T22:12:43.697893+0000","last_became_peered":"2023-10-06T22:12:43.697893+0000","last_unstale":"2023-10-06T22:12:44.834822+0000","last_undegraded":"2023-10-06T22:12:44.834822+0000","last_fullsized":"2023-10-06T22:12:44.834822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:01:37.518096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200632+0000","last_change":"2023-10-06T22:12:43.702946+0000","last_active":"2023-10-06T22:12:45.200632+0000","last_peered":"2023-10-06T22:12:45.200632+0000","last_clean":"2023-10-06T22:12:45.200632+0000","last_became_active":"2023-10-06T22:12:43.702628+0000","last_became_peered":"2023-10-06T22:12:43.702628+0000","last_unstale":"2023-10-06T22:12:45.200632+0000","last_undegraded":"2023-10-06T22:12:45.200632+0000","last_fullsized":"2023-10-06T22:12:45.200632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:58:17.040874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.685426+0000","last_change":"2023-10-06T22:12:43.700668+0000","last_active":"2023-10-06T22:12:44.685426+0000","last_peered":"2023-10-06T22:12:44.685426+0000","last_clean":"2023-10-06T22:12:44.685426+0000","last_became_active":"2023-10-06T22:12:43.700387+0000","last_became_peered":"2023-10-06T22:12:43.700387+0000","last_unstale":"2023-10-06T22:12:44.685426+0000","last_undegraded":"2023-10-06T22:12:44.685426+0000","last_fullsized":"2023-10-06T22:12:44.685426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:36:00.053496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.061280+0000","last_change":"2023-10-06T22:12:43.702658+0000","last_active":"2023-10-06T22:12:45.061280+0000","last_peered":"2023-10-06T22:12:45.061280+0000","last_clean":"2023-10-06T22:12:45.061280+0000","last_became_active":"2023-10-06T22:12:43.702264+0000","last_became_peered":"2023-10-06T22:12:43.702264+0000","last_unstale":"2023-10-06T22:12:45.061280+0000","last_undegraded":"2023-10-06T22:12:45.061280+0000","last_fullsized":"2023-10-06T22:12:45.061280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:56.472029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.059936+0000","last_change":"2023-10-06T22:12:43.705718+0000","last_active":"2023-10-06T22:12:45.059936+0000","last_peered":"2023-10-06T22:12:45.059936+0000","last_clean":"2023-10-06T22:12:45.059936+0000","last_became_active":"2023-10-06T22:12:43.705479+0000","last_became_peered":"2023-10-06T22:12:43.705479+0000","last_unstale":"2023-10-06T22:12:45.059936+0000","last_undegraded":"2023-10-06T22:12:45.059936+0000","last_fullsized":"2023-10-06T22:12:45.059936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:55:36.393577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.060870+0000","last_change":"2023-10-06T22:12:43.701891+0000","last_active":"2023-10-06T22:12:45.060870+0000","last_peered":"2023-10-06T22:12:45.060870+0000","last_clean":"2023-10-06T22:12:45.060870+0000","last_became_active":"2023-10-06T22:12:43.701535+0000","last_became_peered":"2023-10-06T22:12:43.701535+0000","last_unstale":"2023-10-06T22:12:45.060870+0000","last_undegraded":"2023-10-06T22:12:45.060870+0000","last_fullsized":"2023-10-06T22:12:45.060870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:17:40.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.684533+0000","last_change":"2023-10-06T22:12:43.707075+0000","last_active":"2023-10-06T22:12:44.684533+0000","last_peered":"2023-10-06T22:12:44.684533+0000","last_clean":"2023-10-06T22:12:44.684533+0000","last_became_active":"2023-10-06T22:12:43.706723+0000","last_became_peered":"2023-10-06T22:12:43.706723+0000","last_unstale":"2023-10-06T22:12:44.684533+0000","last_undegraded":"2023-10-06T22:12:44.684533+0000","last_fullsized":"2023-10-06T22:12:44.684533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:18.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:44.834964+0000","last_change":"2023-10-06T22:12:43.704656+0000","last_active":"2023-10-06T22:12:44.834964+0000","last_peered":"2023-10-06T22:12:44.834964+0000","last_clean":"2023-10-06T22:12:44.834964+0000","last_became_active":"2023-10-06T22:12:43.704312+0000","last_became_peered":"2023-10-06T22:12:43.704312+0000","last_unstale":"2023-10-06T22:12:44.834964+0000","last_undegraded":"2023-10-06T22:12:44.834964+0000","last_fullsized":"2023-10-06T22:12:44.834964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:21:37.951180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-06T22:12:45.200727+0000","last_change":"2023-10-06T22:12:43.702239+0000","last_active":"2023-10-06T22:12:45.200727+0000","last_peered":"2023-10-06T22:12:45.200727+0000","last_clean":"2023-10-06T22:12:45.200727+0000","last_became_active":"2023-10-06T22:12:43.701851+0000","last_became_peered":"2023-10-06T22:12:43.701851+0000","last_unstale":"2023-10-06T22:12:45.200727+0000","last_undegraded":"2023-10-06T22:12:45.200727+0000","last_fullsized":"2023-10-06T22:12:45.200727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:40:53.661587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6726,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":18404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":427400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1019045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.80400000000000005}]},{"osd":6,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.376},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.505}]},{"osd":8,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":426400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.497},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.81799999999999995}]},{"osd":8,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.91700000000000004}]},{"osd":10,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.097},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.96799999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103195,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":426225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.30199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40699999999999997}]},{"osd":3,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.623,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.629},{"interface":"front","average":{"1min":0.755,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.50900000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66200000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.75600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1017705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.42599999999999999},"min":{"1min":0.27300000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.437,"15min":0.435},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.441},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68999999999999995,"5min":0.875,"15min":0.875},"last":0.34200000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80600000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60799999999999998}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.625}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70899999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":425545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46999999999999997,"15min":0.45800000000000002},"min":{"1min":0.38200000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82199999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.65800000000000003,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.55100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.54600000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.52900000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.379},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.54700000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.55900000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.075,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.006,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.60499999999999998,"5min":0.499,"15min":0.499},"max":{"1min":1.0740000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.63700000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0840000000000001,"5min":1.173,"15min":1.173},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84399999999999997,"15min":0.84299999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0269999999999999,"5min":1.145,"15min":1.145},"last":0.74199999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.873},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.94299999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":427430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.627},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86099999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.39700000000000002,"15min":0.38100000000000001},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.434,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.753}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77600000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89100000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.84299999999999997}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.90600000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":425304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57299999999999995,"15min":0.52300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.55100000000000005,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Oct 6 22:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 22:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82999999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.46899999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.55000000000000004,"5min":0.376,"15min":0.376},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.86899999999999999},"min":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.121,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.103,"15min":1.103},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.749},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.90900000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.875},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":429071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.57099999999999995},"min":{"1min":0.42899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.83799999999999997,"5min":1.125,"15min":1.125},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52600000000000002,"15min":0.498},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.53900000000000003,"15min":0.495},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81899999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Oct 6 22:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.433,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.46200000000000002,"15min":0.45000000000000001},"min":{"1min":0.32200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72399999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":1.002,"15min":1.002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60199999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.754}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":425649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41299999999999998,"15min":0.38400000000000001},"min":{"1min":0.35499999999999998,"5min":0.221,"15min":0.221},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.433,"15min":0.39900000000000002},"min":{"1min":0.38700000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.63700000000000001,"15min":0.59099999999999997},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.63300000000000001,"15min":0.59799999999999998},"min":{"1min":0.495,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.92700000000000005}]},{"osd":2,"last update":"Fri Oct 6 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.42399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94699999999999995,"5min":1.046,"15min":1.046},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.67700000000000005,"15min":0.65000000000000002},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0700000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.67300000000000004,"15min":0.63300000000000001},"min":{"1min":0.64900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65000000000000002,"15min":0.60799999999999998},"min":{"1min":0.61699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.64200000000000002},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.95699999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.746},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95499999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.54500000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.60799999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.1180000000000001,"15min":1.155},"min":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0649999999999999,"5min":7.0529999999999999,"15min":7.0529999999999999},"last":0.93300000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.83899999999999997}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.87},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.96599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":425840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.443},{"interface":"front","average":{"1min":0.432,"5min":0.39100000000000001,"15min":0.379},"min":{"1min":0.29699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80500000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.45500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.628}]},{"osd":5,"last update":"Fri Oct 6 22:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.54500000000000004},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.53700000000000003},"min":{"1min":0.49399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57599999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.627,"15min":0.61199999999999999},"min":{"1min":0.52700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1,"5min":1,"15min":1},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Oct 6 22:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.52800000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.58899999999999997}]},{"osd":9,"last update":"Fri Oct 6 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.74099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757290,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1018424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.39800000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.56399999999999995}]},{"osd":3,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.60199999999999998,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 22:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.755},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.50800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.437,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.7440000000000002,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.81100000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89900000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.77600000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.87}]},{"osd":9,"last update":"Fri Oct 6 22:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T22:12:53.272 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T22:12:53.272 INFO:tasks.cephadm:Setting up client nodes... 2023-10-06T22:12:53.272 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-06T22:12:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:12:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:53 smithi190 ceph-mon[134891]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:12:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:53 smithi190 ceph-mon[134891]: pgmap v277: 97 pgs: 97 active+clean; 618 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.8 KiB/s wr, 22 op/s 2023-10-06T22:12:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:53 smithi057 ceph-mon[98492]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:12:54.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:53 smithi057 ceph-mon[98492]: pgmap v277: 97 pgs: 97 active+clean; 618 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.8 KiB/s wr, 22 op/s 2023-10-06T22:12:54.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:53 smithi156 ceph-mon[134358]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:12:54.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:53 smithi156 ceph-mon[134358]: pgmap v277: 97 pgs: 97 active+clean; 618 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.8 KiB/s wr, 22 op/s 2023-10-06T22:12:55.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.472+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/279461816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0074ee0 msgr2=0x7fa7d0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.472+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/279461816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0074ee0 0x7fa7d0073340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009990 tx=0x7fa7c0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.472+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/279461816 shutdown_connections 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.472+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/279461816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0108e80 0x7fa7d010b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.472+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/279461816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0074ee0 0x7fa7d0073340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.472+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/279461816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d0073910 0x7fa7d0073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.472+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/279461816 >> 172.21.15.57:0/279461816 conn(0x7fa7d00fc570 msgr2=0x7fa7d00fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.473+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/279461816 shutdown_connections 2023-10-06T22:12:55.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.473+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/279461816 wait complete. 2023-10-06T22:12:55.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 Processor -- start 2023-10-06T22:12:55.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 -- start start 2023-10-06T22:12:55.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0073910 0x7fa7d0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:55.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d0074ee0 0x7fa7d01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:55.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0108e80 0x7fa7d01a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:55.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d01a0a40 con 0x7fa7d0108e80 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d01a0b80 con 0x7fa7d0073910 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7d4b8e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d01a0d10 con 0x7fa7d0074ee0 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7ce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0073910 0x7fa7d0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.474+0000 7fa7ce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0073910 0x7fa7d0198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46112/0 (socket says 172.21.15.57:46112) 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ce59c700 1 -- 172.21.15.57:0/1593137556 learned_addr learned my addr 172.21.15.57:0/1593137556 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ced9d700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0108e80 0x7fa7d01a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7cdd9b700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d0074ee0 0x7fa7d01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ce59c700 1 -- 172.21.15.57:0/1593137556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d0074ee0 msgr2=0x7fa7d01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ce59c700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d0074ee0 0x7fa7d01993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ce59c700 1 -- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0108e80 msgr2=0x7fa7d01a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:55.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ce59c700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0108e80 0x7fa7d01a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ce59c700 1 -- 172.21.15.57:0/1593137556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00095f0 con 0x7fa7d0073910 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7ce59c700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0073910 0x7fa7d0198eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0004710 tx=0x7fa7c0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7bf7fe700 1 -- 172.21.15.57:0/1593137556 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c001d070 con 0x7fa7d0073910 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7bf7fe700 1 -- 172.21.15.57:0/1593137556 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7c0021410 con 0x7fa7d0073910 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.475+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d01a0f90 con 0x7fa7d0073910 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.476+0000 7fa7bf7fe700 1 -- 172.21.15.57:0/1593137556 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0017650 con 0x7fa7d0073910 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.476+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d01a12f0 con 0x7fa7d0073910 2023-10-06T22:12:55.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.477+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d0107530 con 0x7fa7d0073910 2023-10-06T22:12:55.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.479+0000 7fa7bf7fe700 1 -- 172.21.15.57:0/1593137556 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa7c0021580 con 0x7fa7d0073910 2023-10-06T22:12:55.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.480+0000 7fa7bf7fe700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7b80a1510 0x7fa7b80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:55.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.480+0000 7fa7bf7fe700 1 -- 172.21.15.57:0/1593137556 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7c00ce2e0 con 0x7fa7d0073910 2023-10-06T22:12:55.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.481+0000 7fa7cdd9b700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7b80a1510 0x7fa7b80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:55.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.481+0000 7fa7cdd9b700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7b80a1510 0x7fa7b80a39d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0074c10 tx=0x7fa7c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:55.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.482+0000 7fa7bf7fe700 1 -- 172.21.15.57:0/1593137556 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa7c0097920 con 0x7fa7d0073910 2023-10-06T22:12:55.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.777+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa7d002cf70 con 0x7fa7d0073910 2023-10-06T22:12:55.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.783+0000 7fa7bf7fe700 1 -- 172.21.15.57:0/1593137556 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa7c009af40 con 0x7fa7d0073910 2023-10-06T22:12:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2023-10-06T22:12:55.786 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQBnhiBl3MyCLhAANf7mfF2p+qh6wDbdAW7FbQ== 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7b80a1510 msgr2=0x7fa7b80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7b80a1510 0x7fa7b80a39d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0074c10 tx=0x7fa7c4009260 comp rx=0 tx=0).stop 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0073910 msgr2=0x7fa7d0198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0073910 0x7fa7d0198eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0004710 tx=0x7fa7c0009580 comp rx=0 tx=0).stop 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 shutdown_connections 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7b80a1510 0x7fa7b80a39d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7d0108e80 0x7fa7d01a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7d0073910 0x7fa7d0198eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 --2- 172.21.15.57:0/1593137556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d0074ee0 0x7fa7d01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:55.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 >> 172.21.15.57:0/1593137556 conn(0x7fa7d00fc570 msgr2=0x7fa7d00fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:55.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 shutdown_connections 2023-10-06T22:12:55.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:55.786+0000 7fa7d4b8e700 1 -- 172.21.15.57:0/1593137556 wait complete. 2023-10-06T22:12:56.214 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:12:56.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-06T22:12:56.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T22:12:56.316 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T22:12:56.316 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-06T22:12:56.316 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-06T22:12:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:56 smithi057 ceph-mon[98492]: pgmap v278: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2023-10-06T22:12:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:56 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1593137556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T22:12:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:56 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T22:12:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:56 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T22:12:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:56 smithi156 ceph-mon[134358]: pgmap v278: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2023-10-06T22:12:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:56 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1593137556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T22:12:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:56 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T22:12:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:56 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T22:12:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:56 smithi190 ceph-mon[134891]: pgmap v278: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2023-10-06T22:12:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:56 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1593137556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T22:12:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:56 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T22:12:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:56 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T22:12:57.098 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:12:58.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.722+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/1604894218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4106f90 msgr2=0x7f0ea4107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.722+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/1604894218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4106f90 0x7f0ea4107370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e940099c0 tx=0x7f0e94009cd0 comp rx=0 tx=0).stop 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.723+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/1604894218 shutdown_connections 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.723+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/1604894218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4101940 0x7f0ea410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.723+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/1604894218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4106f90 0x7f0ea4107370 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.723+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/1604894218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4100f80 0x7f0ea4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.723+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/1604894218 >> 172.21.15.57:0/1604894218 conn(0x7f0ea40780a0 msgr2=0x7f0ea40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.723+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/1604894218 shutdown_connections 2023-10-06T22:12:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.723+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/1604894218 wait complete. 2023-10-06T22:12:58.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 Processor -- start 2023-10-06T22:12:58.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 -- start start 2023-10-06T22:12:58.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4100f80 0x7f0ea4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:58.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4101940 0x7f0ea41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:58.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4106f90 0x7f0ea419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:58.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419e130 con 0x7f0ea4101940 2023-10-06T22:12:58.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419e2a0 con 0x7f0ea4106f90 2023-10-06T22:12:58.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea8a3b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea419e460 con 0x7f0ea4100f80 2023-10-06T22:12:58.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.724+0000 7f0ea1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4101940 0x7f0ea41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:58.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4100f80 0x7f0ea4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:58.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4106f90 0x7f0ea419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4100f80 0x7f0ea4199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:51308/0 (socket says 172.21.15.57:51308) 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4101940 0x7f0ea41996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54576/0 (socket says 172.21.15.57:54576) 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 -- 172.21.15.57:0/2415452404 learned_addr learned my addr 172.21.15.57:0/2415452404 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 -- 172.21.15.57:0/2415452404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4106f90 msgr2=0x7f0ea419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4106f90 0x7f0ea419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 -- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4101940 msgr2=0x7f0ea41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4101940 0x7f0ea41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:58.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 -- 172.21.15.57:0/2415452404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e940095f0 con 0x7f0ea4100f80 2023-10-06T22:12:58.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea1d9b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4101940 0x7f0ea41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:12:58.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.725+0000 7f0ea259c700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4100f80 0x7f0ea4199170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e940098e0 tx=0x7f0e9400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:58.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.726+0000 7f0e937fe700 1 -- 172.21.15.57:0/2415452404 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9401d070 con 0x7f0ea4100f80 2023-10-06T22:12:58.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.726+0000 7f0e937fe700 1 -- 172.21.15.57:0/2415452404 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e94022470 con 0x7f0ea4100f80 2023-10-06T22:12:58.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.726+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea419e6e0 con 0x7f0ea4100f80 2023-10-06T22:12:58.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.726+0000 7f0e937fe700 1 -- 172.21.15.57:0/2415452404 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94017650 con 0x7f0ea4100f80 2023-10-06T22:12:58.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.726+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea41a5fa0 con 0x7f0ea4100f80 2023-10-06T22:12:58.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.727+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea404ebb0 con 0x7f0ea4100f80 2023-10-06T22:12:58.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.729+0000 7f0e937fe700 1 -- 172.21.15.57:0/2415452404 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0e94022850 con 0x7f0ea4100f80 2023-10-06T22:12:58.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.730+0000 7f0e937fe700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0e8c0a1510 0x7f0e8c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:12:58.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.730+0000 7f0e937fe700 1 -- 172.21.15.57:0/2415452404 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e940cd9a0 con 0x7f0ea4100f80 2023-10-06T22:12:58.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.731+0000 7f0ea1d9b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0e8c0a1510 0x7f0e8c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:12:58.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.731+0000 7f0ea1d9b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0e8c0a1510 0x7f0e8c0a39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ea419a790 tx=0x7f0e98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:12:58.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:58.735+0000 7f0e937fe700 1 -- 172.21.15.57:0/2415452404 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0e940970f0 con 0x7f0ea4100f80 2023-10-06T22:12:58.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:58 smithi057 ceph-mon[98492]: pgmap v279: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-10-06T22:12:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:58 smithi156 ceph-mon[134358]: pgmap v279: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-10-06T22:12:59.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.005+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0ea4110ea0 con 0x7f0ea4100f80 2023-10-06T22:12:59.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.010+0000 7f0e937fe700 1 -- 172.21.15.57:0/2415452404 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260403 (secure 0 0 0) 0x7f0e9409a710 con 0x7f0ea4100f80 2023-10-06T22:12:59.017 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:12:59.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0e8c0a1510 msgr2=0x7f0e8c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:59.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0e8c0a1510 0x7f0e8c0a39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ea419a790 tx=0x7f0e98009260 comp rx=0 tx=0).stop 2023-10-06T22:12:59.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4100f80 msgr2=0x7f0ea4199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:12:59.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4100f80 0x7f0ea4199170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e940098e0 tx=0x7f0e9400f690 comp rx=0 tx=0).stop 2023-10-06T22:12:59.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 shutdown_connections 2023-10-06T22:12:59.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0e8c0a1510 0x7f0e8c0a39d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:59.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ea4106f90 0x7f0ea419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:59.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ea4100f80 0x7f0ea4199170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:59.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 --2- 172.21.15.57:0/2415452404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4101940 0x7f0ea41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:12:59.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 >> 172.21.15.57:0/2415452404 conn(0x7f0ea40780a0 msgr2=0x7f0ea410c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:12:59.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 shutdown_connections 2023-10-06T22:12:59.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:12:59.016+0000 7f0ea8a3b700 1 -- 172.21.15.57:0/2415452404 wait complete. 2023-10-06T22:12:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:58 smithi190 ceph-mon[134891]: pgmap v279: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-10-06T22:12:59.480 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":409950778},{"type":"v1","addr":"172.21.15.57:6801","nonce":409950778}]},"active_addr":"172.21.15.57:6801/409950778","active_change":"2023-10-06T22:06:05.527922+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.57:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2575175349}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":339414861}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":3940203258}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":117215786}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2323040171}]}]} 2023-10-06T22:12:59.486 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-06T22:12:59.487 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-06T22:12:59.487 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T22:12:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:12:59 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2415452404' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T22:13:00.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:12:59 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2415452404' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T22:13:00.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:12:59 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2415452404' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T22:13:00.231 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:00.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:00 smithi057 ceph-mon[98492]: pgmap v280: 97 pgs: 97 active+clean; 625 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 26 op/s 2023-10-06T22:13:01.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:00 smithi156 ceph-mon[134358]: pgmap v280: 97 pgs: 97 active+clean; 625 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 26 op/s 2023-10-06T22:13:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:00 smithi190 ceph-mon[134891]: pgmap v280: 97 pgs: 97 active+clean; 625 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 26 op/s 2023-10-06T22:13:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 -- 172.21.15.57:0/1980782628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec073a00 msgr2=0x7f55ec10d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/1980782628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec073a00 0x7f55ec10d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009990 tx=0x7f55dc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 -- 172.21.15.57:0/1980782628 shutdown_connections 2023-10-06T22:13:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/1980782628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55ec10d930 0x7f55ec10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/1980782628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55ec0730e0 0x7f55ec0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/1980782628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec073a00 0x7f55ec10d3f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 -- 172.21.15.57:0/1980782628 >> 172.21.15.57:0/1980782628 conn(0x7f55ec0780a0 msgr2=0x7f55ec0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 -- 172.21.15.57:0/1980782628 shutdown_connections 2023-10-06T22:13:01.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.736+0000 7f55f3ca3700 1 -- 172.21.15.57:0/1980782628 wait complete. 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.737+0000 7f55f3ca3700 1 Processor -- start 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f3ca3700 1 -- start start 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f3ca3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec0730e0 0x7f55ec199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f3ca3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55ec073a00 0x7f55ec1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f3ca3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55ec10d930 0x7f55ec19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f3ca3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec19e250 con 0x7f55ec0730e0 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f3ca3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec19e3c0 con 0x7f55ec073a00 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f3ca3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec19e580 con 0x7f55ec10d930 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f1a3f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec0730e0 0x7f55ec199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f1a3f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec0730e0 0x7f55ec199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54584/0 (socket says 172.21.15.57:54584) 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f1a3f700 1 -- 172.21.15.57:0/3305036162 learned_addr learned my addr 172.21.15.57:0/3305036162 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f123e700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55ec073a00 0x7f55ec1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.738+0000 7f55f2240700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55ec10d930 0x7f55ec19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f1a3f700 1 -- 172.21.15.57:0/3305036162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55ec10d930 msgr2=0x7f55ec19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f1a3f700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55ec10d930 0x7f55ec19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f1a3f700 1 -- 172.21.15.57:0/3305036162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55ec073a00 msgr2=0x7f55ec1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f1a3f700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55ec073a00 0x7f55ec1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f1a3f700 1 -- 172.21.15.57:0/3305036162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0095f0 con 0x7f55ec0730e0 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f1a3f700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec0730e0 0x7f55ec199290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9a0 tx=0x7f55e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55e2ffd700 1 -- 172.21.15.57:0/3305036162 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8009790 con 0x7f55ec0730e0 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55e2ffd700 1 -- 172.21.15.57:0/3305036162 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e8010430 con 0x7f55ec0730e0 2023-10-06T22:13:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ec19e860 con 0x7f55ec0730e0 2023-10-06T22:13:01.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55e2ffd700 1 -- 172.21.15.57:0/3305036162 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8018680 con 0x7f55ec0730e0 2023-10-06T22:13:01.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.739+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec1a60c0 con 0x7f55ec0730e0 2023-10-06T22:13:01.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.740+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ec04ebb0 con 0x7f55ec0730e0 2023-10-06T22:13:01.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.742+0000 7f55e2ffd700 1 -- 172.21.15.57:0/3305036162 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f55e80105a0 con 0x7f55ec0730e0 2023-10-06T22:13:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.744+0000 7f55e2ffd700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55d80a1590 0x7f55d80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.744+0000 7f55e2ffd700 1 -- 172.21.15.57:0/3305036162 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55e8014070 con 0x7f55ec0730e0 2023-10-06T22:13:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.744+0000 7f55f123e700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55d80a1590 0x7f55d80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:01.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.745+0000 7f55f123e700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55d80a1590 0x7f55d80a3a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55ec19a8b0 tx=0x7f55dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.748+0000 7f55e2ffd700 1 -- 172.21.15.57:0/3305036162 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f55e80964b0 con 0x7f55ec0730e0 2023-10-06T22:13:01.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.956+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55ec0008d0 con 0x7f55ec0730e0 2023-10-06T22:13:01.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.958+0000 7f55e2ffd700 1 -- 172.21.15.57:0/3305036162 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22109 (secure 0 0 0) 0x7f55e8099ad0 con 0x7f55ec0730e0 2023-10-06T22:13:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:01.961 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":83,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:12:44.676689+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55d80a1590 msgr2=0x7f55d80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55d80a1590 0x7f55d80a3a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55ec19a8b0 tx=0x7f55dc005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec0730e0 msgr2=0x7f55ec199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec0730e0 0x7f55ec199290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9a0 tx=0x7f55e800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 shutdown_connections 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f55d80a1590 0x7f55d80a3a50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55ec10d930 0x7f55ec19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55ec0730e0 0x7f55ec199290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 --2- 172.21.15.57:0/3305036162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55ec073a00 0x7f55ec1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.960+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 >> 172.21.15.57:0/3305036162 conn(0x7f55ec0780a0 msgr2=0x7f55ec0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:01.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.961+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 shutdown_connections 2023-10-06T22:13:01.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:01.961+0000 7f55f3ca3700 1 -- 172.21.15.57:0/3305036162 wait complete. 2023-10-06T22:13:02.417 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-06T22:13:02.417 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T22:13:03.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:02 smithi156 ceph-mon[134358]: pgmap v281: 97 pgs: 97 active+clean; 625 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2023-10-06T22:13:03.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:02 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3305036162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:02 smithi190 ceph-mon[134891]: pgmap v281: 97 pgs: 97 active+clean; 625 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2023-10-06T22:13:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:02 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3305036162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:03.157 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:02 smithi057 ceph-mon[98492]: pgmap v281: 97 pgs: 97 active+clean; 625 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 21 op/s 2023-10-06T22:13:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:02 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3305036162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 -- 172.21.15.57:0/2019016123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc304068c00 msgr2=0x7fc304068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 --2- 172.21.15.57:0/2019016123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc304068c00 0x7fc304068fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc300009990 tx=0x7fc300009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 -- 172.21.15.57:0/2019016123 shutdown_connections 2023-10-06T22:13:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 --2- 172.21.15.57:0/2019016123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3041062b0 0x7fc30410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 --2- 172.21.15.57:0/2019016123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc304068c00 0x7fc304068fe0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 --2- 172.21.15.57:0/2019016123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3040695b0 0x7fc304105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 -- 172.21.15.57:0/2019016123 >> 172.21.15.57:0/2019016123 conn(0x7fc304075e00 msgr2=0x7fc304076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 -- 172.21.15.57:0/2019016123 shutdown_connections 2023-10-06T22:13:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.756+0000 7fc30b339700 1 -- 172.21.15.57:0/2019016123 wait complete. 2023-10-06T22:13:04.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.757+0000 7fc30b339700 1 Processor -- start 2023-10-06T22:13:04.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc30b339700 1 -- start start 2023-10-06T22:13:04.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc30b339700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304068c00 0x7fc3041992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:04.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc30b339700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3040695b0 0x7fc3041997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc30b339700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3041062b0 0x7fc30419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc30b339700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30419e270 con 0x7fc3041062b0 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc30b339700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30419e3e0 con 0x7fc304068c00 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc30b339700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30419e5a0 con 0x7fc3040695b0 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc3098d6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3041062b0 0x7fc30419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc3098d6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3041062b0 0x7fc30419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48830/0 (socket says 172.21.15.57:48830) 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc3098d6700 1 -- 172.21.15.57:0/1198096860 learned_addr learned my addr 172.21.15.57:0/1198096860 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc3090d5700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304068c00 0x7fc3041992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.758+0000 7fc3088d4700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3040695b0 0x7fc3041997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc3090d5700 1 -- 172.21.15.57:0/1198096860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3040695b0 msgr2=0x7fc3041997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:04.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc3090d5700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3040695b0 0x7fc3041997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc3090d5700 1 -- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3041062b0 msgr2=0x7fc30419dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:04.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc3090d5700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3041062b0 0x7fc30419dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc3090d5700 1 -- 172.21.15.57:0/1198096860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000095f0 con 0x7fc304068c00 2023-10-06T22:13:04.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc3098d6700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3041062b0 0x7fc30419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:04.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc3090d5700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304068c00 0x7fc3041992b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc300009990 tx=0x7fc300009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:04.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc2fa7fc700 1 -- 172.21.15.57:0/1198096860 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30001d070 con 0x7fc304068c00 2023-10-06T22:13:04.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc2fa7fc700 1 -- 172.21.15.57:0/1198096860 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc300021410 con 0x7fc304068c00 2023-10-06T22:13:04.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30419e820 con 0x7fc304068c00 2023-10-06T22:13:04.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.759+0000 7fc2fa7fc700 1 -- 172.21.15.57:0/1198096860 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300017650 con 0x7fc304068c00 2023-10-06T22:13:04.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.760+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3041a60e0 con 0x7fc304068c00 2023-10-06T22:13:04.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.761+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30404ebb0 con 0x7fc304068c00 2023-10-06T22:13:04.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.762+0000 7fc2fa7fc700 1 -- 172.21.15.57:0/1198096860 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc300021810 con 0x7fc304068c00 2023-10-06T22:13:04.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.764+0000 7fc2fa7fc700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc2f00a1510 0x7fc2f00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:04.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.764+0000 7fc2fa7fc700 1 -- 172.21.15.57:0/1198096860 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3000ce070 con 0x7fc304068c00 2023-10-06T22:13:04.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.764+0000 7fc3088d4700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc2f00a1510 0x7fc2f00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:04.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.765+0000 7fc3088d4700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc2f00a1510 0x7fc2f00a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc30419a8d0 tx=0x7fc2f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:04.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.768+0000 7fc2fa7fc700 1 -- 172.21.15.57:0/1198096860 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc3000976b0 con 0x7fc304068c00 2023-10-06T22:13:04.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:04 smithi057 ceph-mon[98492]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2023-10-06T22:13:04.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.977+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc304110ec0 con 0x7fc304068c00 2023-10-06T22:13:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.979+0000 7fc2fa7fc700 1 -- 172.21.15.57:0/1198096860 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22109 (secure 0 0 0) 0x7fc30009acd0 con 0x7fc304068c00 2023-10-06T22:13:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:04.982 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":83,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:12:44.676689+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:13:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.981+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc2f00a1510 msgr2=0x7fc2f00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc2f00a1510 0x7fc2f00a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc30419a8d0 tx=0x7fc2f4009260 comp rx=0 tx=0).stop 2023-10-06T22:13:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304068c00 msgr2=0x7fc3041992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304068c00 0x7fc3041992b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc300009990 tx=0x7fc300009580 comp rx=0 tx=0).stop 2023-10-06T22:13:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 shutdown_connections 2023-10-06T22:13:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc2f00a1510 0x7fc2f00a39d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3041062b0 0x7fc30419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304068c00 0x7fc3041992b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 --2- 172.21.15.57:0/1198096860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3040695b0 0x7fc3041997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 >> 172.21.15.57:0/1198096860 conn(0x7fc304075e00 msgr2=0x7fc30410c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 shutdown_connections 2023-10-06T22:13:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:04.982+0000 7fc30b339700 1 -- 172.21.15.57:0/1198096860 wait complete. 2023-10-06T22:13:05.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:04 smithi156 ceph-mon[134358]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2023-10-06T22:13:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:04 smithi190 ceph-mon[134891]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2023-10-06T22:13:05.537 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-06T22:13:05.537 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-06T22:13:05.537 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-06T22:13:05.538 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-06T22:13:05.538 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-06T22:13:05.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-06T22:13:05.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-06T22:13:05.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-06T22:13:05.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-06T22:13:05.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-06T22:13:05.541 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-06T22:13:05.541 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-06T22:13:05.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1198096860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:05.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-06T22:13:05.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2023-10-06T22:13:05.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-10-06T22:13:05.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-10-06T22:13:05.791 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-10-06T22:13:05.792 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-06T22:13:05.792 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:05 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-10-06T22:13:06.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1198096860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:06.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-06T22:13:06.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2023-10-06T22:13:06.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-10-06T22:13:06.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-10-06T22:13:06.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-10-06T22:13:06.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-06T22:13:06.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:05 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-10-06T22:13:06.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1198096860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:06.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-06T22:13:06.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2023-10-06T22:13:06.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-10-06T22:13:06.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-10-06T22:13:06.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-10-06T22:13:06.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-06T22:13:06.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:05 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-10-06T22:13:07.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-10-06T22:13:07.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:06 smithi156 ceph-mon[134358]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T22:13:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2023-10-06T22:13:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-06T22:13:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2023-10-06T22:13:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-10-06T22:13:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-10-06T22:13:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-10-06T22:13:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-06T22:13:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-10-06T22:13:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:06 smithi190 ceph-mon[134891]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T22:13:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2023-10-06T22:13:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-06T22:13:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2023-10-06T22:13:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-10-06T22:13:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-10-06T22:13:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-10-06T22:13:07.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-06T22:13:07.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-10-06T22:13:07.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:06 smithi057 ceph-mon[98492]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T22:13:07.456 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:07.516 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:07.536 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:07.710 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:07.864 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:08.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:07 smithi156 ceph-mon[134358]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T22:13:08.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:07 smithi190 ceph-mon[134891]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T22:13:08.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:07 smithi057 ceph-mon[98492]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T22:13:08.425 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:08.426 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:08.432 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:08.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:08 smithi057 ceph-mon[98492]: pgmap v286: 97 pgs: 7 peering, 90 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-10-06T22:13:09.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:08 smithi156 ceph-mon[134358]: pgmap v286: 97 pgs: 7 peering, 90 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-10-06T22:13:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:08 smithi190 ceph-mon[134891]: pgmap v286: 97 pgs: 7 peering, 90 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-10-06T22:13:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:10.281 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:10.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:10 smithi057 ceph-mon[98492]: pgmap v287: 97 pgs: 7 peering, 90 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-06T22:13:11.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:10 smithi156 ceph-mon[134358]: pgmap v287: 97 pgs: 7 peering, 90 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-06T22:13:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:10 smithi190 ceph-mon[134891]: pgmap v287: 97 pgs: 7 peering, 90 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-06T22:13:13.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:12 smithi156 ceph-mon[134358]: pgmap v288: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-10-06T22:13:13.067 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:12 smithi057 ceph-mon[98492]: pgmap v288: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-10-06T22:13:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:12 smithi190 ceph-mon[134891]: pgmap v288: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-10-06T22:13:13.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 -- 172.21.15.57:0/4292496382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 msgr2=0x7f3f3010dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 --2- 172.21.15.57:0/4292496382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 0x7f3f3010dc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b0a0 tx=0x7f3f2800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 -- 172.21.15.57:0/4292496382 shutdown_connections 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 --2- 172.21.15.57:0/4292496382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 0x7f3f3010dc50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 --2- 172.21.15.57:0/4292496382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f30100fa0 0x7f3f30101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 --2- 172.21.15.57:0/4292496382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f30101950 0x7f3f301059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 -- 172.21.15.57:0/4292496382 >> 172.21.15.57:0/4292496382 conn(0x7f3f300fc8b0 msgr2=0x7f3f300fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 -- 172.21.15.57:0/4292496382 shutdown_connections 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.800+0000 7f3f35c6a700 1 -- 172.21.15.57:0/4292496382 wait complete. 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.801+0000 7f3f35c6a700 1 Processor -- start 2023-10-06T22:13:13.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.801+0000 7f3f35c6a700 1 -- start start 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35c6a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f30100fa0 0x7f3f301991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35c6a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f30101950 0x7f3f30199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35c6a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 0x7f3f3019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35c6a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3019e180 con 0x7f3f30106080 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35c6a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3019e2f0 con 0x7f3f30100fa0 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35c6a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3019e4b0 con 0x7f3f30101950 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 0x7f3f3019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 0x7f3f3019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45026/0 (socket says 172.21.15.57:45026) 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 -- 172.21.15.57:0/1449252018 learned_addr learned my addr 172.21.15.57:0/1449252018 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f34c68700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f30100fa0 0x7f3f301991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f2ffff700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f30101950 0x7f3f30199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:13.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 -- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f30101950 msgr2=0x7f3f30199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f30101950 0x7f3f30199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 -- 172.21.15.57:0/1449252018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f30100fa0 msgr2=0x7f3f301991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f30100fa0 0x7f3f301991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.802+0000 7f3f35469700 1 -- 172.21.15.57:0/1449252018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c009520 con 0x7f3f30106080 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.803+0000 7f3f35469700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 0x7f3f3019db70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009fd0 tx=0x7f3f2800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.803+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800e040 con 0x7f3f30106080 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.803+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f280094c0 con 0x7f3f30106080 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.803+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f28009d00 con 0x7f3f30106080 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.803+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f280052a0 con 0x7f3f30106080 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.803+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f301a5e20 con 0x7f3f30106080 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.804+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f3004f400 con 0x7f3f30106080 2023-10-06T22:13:13.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.808+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3f2801d470 con 0x7f3f30106080 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.809+0000 7f3f2dffb700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3f200a1590 0x7f3f200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.809+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f3f280ce710 con 0x7f3f30106080 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.810+0000 7f3f34c68700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3f200a1590 0x7f3f200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.810+0000 7f3f2dffb700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] conn(0x7f3f200ab050 0x7f3f200ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.810+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 --> [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f200adbc0 con 0x7f3f200ab050 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.810+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f280cea80 con 0x7f3f30106080 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.810+0000 7f3f34c68700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3f200a1590 0x7f3f200a3a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009c60 tx=0x7f3f24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.810+0000 7f3f2ffff700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] conn(0x7f3f200ab050 0x7f3f200ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.811+0000 7f3f2ffff700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] conn(0x7f3f200ab050 0x7f3f200ad4f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.812+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== osd.2 v2:172.21.15.190:6800/445057088 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3f200adbc0 con 0x7f3f200ab050 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.834+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 --> [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3f30111010 con 0x7f3f200ab050 2023-10-06T22:13:13.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f2dffb700 1 -- 172.21.15.57:0/1449252018 <== osd.2 v2:172.21.15.190:6800/445057088 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3f1c009a00 con 0x7f3f200ab050 2023-10-06T22:13:13.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] conn(0x7f3f200ab050 msgr2=0x7f3f200ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:13.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f35c6a700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] conn(0x7f3f200ab050 0x7f3f200ad4f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3f200a1590 msgr2=0x7f3f200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:13.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f35c6a700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f3f200a1590 0x7f3f200a3a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009c60 tx=0x7f3f24009260 comp rx=0 tx=0).stop 2023-10-06T22:13:13.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 msgr2=0x7f3f3019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:13.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f35c6a700 1 --2- 172.21.15.57:0/1449252018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f30106080 0x7f3f3019db70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009fd0 tx=0x7f3f2800bce0 comp rx=0 tx=0).stop 2023-10-06T22:13:13.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.835+0000 7f3f35469700 1 -- 172.21.15.57:0/1449252018 reap_dead start 2023-10-06T22:13:13.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.836+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 shutdown_connections 2023-10-06T22:13:13.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.836+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 >> 172.21.15.57:0/1449252018 conn(0x7f3f300fc8b0 msgr2=0x7f3f300fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:13.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.836+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 shutdown_connections 2023-10-06T22:13:13.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:13.836+0000 7f3f35c6a700 1 -- 172.21.15.57:0/1449252018 wait complete. 2023-10-06T22:13:14.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 -- 172.21.15.57:0/934261121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101b30 msgr2=0x7fc634105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 --2- 172.21.15.57:0/934261121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101b30 0x7fc634105b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc628009990 tx=0x7fc628009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 -- 172.21.15.57:0/934261121 shutdown_connections 2023-10-06T22:13:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 --2- 172.21.15.57:0/934261121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc634106260 0x7fc63410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 --2- 172.21.15.57:0/934261121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc634101180 0x7fc634101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 --2- 172.21.15.57:0/934261121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101b30 0x7fc634105b80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 -- 172.21.15.57:0/934261121 >> 172.21.15.57:0/934261121 conn(0x7fc6340fc9d0 msgr2=0x7fc6340fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 -- 172.21.15.57:0/934261121 shutdown_connections 2023-10-06T22:13:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.601+0000 7fc63a33a700 1 -- 172.21.15.57:0/934261121 wait complete. 2023-10-06T22:13:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.602+0000 7fc63a33a700 1 Processor -- start 2023-10-06T22:13:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.602+0000 7fc63a33a700 1 -- start start 2023-10-06T22:13:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc63a33a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101180 0x7fc634199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc63a33a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc634101b30 0x7fc6341997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc63a33a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc634106260 0x7fc63419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc63a33a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63419e240 con 0x7fc634101180 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc63a33a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63419e3b0 con 0x7fc634106260 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc63a33a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63419e570 con 0x7fc634101b30 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc633fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101180 0x7fc634199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc633fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101180 0x7fc634199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45044/0 (socket says 172.21.15.57:45044) 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc6337fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc634101b30 0x7fc6341997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc633fff700 1 -- 172.21.15.57:0/1126631632 learned_addr learned my addr 172.21.15.57:0/1126631632 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc6388d7700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc634106260 0x7fc63419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:14.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc6337fe700 1 -- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc634106260 msgr2=0x7fc63419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc6337fe700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc634106260 0x7fc63419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc6337fe700 1 -- 172.21.15.57:0/1126631632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101180 msgr2=0x7fc634199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc6337fe700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc634101180 0x7fc634199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.603+0000 7fc6337fe700 1 -- 172.21.15.57:0/1126631632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280095f0 con 0x7fc634101b30 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.604+0000 7fc6337fe700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc634101b30 0x7fc6341997c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098e0 tx=0x7fc628004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.604+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62801d070 con 0x7fc634101b30 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.604+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63419e7f0 con 0x7fc634101b30 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.604+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63419eab0 con 0x7fc634101b30 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.605+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc628005030 con 0x7fc634101b30 2023-10-06T22:13:14.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.605+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6280049e0 con 0x7fc634101b30 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.605+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc620000ff0 con 0x7fc634101b30 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.608+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc628004b60 con 0x7fc634101b30 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.609+0000 7fc6317fa700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc61c0a1590 0x7fc61c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.609+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fc6280ce060 con 0x7fc634101b30 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.610+0000 7fc633fff700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc61c0a1590 0x7fc61c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.610+0000 7fc6317fa700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] conn(0x7fc61c0ab050 0x7fc61c0ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.610+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 --> [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc61c0adbc0 con 0x7fc61c0ab050 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.610+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6280ce3d0 con 0x7fc634101b30 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.610+0000 7fc6388d7700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] conn(0x7fc61c0ab050 0x7fc61c0ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:14.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.611+0000 7fc6388d7700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] conn(0x7fc61c0ab050 0x7fc61c0ad4f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.611+0000 7fc633fff700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc61c0a1590 0x7fc61c0a3a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc624005950 tx=0x7fc62400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.612+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== osd.4 v2:172.21.15.156:6808/3180512737 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc61c0adbc0 con 0x7fc61c0ab050 2023-10-06T22:13:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.629+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 --> [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc620002d70 con 0x7fc61c0ab050 2023-10-06T22:13:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc6317fa700 1 -- 172.21.15.57:0/1126631632 <== osd.4 v2:172.21.15.156:6808/3180512737 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc620002d70 con 0x7fc61c0ab050 2023-10-06T22:13:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] conn(0x7fc61c0ab050 msgr2=0x7fc61c0ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc63a33a700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] conn(0x7fc61c0ab050 0x7fc61c0ad4f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:14.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc61c0a1590 msgr2=0x7fc61c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc63a33a700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc61c0a1590 0x7fc61c0a3a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc624005950 tx=0x7fc62400b500 comp rx=0 tx=0).stop 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc634101b30 msgr2=0x7fc6341997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc63a33a700 1 --2- 172.21.15.57:0/1126631632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc634101b30 0x7fc6341997c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098e0 tx=0x7fc628004730 comp rx=0 tx=0).stop 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.630+0000 7fc6388d7700 1 -- 172.21.15.57:0/1126631632 reap_dead start 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.631+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 shutdown_connections 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.631+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 >> 172.21.15.57:0/1126631632 conn(0x7fc6340fc9d0 msgr2=0x7fc6340fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.631+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 shutdown_connections 2023-10-06T22:13:14.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.631+0000 7fc63a33a700 1 -- 172.21.15.57:0/1126631632 wait complete. 2023-10-06T22:13:14.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:14 smithi057 ceph-mon[98492]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 9 op/s; 83 B/s, 1 keys/s, 1 objects/s recovering 2023-10-06T22:13:15.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:14 smithi156 ceph-mon[134358]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 9 op/s; 83 B/s, 1 keys/s, 1 objects/s recovering 2023-10-06T22:13:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:14 smithi190 ceph-mon[134891]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 9 op/s; 83 B/s, 1 keys/s, 1 objects/s recovering 2023-10-06T22:13:15.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 -- 172.21.15.57:0/647891278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79c100f80 msgr2=0x7fa79c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 --2- 172.21.15.57:0/647891278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79c100f80 0x7fa79c101400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7900099c0 tx=0x7fa790009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 -- 172.21.15.57:0/647891278 shutdown_connections 2023-10-06T22:13:15.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 --2- 172.21.15.57:0/647891278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa79c101940 0x7fa79c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 --2- 172.21.15.57:0/647891278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c106f90 0x7fa79c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 --2- 172.21.15.57:0/647891278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79c100f80 0x7fa79c101400 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 -- 172.21.15.57:0/647891278 >> 172.21.15.57:0/647891278 conn(0x7fa79c0780a0 msgr2=0x7fa79c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:15.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 -- 172.21.15.57:0/647891278 shutdown_connections 2023-10-06T22:13:15.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.632+0000 7fa7a14a9700 1 -- 172.21.15.57:0/647891278 wait complete. 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 Processor -- start 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 -- start start 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c100f80 0x7fa79c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa79c101940 0x7fa79c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79c106f90 0x7fa79c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19e180 con 0x7fa79c101940 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19e2f0 con 0x7fa79c106f90 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa7a14a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19e4b0 con 0x7fa79c100f80 2023-10-06T22:13:15.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.634+0000 7fa79a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa79c101940 0x7fa79c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa79c101940 0x7fa79c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45056/0 (socket says 172.21.15.57:45056) 2023-10-06T22:13:15.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 -- 172.21.15.57:0/177200121 learned_addr learned my addr 172.21.15.57:0/177200121 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:15.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79affd700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c100f80 0x7fa79c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79b7fe700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79c106f90 0x7fa79c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 -- 172.21.15.57:0/177200121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c100f80 msgr2=0x7fa79c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c100f80 0x7fa79c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 -- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79c106f90 msgr2=0x7fa79c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa79c106f90 0x7fa79c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 -- 172.21.15.57:0/177200121 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7900095f0 con 0x7fa79c101940 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79affd700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c100f80 0x7fa79c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.635+0000 7fa79a7fc700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa79c101940 0x7fa79c199700 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa79000b580 tx=0x7fa79000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.636+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79001d070 con 0x7fa79c101940 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.636+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa790022470 con 0x7fa79c101940 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.636+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c19e730 con 0x7fa79c101940 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.636+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790017650 con 0x7fa79c101940 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.636+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c1a5ff0 con 0x7fa79c101940 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.637+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa79c19ea00 con 0x7fa79c101940 2023-10-06T22:13:15.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.640+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa7900225e0 con 0x7fa79c101940 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.641+0000 7fa783fff700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7840a1510 0x7fa7840a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.642+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fa7900ce4a0 con 0x7fa79c101940 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.642+0000 7fa79affd700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7840a1510 0x7fa7840a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.642+0000 7fa783fff700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] conn(0x7fa7840aafd0 0x7fa7840ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.642+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 --> [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7840adb40 con 0x7fa7840aafd0 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.642+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7900ce810 con 0x7fa79c101940 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.642+0000 7fa79affd700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7840a1510 0x7fa7840a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009fd0 tx=0x7fa78c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.642+0000 7fa79b7fe700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] conn(0x7fa7840aafd0 0x7fa7840ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.643+0000 7fa79b7fe700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] conn(0x7fa7840aafd0 0x7fa7840ad470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.644+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== osd.7 v2:172.21.15.156:6816/2039885607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa7840adb40 con 0x7fa7840aafd0 2023-10-06T22:13:15.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.662+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 --> [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa79c111150 con 0x7fa7840aafd0 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa783fff700 1 -- 172.21.15.57:0/177200121 <== osd.7 v2:172.21.15.156:6816/2039885607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa79400f040 con 0x7fa7840aafd0 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] conn(0x7fa7840aafd0 msgr2=0x7fa7840ad470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] conn(0x7fa7840aafd0 0x7fa7840ad470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7840a1510 msgr2=0x7fa7840a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa7840a1510 0x7fa7840a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009fd0 tx=0x7fa78c009260 comp rx=0 tx=0).stop 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa79c101940 msgr2=0x7fa79c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 --2- 172.21.15.57:0/177200121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa79c101940 0x7fa79c199700 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa79000b580 tx=0x7fa79000bfd0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa79b7fe700 1 -- 172.21.15.57:0/177200121 reap_dead start 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 shutdown_connections 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.663+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 >> 172.21.15.57:0/177200121 conn(0x7fa79c0780a0 msgr2=0x7fa79c0ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.664+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 shutdown_connections 2023-10-06T22:13:15.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:14.664+0000 7fa7a14a9700 1 -- 172.21.15.57:0/177200121 wait complete. 2023-10-06T22:13:15.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1826563130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d781033b0 msgr2=0x7f5d78103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 --2- 172.21.15.57:0/1826563130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d781033b0 0x7f5d78103790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009990 tx=0x7f5d74009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1826563130 shutdown_connections 2023-10-06T22:13:15.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 --2- 172.21.15.57:0/1826563130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d78108490 0x7f5d78110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 --2- 172.21.15.57:0/1826563130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d781033b0 0x7f5d78103790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 --2- 172.21.15.57:0/1826563130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d78103d60 0x7f5d78107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1826563130 >> 172.21.15.57:0/1826563130 conn(0x7f5d780fec20 msgr2=0x7f5d78101040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:15.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.768+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1826563130 shutdown_connections 2023-10-06T22:13:15.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.769+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1826563130 wait complete. 2023-10-06T22:13:15.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.770+0000 7f5d7ff72700 1 Processor -- start 2023-10-06T22:13:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.770+0000 7f5d7ff72700 1 -- start start 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7ff72700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d781033b0 0x7f5d78199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7ff72700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d78103d60 0x7f5d781997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7ff72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d78108490 0x7f5d7819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7ff72700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7819e250 con 0x7f5d78108490 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7ff72700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7819e3c0 con 0x7f5d781033b0 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7ff72700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7819e580 con 0x7f5d78103d60 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7dd0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d781033b0 0x7f5d78199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7e50f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d78108490 0x7f5d7819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7dd0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d781033b0 0x7f5d78199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:35636/0 (socket says 172.21.15.57:35636) 2023-10-06T22:13:15.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7dd0e700 1 -- 172.21.15.57:0/1760048463 learned_addr learned my addr 172.21.15.57:0/1760048463 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:15.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7e50f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d78108490 0x7f5d7819dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45070/0 (socket says 172.21.15.57:45070) 2023-10-06T22:13:15.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.771+0000 7f5d7d50d700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d78103d60 0x7f5d781997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.772+0000 7f5d7dd0e700 1 -- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d78103d60 msgr2=0x7f5d781997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.772+0000 7f5d7dd0e700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d78103d60 0x7f5d781997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.772+0000 7f5d7dd0e700 1 -- 172.21.15.57:0/1760048463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d78108490 msgr2=0x7f5d7819dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.772+0000 7f5d7dd0e700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d78108490 0x7f5d7819dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.772+0000 7f5d7dd0e700 1 -- 172.21.15.57:0/1760048463 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009c40 con 0x7f5d781033b0 2023-10-06T22:13:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.772+0000 7f5d7dd0e700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d781033b0 0x7f5d78199290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005910 tx=0x7f5d74004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.773+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7401d070 con 0x7f5d781033b0 2023-10-06T22:13:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.773+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d740095f0 con 0x7f5d781033b0 2023-10-06T22:13:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.773+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d74022470 con 0x7f5d781033b0 2023-10-06T22:13:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.773+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74017650 con 0x7f5d781033b0 2023-10-06T22:13:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.773+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d781a5ef0 con 0x7f5d781033b0 2023-10-06T22:13:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.774+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d7819eae0 con 0x7f5d781033b0 2023-10-06T22:13:15.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.778+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5d740225e0 con 0x7f5d781033b0 2023-10-06T22:13:15.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.779+0000 7f5d6effd700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d640a1590 0x7f5d640a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.779+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f5d740cdfd0 con 0x7f5d781033b0 2023-10-06T22:13:15.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.780+0000 7f5d7d50d700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d640a1590 0x7f5d640a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.780+0000 7f5d6effd700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] conn(0x7f5d640ab050 0x7f5d640ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.780+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 --> [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d640adbc0 con 0x7f5d640ab050 2023-10-06T22:13:15.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.780+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d740ce340 con 0x7f5d781033b0 2023-10-06T22:13:15.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.780+0000 7f5d7e50f700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] conn(0x7f5d640ab050 0x7f5d640ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.780+0000 7f5d7d50d700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d640a1590 0x7f5d640a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d7819a8b0 tx=0x7f5d68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.781+0000 7f5d7e50f700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] conn(0x7f5d640ab050 0x7f5d640ad4f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.782+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== osd.8 v2:172.21.15.190:6816/1850516582 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5d640adbc0 con 0x7f5d640ab050 2023-10-06T22:13:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.799+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 --> [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d781133b0 con 0x7f5d640ab050 2023-10-06T22:13:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.800+0000 7f5d6effd700 1 -- 172.21.15.57:0/1760048463 <== osd.8 v2:172.21.15.190:6816/1850516582 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5d781133b0 con 0x7f5d640ab050 2023-10-06T22:13:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.800+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] conn(0x7f5d640ab050 msgr2=0x7f5d640ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.800+0000 7f5d7ff72700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] conn(0x7f5d640ab050 0x7f5d640ad4f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d640a1590 msgr2=0x7f5d640a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d640a1590 0x7f5d640a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d7819a8b0 tx=0x7f5d68009260 comp rx=0 tx=0).stop 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d781033b0 msgr2=0x7f5d78199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 --2- 172.21.15.57:0/1760048463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d781033b0 0x7f5d78199290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005910 tx=0x7f5d74004710 comp rx=0 tx=0).stop 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7e50f700 1 -- 172.21.15.57:0/1760048463 reap_dead start 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 shutdown_connections 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 >> 172.21.15.57:0/1760048463 conn(0x7f5d780fec20 msgr2=0x7f5d780ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 shutdown_connections 2023-10-06T22:13:15.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.801+0000 7f5d7ff72700 1 -- 172.21.15.57:0/1760048463 wait complete. 2023-10-06T22:13:15.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 -- 172.21.15.57:0/1794711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 msgr2=0x7ff550101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 --2- 172.21.15.57:0/1794711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 0x7ff550101400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff544009920 tx=0x7ff544009c30 comp rx=0 tx=0).stop 2023-10-06T22:13:15.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 -- 172.21.15.57:0/1794711 shutdown_connections 2023-10-06T22:13:15.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 --2- 172.21.15.57:0/1794711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550101940 0x7ff55010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 --2- 172.21.15.57:0/1794711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff550106f90 0x7ff550107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 --2- 172.21.15.57:0/1794711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 0x7ff550101400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 -- 172.21.15.57:0/1794711 >> 172.21.15.57:0/1794711 conn(0x7ff5500780a0 msgr2=0x7ff5500784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:15.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 -- 172.21.15.57:0/1794711 shutdown_connections 2023-10-06T22:13:15.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.960+0000 7ff555e37700 1 -- 172.21.15.57:0/1794711 wait complete. 2023-10-06T22:13:15.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.962+0000 7ff555e37700 1 Processor -- start 2023-10-06T22:13:15.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.962+0000 7ff555e37700 1 -- start start 2023-10-06T22:13:15.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff555e37700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 0x7ff5500731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff555e37700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff550101940 0x7ff5500736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff555e37700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550106f90 0x7ff550073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff555e37700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501a5520 con 0x7ff550101940 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff555e37700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501a5690 con 0x7ff550100f80 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff555e37700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501a5850 con 0x7ff550106f90 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff54effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff550101940 0x7ff5500736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff54effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff550101940 0x7ff5500736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45088/0 (socket says 172.21.15.57:45088) 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff54effd700 1 -- 172.21.15.57:0/1395370544 learned_addr learned my addr 172.21.15.57:0/1395370544 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff54f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 0x7ff5500731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.963+0000 7ff54ffff700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550106f90 0x7ff550073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54effd700 1 -- 172.21.15.57:0/1395370544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550106f90 msgr2=0x7ff550073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54effd700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550106f90 0x7ff550073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54effd700 1 -- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 msgr2=0x7ff5500731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54effd700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 0x7ff5500731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54effd700 1 -- 172.21.15.57:0/1395370544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009520 con 0x7ff550101940 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54f7fe700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff550100f80 0x7ff5500731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54effd700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff550101940 0x7ff5500736f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff544009840 tx=0x7ff54400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401c070 con 0x7ff550101940 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54400bba0 con 0x7ff550101940 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5501a5ad0 con 0x7ff550101940 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.964+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440038e0 con 0x7ff550101940 2023-10-06T22:13:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.965+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5501a5ee0 con 0x7ff550101940 2023-10-06T22:13:15.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.966+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff550111160 con 0x7ff550101940 2023-10-06T22:13:15.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.968+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff544020030 con 0x7ff550101940 2023-10-06T22:13:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.969+0000 7ff54cff9700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5380a1590 0x7ff5380a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.970+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7ff5440ce3a0 con 0x7ff550101940 2023-10-06T22:13:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.970+0000 7ff54cff9700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] conn(0x7ff5380aaf20 0x7ff5380ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.970+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 --> [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5380ada90 con 0x7ff5380aaf20 2023-10-06T22:13:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.970+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5440ce710 con 0x7ff550101940 2023-10-06T22:13:15.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.970+0000 7ff54f7fe700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5380a1590 0x7ff5380a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.970+0000 7ff54ffff700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] conn(0x7ff5380aaf20 0x7ff5380ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:15.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.971+0000 7ff54f7fe700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5380a1590 0x7ff5380a3a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff540005fd0 tx=0x7ff540009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.971+0000 7ff54ffff700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] conn(0x7ff5380aaf20 0x7ff5380ad3c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:15.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.972+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== osd.1 v2:172.21.15.156:6800/2038383629 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff5380ada90 con 0x7ff5380aaf20 2023-10-06T22:13:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.989+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 --> [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff55004ebb0 con 0x7ff5380aaf20 2023-10-06T22:13:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.990+0000 7ff54cff9700 1 -- 172.21.15.57:0/1395370544 <== osd.1 v2:172.21.15.156:6800/2038383629 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff55004ebb0 con 0x7ff5380aaf20 2023-10-06T22:13:15.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] conn(0x7ff5380aaf20 msgr2=0x7ff5380ad3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] conn(0x7ff5380aaf20 0x7ff5380ad3c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:15.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5380a1590 msgr2=0x7ff5380a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5380a1590 0x7ff5380a3a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff540005fd0 tx=0x7ff540009310 comp rx=0 tx=0).stop 2023-10-06T22:13:15.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff550101940 msgr2=0x7ff5500736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:15.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 --2- 172.21.15.57:0/1395370544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff550101940 0x7ff5500736f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff544009840 tx=0x7ff54400fa00 comp rx=0 tx=0).stop 2023-10-06T22:13:15.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff54ffff700 1 -- 172.21.15.57:0/1395370544 reap_dead start 2023-10-06T22:13:15.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 shutdown_connections 2023-10-06T22:13:15.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 >> 172.21.15.57:0/1395370544 conn(0x7ff5500780a0 msgr2=0x7ff5500ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:15.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 shutdown_connections 2023-10-06T22:13:15.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:15.991+0000 7ff555e37700 1 -- 172.21.15.57:0/1395370544 wait complete. 2023-10-06T22:13:16.102 INFO:teuthology.orchestra.run.smithi057.stdout:77309411387 2023-10-06T22:13:16.102 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-06T22:13:16.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.727+0000 7fb6352f2700 1 -- 172.21.15.57:0/2851079629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630101940 msgr2=0x7fb63010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.727+0000 7fb6352f2700 1 --2- 172.21.15.57:0/2851079629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630101940 0x7fb63010ddb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb62800b0d0 tx=0x7fb62800b3e0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.727+0000 7fb6352f2700 1 -- 172.21.15.57:0/2851079629 shutdown_connections 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.727+0000 7fb6352f2700 1 --2- 172.21.15.57:0/2851079629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630101940 0x7fb63010ddb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.727+0000 7fb6352f2700 1 --2- 172.21.15.57:0/2851079629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb630106f90 0x7fb630107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.727+0000 7fb6352f2700 1 --2- 172.21.15.57:0/2851079629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb630100f80 0x7fb630101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.727+0000 7fb6352f2700 1 -- 172.21.15.57:0/2851079629 >> 172.21.15.57:0/2851079629 conn(0x7fb6300780a0 msgr2=0x7fb6300784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.728+0000 7fb6352f2700 1 -- 172.21.15.57:0/2851079629 shutdown_connections 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.728+0000 7fb6352f2700 1 -- 172.21.15.57:0/2851079629 wait complete. 2023-10-06T22:13:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 Processor -- start 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 -- start start 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630100f80 0x7fb6300731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb630101940 0x7fb6300736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb630106f90 0x7fb630073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301a9aa0 con 0x7fb630100f80 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301a9be0 con 0x7fb630101940 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb6352f2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301a9d20 con 0x7fb630106f90 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb62effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630100f80 0x7fb6300731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb62effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630100f80 0x7fb6300731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45100/0 (socket says 172.21.15.57:45100) 2023-10-06T22:13:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.729+0000 7fb62effd700 1 -- 172.21.15.57:0/778639465 learned_addr learned my addr 172.21.15.57:0/778639465 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62f7fe700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb630106f90 0x7fb630073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62e7fc700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb630101940 0x7fb6300736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62effd700 1 -- 172.21.15.57:0/778639465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb630106f90 msgr2=0x7fb630073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62effd700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb630106f90 0x7fb630073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62effd700 1 -- 172.21.15.57:0/778639465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb630101940 msgr2=0x7fb6300736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62effd700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb630101940 0x7fb6300736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62effd700 1 -- 172.21.15.57:0/778639465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628009d00 con 0x7fb630100f80 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62e7fc700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb630101940 0x7fb6300736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb62effd700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630100f80 0x7fb6300731b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9d0 tx=0x7fb62000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620009790 con 0x7fb630100f80 2023-10-06T22:13:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb62000cbd0 con 0x7fb630100f80 2023-10-06T22:13:16.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6301a9fa0 con 0x7fb630100f80 2023-10-06T22:13:16.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.730+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620018680 con 0x7fb630100f80 2023-10-06T22:13:16.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.731+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6301aa4c0 con 0x7fb630100f80 2023-10-06T22:13:16.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.731+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb63004ebb0 con 0x7fb630100f80 2023-10-06T22:13:16.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.734+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fb62000cd40 con 0x7fb630100f80 2023-10-06T22:13:16.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.735+0000 7fb617fff700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb6180a1590 0x7fb6180a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:16.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.735+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fb620014070 con 0x7fb630100f80 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.735+0000 7fb62e7fc700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb6180a1590 0x7fb6180a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.735+0000 7fb617fff700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] conn(0x7fb6180ab050 0x7fb6180ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.735+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 --> [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6180adbc0 con 0x7fb6180ab050 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.735+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6200cd670 con 0x7fb630100f80 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.736+0000 7fb62f7fe700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] conn(0x7fb6180ab050 0x7fb6180ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.736+0000 7fb62e7fc700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb6180a1590 0x7fb6180a3a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb6300749a0 tx=0x7fb624009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.736+0000 7fb62f7fe700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] conn(0x7fb6180ab050 0x7fb6180ad4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.737+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== osd.3 v2:172.21.15.57:6810/140569851 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb6180adbc0 con 0x7fb6180ab050 2023-10-06T22:13:16.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.755+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 --> [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb6301111a0 con 0x7fb6180ab050 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb617fff700 1 -- 172.21.15.57:0/778639465 <== osd.3 v2:172.21.15.57:6810/140569851 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb6280092e0 con 0x7fb6180ab050 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] conn(0x7fb6180ab050 msgr2=0x7fb6180ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb6352f2700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] conn(0x7fb6180ab050 0x7fb6180ad4f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb6180a1590 msgr2=0x7fb6180a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb6352f2700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb6180a1590 0x7fb6180a3a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb6300749a0 tx=0x7fb624009260 comp rx=0 tx=0).stop 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630100f80 msgr2=0x7fb6300731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb6352f2700 1 --2- 172.21.15.57:0/778639465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb630100f80 0x7fb6300731b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e9d0 tx=0x7fb62000ece0 comp rx=0 tx=0).stop 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.756+0000 7fb62f7fe700 1 -- 172.21.15.57:0/778639465 reap_dead start 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.757+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 shutdown_connections 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.757+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 >> 172.21.15.57:0/778639465 conn(0x7fb6300780a0 msgr2=0x7fb6300ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.757+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 shutdown_connections 2023-10-06T22:13:16.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:16.757+0000 7fb6352f2700 1 -- 172.21.15.57:0/778639465 wait complete. 2023-10-06T22:13:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:16 smithi190 ceph-mon[134891]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 413 B/s wr, 9 op/s; 67 B/s, 1 keys/s, 1 objects/s recovering 2023-10-06T22:13:17.115 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:16 smithi057 ceph-mon[98492]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 413 B/s wr, 9 op/s; 67 B/s, 1 keys/s, 1 objects/s recovering 2023-10-06T22:13:17.148 INFO:teuthology.orchestra.run.smithi057.stdout:219043332132 2023-10-06T22:13:17.148 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 -- 172.21.15.57:0/830229169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30480695b0 msgr2=0x7f3048105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 --2- 172.21.15.57:0/830229169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30480695b0 0x7f3048105b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f303c009990 tx=0x7f303c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 -- 172.21.15.57:0/830229169 shutdown_connections 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 --2- 172.21.15.57:0/830229169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30481062b0 0x7f304810de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 --2- 172.21.15.57:0/830229169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3048068c00 0x7f3048068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 --2- 172.21.15.57:0/830229169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30480695b0 0x7f3048105b80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 -- 172.21.15.57:0/830229169 >> 172.21.15.57:0/830229169 conn(0x7f3048075e00 msgr2=0x7f3048076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 -- 172.21.15.57:0/830229169 shutdown_connections 2023-10-06T22:13:17.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.199+0000 7f304df88700 1 -- 172.21.15.57:0/830229169 wait complete. 2023-10-06T22:13:17.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.200+0000 7f304df88700 1 Processor -- start 2023-10-06T22:13:17.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f304df88700 1 -- start start 2023-10-06T22:13:17.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f304df88700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3048068c00 0x7f30480fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f304df88700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30480695b0 0x7f3048100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f304df88700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30481062b0 0x7f30481009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f304df88700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30481a5460 con 0x7f30481062b0 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f304df88700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30481a55d0 con 0x7f3048068c00 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f304df88700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30481a5790 con 0x7f30480695b0 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f3047fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30481062b0 0x7f30481009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f30477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3048068c00 0x7f30480fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f3046ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30480695b0 0x7f3048100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f3046ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30480695b0 0x7f3048100470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:53452/0 (socket says 172.21.15.57:53452) 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f30477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3048068c00 0x7f30480fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:35690/0 (socket says 172.21.15.57:35690) 2023-10-06T22:13:17.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.201+0000 7f3046ffd700 1 -- 172.21.15.57:0/2489975472 learned_addr learned my addr 172.21.15.57:0/2489975472 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:17.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.202+0000 7f30477fe700 1 -- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30480695b0 msgr2=0x7f3048100470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.202+0000 7f30477fe700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30480695b0 0x7f3048100470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.202+0000 7f30477fe700 1 -- 172.21.15.57:0/2489975472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30481062b0 msgr2=0x7f30481009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.202+0000 7f30477fe700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30481062b0 0x7f30481009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.202+0000 7f30477fe700 1 -- 172.21.15.57:0/2489975472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3040009c40 con 0x7f3048068c00 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.202+0000 7f3046ffd700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30480695b0 0x7f3048100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.202+0000 7f30477fe700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3048068c00 0x7f30480fff30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f303800e9f0 tx=0x7f303800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.203+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800c9c0 con 0x7f3048068c00 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.203+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3038010430 con 0x7f3048068c00 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.203+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c0095f0 con 0x7f3048068c00 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.203+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038018680 con 0x7f3048068c00 2023-10-06T22:13:17.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.203+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30481a5e30 con 0x7f3048068c00 2023-10-06T22:13:17.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.204+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3048111110 con 0x7f3048068c00 2023-10-06T22:13:17.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.206+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3038010860 con 0x7f3048068c00 2023-10-06T22:13:17.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.208+0000 7f3044ff9700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30300a1590 0x7f30300a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.208+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f3038014070 con 0x7f3048068c00 2023-10-06T22:13:17.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.208+0000 7f3046ffd700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30300a1590 0x7f30300a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.208+0000 7f3044ff9700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] conn(0x7f30300ab050 0x7f30300ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.208+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 --> [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30300adbc0 con 0x7f30300ab050 2023-10-06T22:13:17.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.208+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30380cd790 con 0x7f3048068c00 2023-10-06T22:13:17.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.209+0000 7f3047fff700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] conn(0x7f30300ab050 0x7f30300ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.209+0000 7f3046ffd700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30300a1590 0x7f30300a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3048101720 tx=0x7f303c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:17.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.209+0000 7f3047fff700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] conn(0x7f30300ab050 0x7f30300ad4f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:17.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.210+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== osd.5 v2:172.21.15.190:6808/3750215386 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f30300adbc0 con 0x7f30300ab050 2023-10-06T22:13:17.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.231+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 --> [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f304804ebb0 con 0x7f30300ab050 2023-10-06T22:13:17.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.232+0000 7f3044ff9700 1 -- 172.21.15.57:0/2489975472 <== osd.5 v2:172.21.15.190:6808/3750215386 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f304804ebb0 con 0x7f30300ab050 2023-10-06T22:13:17.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] conn(0x7f30300ab050 msgr2=0x7f30300ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] conn(0x7f30300ab050 0x7f30300ad4f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30300a1590 msgr2=0x7f30300a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30300a1590 0x7f30300a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3048101720 tx=0x7f303c005960 comp rx=0 tx=0).stop 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3048068c00 msgr2=0x7f30480fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 --2- 172.21.15.57:0/2489975472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3048068c00 0x7f30480fff30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f303800e9f0 tx=0x7f303800ed00 comp rx=0 tx=0).stop 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f3047fff700 1 -- 172.21.15.57:0/2489975472 reap_dead start 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 shutdown_connections 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 >> 172.21.15.57:0/2489975472 conn(0x7f3048075e00 msgr2=0x7f30480fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 shutdown_connections 2023-10-06T22:13:17.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.233+0000 7f304df88700 1 -- 172.21.15.57:0/2489975472 wait complete. 2023-10-06T22:13:17.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:16 smithi156 ceph-mon[134358]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 413 B/s wr, 9 op/s; 67 B/s, 1 keys/s, 1 objects/s recovering 2023-10-06T22:13:17.549 INFO:teuthology.orchestra.run.smithi057.stdout:55834574913 2023-10-06T22:13:17.549 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-06T22:13:17.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 -- 172.21.15.57:0/447423555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc108460 msgr2=0x7fd5bc110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 --2- 172.21.15.57:0/447423555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc108460 0x7fd5bc110030 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400b0a0 tx=0x7fd5b400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 -- 172.21.15.57:0/447423555 shutdown_connections 2023-10-06T22:13:17.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 --2- 172.21.15.57:0/447423555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc108460 0x7fd5bc110030 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 --2- 172.21.15.57:0/447423555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc103380 0x7fd5bc103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 --2- 172.21.15.57:0/447423555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5bc103d30 0x7fd5bc107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 -- 172.21.15.57:0/447423555 >> 172.21.15.57:0/447423555 conn(0x7fd5bc0fec10 msgr2=0x7fd5bc101030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:17.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 -- 172.21.15.57:0/447423555 shutdown_connections 2023-10-06T22:13:17.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.908+0000 7fd5c46b5700 1 -- 172.21.15.57:0/447423555 wait complete. 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.910+0000 7fd5c46b5700 1 Processor -- start 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.910+0000 7fd5c46b5700 1 -- start start 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c46b5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc103380 0x7fd5bc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c46b5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc103d30 0x7fd5bc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c46b5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5bc108460 0x7fd5bc19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c46b5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc19e3c0 con 0x7fd5bc103380 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c46b5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc19e530 con 0x7fd5bc108460 2023-10-06T22:13:17.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c46b5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc19e6f0 con 0x7fd5bc103d30 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc103380 0x7fd5bc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc103380 0x7fd5bc199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45128/0 (socket says 172.21.15.57:45128) 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 -- 172.21.15.57:0/3458636197 learned_addr learned my addr 172.21.15.57:0/3458636197 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c1c50700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc103d30 0x7fd5bc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2c52700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5bc108460 0x7fd5bc19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 -- 172.21.15.57:0/3458636197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc103d30 msgr2=0x7fd5bc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc103d30 0x7fd5bc199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 -- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5bc108460 msgr2=0x7fd5bc19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5bc108460 0x7fd5bc19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c2451700 1 -- 172.21.15.57:0/3458636197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4009d00 con 0x7fd5bc103380 2023-10-06T22:13:17.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.911+0000 7fd5c1c50700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5bc103d30 0x7fd5bc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.912+0000 7fd5c2451700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc103380 0x7fd5bc199400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9a0 tx=0x7fd5b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.912+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8009790 con 0x7fd5bc103380 2023-10-06T22:13:17.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.912+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5b8010430 con 0x7fd5bc103380 2023-10-06T22:13:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.912+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc19e9d0 con 0x7fd5bc103380 2023-10-06T22:13:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.912+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8018680 con 0x7fd5bc103380 2023-10-06T22:13:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.912+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc1a6230 con 0x7fd5bc103380 2023-10-06T22:13:17.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.913+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5bc113370 con 0x7fd5bc103380 2023-10-06T22:13:18.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.915+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fd5b80105a0 con 0x7fd5bc103380 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.916+0000 7fd5b37fe700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5a80a1590 0x7fd5a80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.917+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fd5b8014070 con 0x7fd5bc103380 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.917+0000 7fd5c1c50700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5a80a1590 0x7fd5a80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.917+0000 7fd5b37fe700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] conn(0x7fd5a80ab050 0x7fd5a80ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.917+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 --> [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5a80adbc0 con 0x7fd5a80ab050 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.917+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b80d2480 con 0x7fd5bc103380 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.917+0000 7fd5c1c50700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5a80a1590 0x7fd5a80a3a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc19aa20 tx=0x7fd5ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.917+0000 7fd5c2c52700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] conn(0x7fd5a80ab050 0x7fd5a80ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.918+0000 7fd5c2c52700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] conn(0x7fd5a80ab050 0x7fd5a80ad4f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:18.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.919+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== osd.10 v2:172.21.15.156:6824/1227399112 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd5a80adbc0 con 0x7fd5a80ab050 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.936+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 --> [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5bc04ebb0 con 0x7fd5a80ab050 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5b37fe700 1 -- 172.21.15.57:0/3458636197 <== osd.10 v2:172.21.15.156:6824/1227399112 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd5bc04ebb0 con 0x7fd5a80ab050 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] conn(0x7fd5a80ab050 msgr2=0x7fd5a80ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] conn(0x7fd5a80ab050 0x7fd5a80ad4f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5a80a1590 msgr2=0x7fd5a80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd5a80a1590 0x7fd5a80a3a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc19aa20 tx=0x7fd5ac009260 comp rx=0 tx=0).stop 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc103380 msgr2=0x7fd5bc199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 --2- 172.21.15.57:0/3458636197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5bc103380 0x7fd5bc199400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9a0 tx=0x7fd5b800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c2c52700 1 -- 172.21.15.57:0/3458636197 reap_dead start 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 shutdown_connections 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 >> 172.21.15.57:0/3458636197 conn(0x7fd5bc0fec10 msgr2=0x7fd5bc0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:18.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 shutdown_connections 2023-10-06T22:13:18.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:17.937+0000 7fd5c46b5700 1 -- 172.21.15.57:0/3458636197 wait complete. 2023-10-06T22:13:18.511 INFO:teuthology.orchestra.run.smithi057.stdout:107374182456 2023-10-06T22:13:18.512 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-06T22:13:18.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.717+0000 7f3073e89700 1 -- 172.21.15.57:0/924863626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f306c10da10 msgr2=0x7f306c10fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.717+0000 7f3073e89700 1 --2- 172.21.15.57:0/924863626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f306c10da10 0x7f306c10fe00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3064009f90 tx=0x7f306400b330 comp rx=0 tx=0).stop 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.717+0000 7f3073e89700 1 -- 172.21.15.57:0/924863626 shutdown_connections 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.717+0000 7f3073e89700 1 --2- 172.21.15.57:0/924863626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f306c10da10 0x7f306c10fe00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.717+0000 7f3073e89700 1 --2- 172.21.15.57:0/924863626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f306c0689a0 0x7f306c068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.718+0000 7f3073e89700 1 --2- 172.21.15.57:0/924863626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c0692c0 0x7f306c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.718+0000 7f3073e89700 1 -- 172.21.15.57:0/924863626 >> 172.21.15.57:0/924863626 conn(0x7f306c075e00 msgr2=0x7f306c076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.718+0000 7f3073e89700 1 -- 172.21.15.57:0/924863626 shutdown_connections 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.718+0000 7f3073e89700 1 -- 172.21.15.57:0/924863626 wait complete. 2023-10-06T22:13:18.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 Processor -- start 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 -- start start 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f306c0689a0 0x7f306c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f306c0692c0 0x7f306c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c10da10 0x7f306c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c19e340 con 0x7f306c0689a0 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c19e4b0 con 0x7f306c0692c0 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3073e89700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c19e670 con 0x7f306c10da10 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.719+0000 7f3071c25700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f306c0689a0 0x7f306c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3071c25700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f306c0689a0 0x7f306c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45152/0 (socket says 172.21.15.57:45152) 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3072426700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c10da10 0x7f306c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3071c25700 1 -- 172.21.15.57:0/2884468887 learned_addr learned my addr 172.21.15.57:0/2884468887 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:18.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3071424700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f306c0692c0 0x7f306c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3072426700 1 -- 172.21.15.57:0/2884468887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f306c0692c0 msgr2=0x7f306c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3072426700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f306c0692c0 0x7f306c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3072426700 1 -- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f306c0689a0 msgr2=0x7f306c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3072426700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f306c0689a0 0x7f306c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3072426700 1 -- 172.21.15.57:0/2884468887 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064009c40 con 0x7f306c10da10 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3072426700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c10da10 0x7f306c19dd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f306400b6c0 tx=0x7f3064004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.720+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306400e040 con 0x7f306c10da10 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.721+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c19e8f0 con 0x7f306c10da10 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.721+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3064004e30 con 0x7f306c10da10 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.721+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30640052a0 con 0x7f306c10da10 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.721+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c1a61b0 con 0x7f306c10da10 2023-10-06T22:13:18.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.722+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f306c1132c0 con 0x7f306c10da10 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.724+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f306401d470 con 0x7f306c10da10 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.725+0000 7f3062ffd700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30580a1590 0x7f30580a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.725+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f30640ce630 con 0x7f306c10da10 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.725+0000 7f3071c25700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30580a1590 0x7f30580a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.726+0000 7f3062ffd700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] conn(0x7f30580ab050 0x7f30580ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.726+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 --> [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30580adbc0 con 0x7f30580ab050 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.726+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30640ce9a0 con 0x7f306c10da10 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.726+0000 7f3071424700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] conn(0x7f30580ab050 0x7f30580ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.726+0000 7f3071c25700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30580a1590 0x7f30580a3a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3068005950 tx=0x7f3068009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.726+0000 7f3071424700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] conn(0x7f30580ab050 0x7f30580ad4f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:18.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.727+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== osd.6 v2:172.21.15.57:6818/2530618700 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f30580adbc0 con 0x7f30580ab050 2023-10-06T22:13:18.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.761+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 --> [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f306c04ebb0 con 0x7f30580ab050 2023-10-06T22:13:18.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.762+0000 7f3062ffd700 1 -- 172.21.15.57:0/2884468887 <== osd.6 v2:172.21.15.57:6818/2530618700 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f306c04ebb0 con 0x7f30580ab050 2023-10-06T22:13:18.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.762+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] conn(0x7f30580ab050 msgr2=0x7f30580ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.762+0000 7f3073e89700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] conn(0x7f30580ab050 0x7f30580ad4f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:18.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30580a1590 msgr2=0x7f30580a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30580a1590 0x7f30580a3a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3068005950 tx=0x7f3068009310 comp rx=0 tx=0).stop 2023-10-06T22:13:18.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c10da10 msgr2=0x7f306c19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:18.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 --2- 172.21.15.57:0/2884468887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c10da10 0x7f306c19dd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f306400b6c0 tx=0x7f3064004a30 comp rx=0 tx=0).stop 2023-10-06T22:13:18.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3072426700 1 -- 172.21.15.57:0/2884468887 reap_dead start 2023-10-06T22:13:18.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 shutdown_connections 2023-10-06T22:13:18.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 >> 172.21.15.57:0/2884468887 conn(0x7f306c075e00 msgr2=0x7f306c10edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:18.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 shutdown_connections 2023-10-06T22:13:18.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:18.763+0000 7f3073e89700 1 -- 172.21.15.57:0/2884468887 wait complete. 2023-10-06T22:13:18.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:18 smithi057 ceph-mon[98492]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 375 B/s wr, 9 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:18.896 INFO:teuthology.orchestra.run.smithi057.stdout:163208757296 2023-10-06T22:13:18.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-06T22:13:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:18 smithi190 ceph-mon[134891]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 375 B/s wr, 9 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:19.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:18 smithi156 ceph-mon[134358]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 375 B/s wr, 9 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:19.276 INFO:teuthology.orchestra.run.smithi057.stdout:141733920818 2023-10-06T22:13:19.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-06T22:13:19.498 INFO:teuthology.orchestra.run.smithi057.stdout:304942678039 2023-10-06T22:13:19.499 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-06T22:13:19.543 INFO:teuthology.orchestra.run.smithi057.stdout:249108103201 2023-10-06T22:13:19.544 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-06T22:13:19.990 INFO:teuthology.orchestra.run.smithi057.stdout:197568495658 2023-10-06T22:13:19.991 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-06T22:13:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.345+0000 7f7651786700 1 -- 172.21.15.57:0/366284198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f764c068c00 msgr2=0x7f764c068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.345+0000 7f7651786700 1 --2- 172.21.15.57:0/366284198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f764c068c00 0x7f764c068fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f763c009990 tx=0x7f763c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.345+0000 7f7651786700 1 -- 172.21.15.57:0/366284198 shutdown_connections 2023-10-06T22:13:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.345+0000 7f7651786700 1 --2- 172.21.15.57:0/366284198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f764c1062b0 0x7f764c10de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.345+0000 7f7651786700 1 --2- 172.21.15.57:0/366284198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f764c068c00 0x7f764c068fe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.345+0000 7f7651786700 1 --2- 172.21.15.57:0/366284198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f764c0695b0 0x7f764c105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.345+0000 7f7651786700 1 -- 172.21.15.57:0/366284198 >> 172.21.15.57:0/366284198 conn(0x7f764c075e00 msgr2=0x7f764c076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.346+0000 7f7651786700 1 -- 172.21.15.57:0/366284198 shutdown_connections 2023-10-06T22:13:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.346+0000 7f7651786700 1 -- 172.21.15.57:0/366284198 wait complete. 2023-10-06T22:13:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.347+0000 7f7651786700 1 Processor -- start 2023-10-06T22:13:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.348+0000 7f7651786700 1 -- start start 2023-10-06T22:13:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f7651786700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f764c068c00 0x7f764c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f7651786700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f764c0695b0 0x7f764c100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f7651786700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f764c1062b0 0x7f764c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f7651786700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1a54b0 con 0x7f764c1062b0 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f7651786700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1a5620 con 0x7f764c0695b0 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f7651786700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1a57e0 con 0x7f764c068c00 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f764a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f764c0695b0 0x7f764c100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f764affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f764c068c00 0x7f764c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f764a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f764c0695b0 0x7f764c100470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:35744/0 (socket says 172.21.15.57:35744) 2023-10-06T22:13:20.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.349+0000 7f764a7fc700 1 -- 172.21.15.57:0/3482939650 learned_addr learned my addr 172.21.15.57:0/3482939650 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.350+0000 7f764b7fe700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f764c1062b0 0x7f764c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.350+0000 7f764affd700 1 -- 172.21.15.57:0/3482939650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f764c0695b0 msgr2=0x7f764c100470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.350+0000 7f764affd700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f764c0695b0 0x7f764c100470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.350+0000 7f764affd700 1 -- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f764c1062b0 msgr2=0x7f764c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.350+0000 7f764affd700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f764c1062b0 0x7f764c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.350+0000 7f764affd700 1 -- 172.21.15.57:0/3482939650 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0095f0 con 0x7f764c068c00 2023-10-06T22:13:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.350+0000 7f764affd700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f764c068c00 0x7f764c0fff30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f763c006010 tx=0x7f763c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.351+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c01d070 con 0x7f764c068c00 2023-10-06T22:13:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.351+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f763c005030 con 0x7f764c068c00 2023-10-06T22:13:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.351+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c004bc0 con 0x7f764c068c00 2023-10-06T22:13:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.352+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c1a5a60 con 0x7f764c068c00 2023-10-06T22:13:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.352+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c1a5dc0 con 0x7f764c068c00 2023-10-06T22:13:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.353+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f764c1111d0 con 0x7f764c068c00 2023-10-06T22:13:20.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.357+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f763c004d20 con 0x7f764c068c00 2023-10-06T22:13:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.358+0000 7f7633fff700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76340a1690 0x7f76340a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.359+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f763c0ce190 con 0x7f764c068c00 2023-10-06T22:13:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.359+0000 7f7633fff700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] conn(0x7f76340ab150 0x7f76340ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.359+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 --> [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76340adcc0 con 0x7f76340ab150 2023-10-06T22:13:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.359+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f763c0ce5a0 con 0x7f764c068c00 2023-10-06T22:13:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.359+0000 7f764a7fc700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76340a1690 0x7f76340a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.360+0000 7f764b7fe700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] conn(0x7f76340ab150 0x7f76340ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.360+0000 7f764a7fc700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76340a1690 0x7f76340a3b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f764c101720 tx=0x7f7640009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.363+0000 7f764b7fe700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] conn(0x7f76340ab150 0x7f76340ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:20.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.364+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== osd.9 v2:172.21.15.57:6826/377680664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f76340adcc0 con 0x7f76340ab150 2023-10-06T22:13:20.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.393+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 --> [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f764c04ebb0 con 0x7f76340ab150 2023-10-06T22:13:20.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.394+0000 7f7633fff700 1 -- 172.21.15.57:0/3482939650 <== osd.9 v2:172.21.15.57:6826/377680664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f764c04ebb0 con 0x7f76340ab150 2023-10-06T22:13:20.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.397+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] conn(0x7f76340ab150 msgr2=0x7f76340ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.397+0000 7f7651786700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] conn(0x7f76340ab150 0x7f76340ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.398+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76340a1690 msgr2=0x7f76340a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.398+0000 7f7651786700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f76340a1690 0x7f76340a3b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f764c101720 tx=0x7f7640009260 comp rx=0 tx=0).stop 2023-10-06T22:13:20.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.398+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f764c068c00 msgr2=0x7f764c0fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.398+0000 7f7651786700 1 --2- 172.21.15.57:0/3482939650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f764c068c00 0x7f764c0fff30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f763c006010 tx=0x7f763c004730 comp rx=0 tx=0).stop 2023-10-06T22:13:20.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.398+0000 7f764b7fe700 1 -- 172.21.15.57:0/3482939650 reap_dead start 2023-10-06T22:13:20.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.398+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 shutdown_connections 2023-10-06T22:13:20.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.398+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 >> 172.21.15.57:0/3482939650 conn(0x7f764c075e00 msgr2=0x7f764c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:20.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.399+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 shutdown_connections 2023-10-06T22:13:20.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.399+0000 7f7651786700 1 -- 172.21.15.57:0/3482939650 wait complete. 2023-10-06T22:13:20.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 -- 172.21.15.57:0/3410870209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8100f80 msgr2=0x7f28a8101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 --2- 172.21.15.57:0/3410870209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8100f80 0x7f28a8101400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f28980099c0 tx=0x7f2898009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 -- 172.21.15.57:0/3410870209 shutdown_connections 2023-10-06T22:13:20.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 --2- 172.21.15.57:0/3410870209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28a8101940 0x7f28a810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 --2- 172.21.15.57:0/3410870209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a8106f90 0x7f28a8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 --2- 172.21.15.57:0/3410870209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8100f80 0x7f28a8101400 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 -- 172.21.15.57:0/3410870209 >> 172.21.15.57:0/3410870209 conn(0x7f28a80780a0 msgr2=0x7f28a80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:20.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 -- 172.21.15.57:0/3410870209 shutdown_connections 2023-10-06T22:13:20.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.448+0000 7f28af5a4700 1 -- 172.21.15.57:0/3410870209 wait complete. 2023-10-06T22:13:20.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.450+0000 7f28af5a4700 1 Processor -- start 2023-10-06T22:13:20.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.450+0000 7f28af5a4700 1 -- start start 2023-10-06T22:13:20.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.451+0000 7f28af5a4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28a8100f80 0x7f28a8197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.451+0000 7f28af5a4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a8101940 0x7f28a81975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.451+0000 7f28af5a4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8106f90 0x7f28a819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.451+0000 7f28af5a4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819c020 con 0x7f28a8100f80 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.451+0000 7f28af5a4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819c190 con 0x7f28a8106f90 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.451+0000 7f28af5a4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819c350 con 0x7f28a8101940 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.451+0000 7f28ad340700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28a8100f80 0x7f28a8197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28ad340700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28a8100f80 0x7f28a8197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45184/0 (socket says 172.21.15.57:45184) 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28ad340700 1 -- 172.21.15.57:0/1938760039 learned_addr learned my addr 172.21.15.57:0/1938760039 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:20.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28adb41700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8106f90 0x7f28a819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28acb3f700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a8101940 0x7f28a81975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28adb41700 1 -- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a8101940 msgr2=0x7f28a81975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28adb41700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a8101940 0x7f28a81975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28adb41700 1 -- 172.21.15.57:0/1938760039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28a8100f80 msgr2=0x7f28a8197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28adb41700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f28a8100f80 0x7f28a8197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28adb41700 1 -- 172.21.15.57:0/1938760039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a4009520 con 0x7f28a8106f90 2023-10-06T22:13:20.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.452+0000 7f28adb41700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8106f90 0x7f28a819ba10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28a0007f10 tx=0x7f28a000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:20.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.453+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0004db0 con 0x7f28a8106f90 2023-10-06T22:13:20.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.453+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a000f040 con 0x7f28a8106f90 2023-10-06T22:13:20.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.453+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28980095f0 con 0x7f28a8106f90 2023-10-06T22:13:20.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.453+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0005830 con 0x7f28a8106f90 2023-10-06T22:13:20.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.453+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a81a3cc0 con 0x7f28a8106f90 2023-10-06T22:13:20.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.454+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f288c000ff0 con 0x7f28a8106f90 2023-10-06T22:13:20.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.458+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f28a000db20 con 0x7f28a8106f90 2023-10-06T22:13:20.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.461+0000 7f289e7fc700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28940a1590 0x7f28940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.461+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f28a00ccfc0 con 0x7f28a8106f90 2023-10-06T22:13:20.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.461+0000 7f28ad340700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28940a1590 0x7f28940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.461+0000 7f289e7fc700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] conn(0x7f28940ab050 0x7f28940ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:20.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.461+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 --> [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28940adbc0 con 0x7f28940ab050 2023-10-06T22:13:20.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.462+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28a00cd460 con 0x7f28a8106f90 2023-10-06T22:13:20.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.462+0000 7f28acb3f700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] conn(0x7f28940ab050 0x7f28940ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:20.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.462+0000 7f28ad340700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28940a1590 0x7f28940a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009e70 tx=0x7f28a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:20.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.463+0000 7f28acb3f700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] conn(0x7f28940ab050 0x7f28940ad4f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:20.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.464+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== osd.11 v2:172.21.15.190:6824/826697758 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f28940adbc0 con 0x7f28940ab050 2023-10-06T22:13:20.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.483+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 --> [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f288c002d70 con 0x7f28940ab050 2023-10-06T22:13:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.483+0000 7f289e7fc700 1 -- 172.21.15.57:0/1938760039 <== osd.11 v2:172.21.15.190:6824/826697758 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f288c002d70 con 0x7f28940ab050 2023-10-06T22:13:20.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.484+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] conn(0x7f28940ab050 msgr2=0x7f28940ad4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.484+0000 7f28af5a4700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] conn(0x7f28940ab050 0x7f28940ad4f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.484+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28940a1590 msgr2=0x7f28940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.484+0000 7f28af5a4700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f28940a1590 0x7f28940a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009e70 tx=0x7f28a4009260 comp rx=0 tx=0).stop 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.484+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8106f90 msgr2=0x7f28a819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.484+0000 7f28af5a4700 1 --2- 172.21.15.57:0/1938760039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28a8106f90 0x7f28a819ba10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28a0007f10 tx=0x7f28a000d3b0 comp rx=0 tx=0).stop 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.484+0000 7f28adb41700 1 -- 172.21.15.57:0/1938760039 reap_dead start 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.485+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 shutdown_connections 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.485+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 >> 172.21.15.57:0/1938760039 conn(0x7f28a80780a0 msgr2=0x7f28a80ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.485+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 shutdown_connections 2023-10-06T22:13:20.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:20.486+0000 7f28af5a4700 1 -- 172.21.15.57:0/1938760039 wait complete. 2023-10-06T22:13:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:20.621 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:21.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:20 smithi190 ceph-mon[134891]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:20 smithi057 ceph-mon[98492]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:21.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:20 smithi156 ceph-mon[134358]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:21.254 INFO:teuthology.orchestra.run.smithi057.stdout:283467841564 2023-10-06T22:13:21.255 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-06T22:13:21.361 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:21.430 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 -- 172.21.15.57:0/2091620824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e94073620 msgr2=0x7f9e94073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 --2- 172.21.15.57:0/2091620824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e94073620 0x7f9e94073a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009990 tx=0x7f9e88009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 -- 172.21.15.57:0/2091620824 shutdown_connections 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 --2- 172.21.15.57:0/2091620824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e94111bc0 0x7f9e94113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 --2- 172.21.15.57:0/2091620824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e94074ef0 0x7f9e940730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 --2- 172.21.15.57:0/2091620824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e94073620 0x7f9e94073a80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 -- 172.21.15.57:0/2091620824 >> 172.21.15.57:0/2091620824 conn(0x7f9e940780a0 msgr2=0x7f9e940784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 -- 172.21.15.57:0/2091620824 shutdown_connections 2023-10-06T22:13:21.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.590+0000 7f9e98e44700 1 -- 172.21.15.57:0/2091620824 wait complete. 2023-10-06T22:13:21.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.592+0000 7f9e98e44700 1 Processor -- start 2023-10-06T22:13:21.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.592+0000 7f9e98e44700 1 -- start start 2023-10-06T22:13:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e98e44700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e94073620 0x7f9e9419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e98e44700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e94074ef0 0x7f9e9419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e98e44700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e94111bc0 0x7f9e941a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e98e44700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e941a2620 con 0x7f9e94111bc0 2023-10-06T22:13:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e98e44700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e941a2790 con 0x7f9e94074ef0 2023-10-06T22:13:21.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e98e44700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e941a2950 con 0x7f9e94073620 2023-10-06T22:13:21.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e91d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e94074ef0 0x7f9e9419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:21.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e9259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e94073620 0x7f9e9419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:21.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e91d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e94074ef0 0x7f9e9419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:35782/0 (socket says 172.21.15.57:35782) 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e91d9b700 1 -- 172.21.15.57:0/1974915274 learned_addr learned my addr 172.21.15.57:0/1974915274 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e92d9d700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e94111bc0 0x7f9e941a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e91d9b700 1 -- 172.21.15.57:0/1974915274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e94073620 msgr2=0x7f9e9419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.593+0000 7f9e91d9b700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e94073620 0x7f9e9419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.594+0000 7f9e91d9b700 1 -- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e94111bc0 msgr2=0x7f9e941a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.594+0000 7f9e91d9b700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e94111bc0 0x7f9e941a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.594+0000 7f9e91d9b700 1 -- 172.21.15.57:0/1974915274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880095f0 con 0x7f9e94074ef0 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.594+0000 7f9e91d9b700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e94074ef0 0x7f9e9419dba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e88005910 tx=0x7f9e88009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.594+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8801d070 con 0x7f9e94074ef0 2023-10-06T22:13:21.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.594+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e88021410 con 0x7f9e94074ef0 2023-10-06T22:13:21.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.594+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88017650 con 0x7f9e94074ef0 2023-10-06T22:13:21.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.595+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e941a2bd0 con 0x7f9e94074ef0 2023-10-06T22:13:21.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.595+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e941aa2c0 con 0x7f9e94074ef0 2023-10-06T22:13:21.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.596+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e9404ebb0 con 0x7f9e94074ef0 2023-10-06T22:13:21.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.600+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9e88021580 con 0x7f9e94074ef0 2023-10-06T22:13:21.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.601+0000 7f9e837fe700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9e7c0a15d0 0x7f9e7c0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:21.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.601+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f9e880ce6f0 con 0x7f9e94074ef0 2023-10-06T22:13:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.601+0000 7f9e837fe700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] conn(0x7f9e7c0ab090 0x7f9e7c0ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.601+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 --> [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e7c0adc00 con 0x7f9e7c0ab090 2023-10-06T22:13:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.601+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e880cea60 con 0x7f9e94074ef0 2023-10-06T22:13:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.602+0000 7f9e92d9d700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] conn(0x7f9e7c0ab090 0x7f9e7c0ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.602+0000 7f9e9259c700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9e7c0a15d0 0x7f9e7c0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:21.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.603+0000 7f9e92d9d700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] conn(0x7f9e7c0ab090 0x7f9e7c0ad530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:21.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.603+0000 7f9e9259c700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9e7c0a15d0 0x7f9e7c0a3a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009bf0 tx=0x7f9e84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:21.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.604+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== osd.0 v2:172.21.15.57:6802/2194428703 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9e7c0adc00 con 0x7f9e7c0ab090 2023-10-06T22:13:21.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.621+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 --> [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e94117840 con 0x7f9e7c0ab090 2023-10-06T22:13:21.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e837fe700 1 -- 172.21.15.57:0/1974915274 <== osd.0 v2:172.21.15.57:6802/2194428703 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e8c00f040 con 0x7f9e7c0ab090 2023-10-06T22:13:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] conn(0x7f9e7c0ab090 msgr2=0x7f9e7c0ad530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e98e44700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] conn(0x7f9e7c0ab090 0x7f9e7c0ad530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9e7c0a15d0 msgr2=0x7f9e7c0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e98e44700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9e7c0a15d0 0x7f9e7c0a3a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009bf0 tx=0x7f9e84009260 comp rx=0 tx=0).stop 2023-10-06T22:13:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e94074ef0 msgr2=0x7f9e9419dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e98e44700 1 --2- 172.21.15.57:0/1974915274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e94074ef0 0x7f9e9419dba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e88005910 tx=0x7f9e88009580 comp rx=0 tx=0).stop 2023-10-06T22:13:21.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.622+0000 7f9e92d9d700 1 -- 172.21.15.57:0/1974915274 reap_dead start 2023-10-06T22:13:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.623+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 shutdown_connections 2023-10-06T22:13:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.623+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 >> 172.21.15.57:0/1974915274 conn(0x7f9e940780a0 msgr2=0x7f9e94110680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.623+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 shutdown_connections 2023-10-06T22:13:21.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:21.623+0000 7f9e98e44700 1 -- 172.21.15.57:0/1974915274 wait complete. 2023-10-06T22:13:22.071 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:21 smithi057 ceph-mon[98492]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:22.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:21 smithi156 ceph-mon[134358]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:22.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:21 smithi190 ceph-mon[134891]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2023-10-06T22:13:22.544 INFO:teuthology.orchestra.run.smithi057.stdout:34359738439 2023-10-06T22:13:22.545 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-06T22:13:23.800 INFO:teuthology.orchestra.run.smithi057.stdout:335007449107 2023-10-06T22:13:23.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-06T22:13:23.944 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:24.421 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:24.422 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:24.527 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:24.558 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:24 smithi057 ceph-mon[98492]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 341 B/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2023-10-06T22:13:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.961+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/2888808688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4100f80 msgr2=0x7fe7f4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.961+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/2888808688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4100f80 0x7fe7f4101400 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40099c0 tx=0x7fe7e4009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.962+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/2888808688 shutdown_connections 2023-10-06T22:13:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.962+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/2888808688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f4101940 0x7fe7f410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:24.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.962+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/2888808688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f4106f90 0x7fe7f4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:24.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.962+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/2888808688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4100f80 0x7fe7f4101400 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:24.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.962+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/2888808688 >> 172.21.15.57:0/2888808688 conn(0x7fe7f40780a0 msgr2=0x7fe7f40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:24.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.962+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/2888808688 shutdown_connections 2023-10-06T22:13:24.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.962+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/2888808688 wait complete. 2023-10-06T22:13:24.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.963+0000 7fe7fb5b5700 1 Processor -- start 2023-10-06T22:13:24.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.963+0000 7fe7fb5b5700 1 -- start start 2023-10-06T22:13:24.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7fb5b5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f4100f80 0x7fe7f4197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:24.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7fb5b5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4101940 0x7fe7f4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:24.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7fb5b5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f4106f90 0x7fe7f419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:24.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7fb5b5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f419c010 con 0x7fe7f4101940 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7fb5b5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f419c180 con 0x7fe7f4106f90 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7fb5b5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f419c340 con 0x7fe7f4100f80 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7f8b50700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4101940 0x7fe7f4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7f8b50700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4101940 0x7fe7f4197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52570/0 (socket says 172.21.15.57:52570) 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7f8b50700 1 -- 172.21.15.57:0/406379616 learned_addr learned my addr 172.21.15.57:0/406379616 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7f9351700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f4100f80 0x7fe7f4197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.964+0000 7fe7f9b52700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f4106f90 0x7fe7f419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:24.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7f8b50700 1 -- 172.21.15.57:0/406379616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f4100f80 msgr2=0x7fe7f4197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:24.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7f8b50700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f4100f80 0x7fe7f4197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:24.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7f8b50700 1 -- 172.21.15.57:0/406379616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f4106f90 msgr2=0x7fe7f419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:24.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7f8b50700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f4106f90 0x7fe7f419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:24.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7f8b50700 1 -- 172.21.15.57:0/406379616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e40095f0 con 0x7fe7f4101940 2023-10-06T22:13:24.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7f8b50700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4101940 0x7fe7f4197590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009990 tx=0x7fe7e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:24.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7ea7fc700 1 -- 172.21.15.57:0/406379616 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e401d070 con 0x7fe7f4101940 2023-10-06T22:13:24.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7ea7fc700 1 -- 172.21.15.57:0/406379616 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7e40054f0 con 0x7fe7f4101940 2023-10-06T22:13:24.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f419c5c0 con 0x7fe7f4101940 2023-10-06T22:13:24.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7ea7fc700 1 -- 172.21.15.57:0/406379616 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4004ae0 con 0x7fe7f4101940 2023-10-06T22:13:24.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.965+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f41a3e80 con 0x7fe7f4101940 2023-10-06T22:13:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.966+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f404ebb0 con 0x7fe7f4101940 2023-10-06T22:13:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.968+0000 7fe7ea7fc700 1 -- 172.21.15.57:0/406379616 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe7e400fb80 con 0x7fe7f4101940 2023-10-06T22:13:24.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.970+0000 7fe7ea7fc700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7e00a1510 0x7fe7e00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:24.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.970+0000 7fe7ea7fc700 1 -- 172.21.15.57:0/406379616 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fe7e40ce050 con 0x7fe7f4101940 2023-10-06T22:13:24.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.970+0000 7fe7f9351700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7e00a1510 0x7fe7e00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:24.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.970+0000 7fe7f9351700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7e00a1510 0x7fe7e00a39d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009be0 tx=0x7fe7f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:24.972+0000 7fe7ea7fc700 1 -- 172.21.15.57:0/406379616 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe7e4097610 con 0x7fe7f4101940 2023-10-06T22:13:25.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:24 smithi156 ceph-mon[134358]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 341 B/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2023-10-06T22:13:25.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:24 smithi190 ceph-mon[134891]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 341 B/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2023-10-06T22:13:25.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.196+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fe7f4110ea0 con 0x7fe7f4101940 2023-10-06T22:13:25.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.196+0000 7fe7ea7fc700 1 -- 172.21.15.57:0/406379616 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe7e409ac30 con 0x7fe7f4101940 2023-10-06T22:13:25.199 INFO:teuthology.orchestra.run.smithi057.stdout:249108103202 2023-10-06T22:13:25.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7e00a1510 msgr2=0x7fe7e00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:25.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7e00a1510 0x7fe7e00a39d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009be0 tx=0x7fe7f0009260 comp rx=0 tx=0).stop 2023-10-06T22:13:25.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4101940 msgr2=0x7fe7f4197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:25.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4101940 0x7fe7f4197590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009990 tx=0x7fe7e400f690 comp rx=0 tx=0).stop 2023-10-06T22:13:25.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 shutdown_connections 2023-10-06T22:13:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7e00a1510 0x7fe7e00a39d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7f4106f90 0x7fe7f419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f4100f80 0x7fe7f4197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 --2- 172.21.15.57:0/406379616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7f4101940 0x7fe7f4197590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 >> 172.21.15.57:0/406379616 conn(0x7fe7f40780a0 msgr2=0x7fe7f40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 shutdown_connections 2023-10-06T22:13:25.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.199+0000 7fe7fb5b5700 1 -- 172.21.15.57:0/406379616 wait complete. 2023-10-06T22:13:25.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.293+0000 7f4f18eac700 1 -- 172.21.15.57:0/282963296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f140689a0 msgr2=0x7f4f14068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:25.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 --2- 172.21.15.57:0/282963296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f140689a0 0x7f4f14068d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009990 tx=0x7f4f04009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 -- 172.21.15.57:0/282963296 shutdown_connections 2023-10-06T22:13:25.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 --2- 172.21.15.57:0/282963296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f1410da10 0x7f4f1410fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 --2- 172.21.15.57:0/282963296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f140689a0 0x7f4f14068d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 --2- 172.21.15.57:0/282963296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 0x7f4f1410d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 -- 172.21.15.57:0/282963296 >> 172.21.15.57:0/282963296 conn(0x7f4f14075e00 msgr2=0x7f4f14076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:25.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 -- 172.21.15.57:0/282963296 shutdown_connections 2023-10-06T22:13:25.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.294+0000 7f4f18eac700 1 -- 172.21.15.57:0/282963296 wait complete. 2023-10-06T22:13:25.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.295+0000 7f4f18eac700 1 Processor -- start 2023-10-06T22:13:25.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.295+0000 7f4f18eac700 1 -- start start 2023-10-06T22:13:25.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f18eac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f140689a0 0x7f4f14199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:25.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f18eac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 0x7f4f141997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:25.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f18eac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f1410da10 0x7f4f1419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:25.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f18eac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419e230 con 0x7f4f1410da10 2023-10-06T22:13:25.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f18eac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419e3a0 con 0x7f4f140689a0 2023-10-06T22:13:25.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f18eac700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419e560 con 0x7f4f140692c0 2023-10-06T22:13:25.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f11d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 0x7f4f141997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:25.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f11d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 0x7f4f141997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:37760/0 (socket says 172.21.15.57:37760) 2023-10-06T22:13:25.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f1259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f140689a0 0x7f4f14199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:25.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f12d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f1410da10 0x7f4f1419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:25.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f11d9b700 1 -- 172.21.15.57:0/1953816331 learned_addr learned my addr 172.21.15.57:0/1953816331 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:25.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.296+0000 7f4f11d9b700 1 -- 172.21.15.57:0/1953816331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f140689a0 msgr2=0x7f4f14199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:25.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f11d9b700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f140689a0 0x7f4f14199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f11d9b700 1 -- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f1410da10 msgr2=0x7f4f1419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:25.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f11d9b700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f1410da10 0x7f4f1419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f11d9b700 1 -- 172.21.15.57:0/1953816331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f040095f0 con 0x7f4f140692c0 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f11d9b700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 0x7f4f141997b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9a0 tx=0x7f4f0800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f037fe700 1 -- 172.21.15.57:0/1953816331 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800ca10 con 0x7f4f140692c0 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1419e840 con 0x7f4f140692c0 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f037fe700 1 -- 172.21.15.57:0/1953816331 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0800cb70 con 0x7f4f140692c0 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f037fe700 1 -- 172.21.15.57:0/1953816331 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08018680 con 0x7f4f140692c0 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.297+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f141a60a0 con 0x7f4f140692c0 2023-10-06T22:13:25.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.299+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1404ebb0 con 0x7f4f140692c0 2023-10-06T22:13:25.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.302+0000 7f4f037fe700 1 -- 172.21.15.57:0/1953816331 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4f0800cce0 con 0x7f4f140692c0 2023-10-06T22:13:25.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.303+0000 7f4f037fe700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4efc0a1510 0x7f4efc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:25.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.304+0000 7f4f037fe700 1 -- 172.21.15.57:0/1953816331 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f4f08014070 con 0x7f4f140692c0 2023-10-06T22:13:25.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.304+0000 7f4f1259c700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4efc0a1510 0x7f4efc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:25.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.305+0000 7f4f1259c700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4efc0a1510 0x7f4efc0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400b2e0 tx=0x7f4f04005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:25.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.308+0000 7f4f037fe700 1 -- 172.21.15.57:0/1953816331 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4f08096580 con 0x7f4f140692c0 2023-10-06T22:13:25.381 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:25.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.572+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4f14113290 con 0x7f4f140692c0 2023-10-06T22:13:25.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.573+0000 7f4f037fe700 1 -- 172.21.15.57:0/1953816331 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f08099ba0 con 0x7f4f140692c0 2023-10-06T22:13:25.576 INFO:teuthology.orchestra.run.smithi057.stdout:219043332133 2023-10-06T22:13:25.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4efc0a1510 msgr2=0x7f4efc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:25.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4efc0a1510 0x7f4efc0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400b2e0 tx=0x7f4f04005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 msgr2=0x7f4f141997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:25.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 0x7f4f141997b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9a0 tx=0x7f4f0800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 shutdown_connections 2023-10-06T22:13:25.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4efc0a1510 0x7f4efc0a39d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f1410da10 0x7f4f1419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f140689a0 0x7f4f14199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 --2- 172.21.15.57:0/1953816331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f140692c0 0x7f4f141997b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:25.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.575+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 >> 172.21.15.57:0/1953816331 conn(0x7f4f14075e00 msgr2=0x7f4f1410ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:25.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.576+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 shutdown_connections 2023-10-06T22:13:25.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:25.576+0000 7f4f18eac700 1 -- 172.21.15.57:0/1953816331 wait complete. 2023-10-06T22:13:25.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:25 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/406379616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T22:13:25.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:25 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1953816331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T22:13:26.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:25 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/406379616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T22:13:26.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:25 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1953816331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T22:13:26.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:25 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/406379616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T22:13:26.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:25 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1953816331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T22:13:26.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:26 smithi057 ceph-mon[98492]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:27.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:26 smithi156 ceph-mon[134358]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:26 smithi190 ceph-mon[134891]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:27 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1045289389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T22:13:28.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:27 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1045289389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T22:13:28.048 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332132 got 219043332133 for osd.7 2023-10-06T22:13:28.049 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:28.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:27 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1045289389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T22:13:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.706+0000 7f165d9d2700 1 -- 172.21.15.57:0/3427392769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 msgr2=0x7f1658101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.706+0000 7f165d9d2700 1 --2- 172.21.15.57:0/3427392769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658101400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f164c0099c0 tx=0x7f164c009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.707+0000 7f165d9d2700 1 -- 172.21.15.57:0/3427392769 shutdown_connections 2023-10-06T22:13:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.707+0000 7f165d9d2700 1 --2- 172.21.15.57:0/3427392769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1658101940 0x7f165810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.707+0000 7f165d9d2700 1 --2- 172.21.15.57:0/3427392769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658106f90 0x7f1658107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.707+0000 7f165d9d2700 1 --2- 172.21.15.57:0/3427392769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658101400 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.707+0000 7f165d9d2700 1 -- 172.21.15.57:0/3427392769 >> 172.21.15.57:0/3427392769 conn(0x7f16580780a0 msgr2=0x7f16580784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.707+0000 7f165d9d2700 1 -- 172.21.15.57:0/3427392769 shutdown_connections 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.707+0000 7f165d9d2700 1 -- 172.21.15.57:0/3427392769 wait complete. 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 Processor -- start 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 -- start start 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101940 0x7f16581974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1658106f90 0x7f165819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165819bf70 con 0x7f1658100f80 2023-10-06T22:13:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165819c0e0 con 0x7f1658101940 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.708+0000 7f165d9d2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165819c2a0 con 0x7f1658106f90 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f16567fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101940 0x7f16581974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52610/0 (socket says 172.21.15.57:52610) 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 -- 172.21.15.57:0/1045289389 learned_addr learned my addr 172.21.15.57:0/1045289389 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f16577fe700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1658106f90 0x7f165819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 -- 172.21.15.57:0/1045289389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1658106f90 msgr2=0x7f165819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1658106f90 0x7f165819b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 -- 172.21.15.57:0/1045289389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101940 msgr2=0x7f16581974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101940 0x7f16581974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 -- 172.21.15.57:0/1045289389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c0095f0 con 0x7f1658100f80 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.709+0000 7f1656ffd700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658196fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f164800d950 tx=0x7f164800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:28.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.710+0000 7f165c9d0700 1 -- 172.21.15.57:0/1045289389 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164800f890 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.710+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165819c580 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.710+0000 7f165c9d0700 1 -- 172.21.15.57:0/1045289389 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1648010460 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.710+0000 7f165c9d0700 1 -- 172.21.15.57:0/1045289389 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164800e610 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.710+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16580732b0 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.711+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165804ebb0 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.713+0000 7f165c9d0700 1 -- 172.21.15.57:0/1045289389 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f16480108d0 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.714+0000 7f165c9d0700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f16400a1510 0x7f16400a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.714+0000 7f165c9d0700 1 -- 172.21.15.57:0/1045289389 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f16480ccc50 con 0x7f1658100f80 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.714+0000 7f16567fc700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f16400a1510 0x7f16400a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.715+0000 7f16567fc700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f16400a1510 0x7f16400a39d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f164c009990 tx=0x7f164c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:28.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.719+0000 7f165c9d0700 1 -- 172.21.15.57:0/1045289389 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1648096120 con 0x7f1658100f80 2023-10-06T22:13:28.207 INFO:teuthology.orchestra.run.smithi057.stdout:77309411390 2023-10-06T22:13:28.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.976+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1658198310 con 0x7f1658100f80 2023-10-06T22:13:28.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.976+0000 7f165c9d0700 1 -- 172.21.15.57:0/1045289389 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1648016090 con 0x7f1658100f80 2023-10-06T22:13:28.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f16400a1510 msgr2=0x7f16400a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:28.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f16400a1510 0x7f16400a39d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f164c009990 tx=0x7f164c003680 comp rx=0 tx=0).stop 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 msgr2=0x7f1658196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658196fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f164800d950 tx=0x7f164800dc60 comp rx=0 tx=0).stop 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 shutdown_connections 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f16400a1510 0x7f16400a39d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1658106f90 0x7f165819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1658100f80 0x7f1658196fb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 --2- 172.21.15.57:0/1045289389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1658101940 0x7f16581974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.978+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 >> 172.21.15.57:0/1045289389 conn(0x7f16580780a0 msgr2=0x7f16580ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.979+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 shutdown_connections 2023-10-06T22:13:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:26.979+0000 7f165d9d2700 1 -- 172.21.15.57:0/1045289389 wait complete. 2023-10-06T22:13:28.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:28 smithi057 ceph-mon[98492]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:29.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:28 smithi156 ceph-mon[134358]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:29.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:28 smithi190 ceph-mon[134891]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:29.601 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411387 got 77309411390 for osd.2 2023-10-06T22:13:29.601 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.783+0000 7fe98a17c700 1 -- 172.21.15.57:0/2439482036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984106f90 msgr2=0x7fe984107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.783+0000 7fe98a17c700 1 --2- 172.21.15.57:0/2439482036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984106f90 0x7fe984107370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe9740099c0 tx=0x7fe974009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.784+0000 7fe98a17c700 1 -- 172.21.15.57:0/2439482036 shutdown_connections 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.784+0000 7fe98a17c700 1 --2- 172.21.15.57:0/2439482036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe984101940 0x7fe98410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.784+0000 7fe98a17c700 1 --2- 172.21.15.57:0/2439482036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984106f90 0x7fe984107370 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.784+0000 7fe98a17c700 1 --2- 172.21.15.57:0/2439482036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe984100f80 0x7fe984101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.784+0000 7fe98a17c700 1 -- 172.21.15.57:0/2439482036 >> 172.21.15.57:0/2439482036 conn(0x7fe9840780a0 msgr2=0x7fe9840784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.784+0000 7fe98a17c700 1 -- 172.21.15.57:0/2439482036 shutdown_connections 2023-10-06T22:13:29.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.784+0000 7fe98a17c700 1 -- 172.21.15.57:0/2439482036 wait complete. 2023-10-06T22:13:29.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.785+0000 7fe98a17c700 1 Processor -- start 2023-10-06T22:13:29.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.785+0000 7fe98a17c700 1 -- start start 2023-10-06T22:13:29.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe98a17c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984100f80 0x7fe9841991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:29.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe98a17c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe984101940 0x7fe984199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:29.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe98a17c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe984106f90 0x7fe98419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:29.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe98a17c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98419e190 con 0x7fe984100f80 2023-10-06T22:13:29.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe98a17c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98419e300 con 0x7fe984106f90 2023-10-06T22:13:29.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe98a17c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98419e4c0 con 0x7fe984101940 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe9837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984100f80 0x7fe9841991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe9837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984100f80 0x7fe9841991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52636/0 (socket says 172.21.15.57:52636) 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe9837fe700 1 -- 172.21.15.57:0/3197854639 learned_addr learned my addr 172.21.15.57:0/3197854639 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.786+0000 7fe983fff700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe984106f90 0x7fe98419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe982ffd700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe984101940 0x7fe984199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe9837fe700 1 -- 172.21.15.57:0/3197854639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe984101940 msgr2=0x7fe984199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe9837fe700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe984101940 0x7fe984199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:29.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe9837fe700 1 -- 172.21.15.57:0/3197854639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe984106f90 msgr2=0x7fe98419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe9837fe700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe984106f90 0x7fe98419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe9837fe700 1 -- 172.21.15.57:0/3197854639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740095f0 con 0x7fe984100f80 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe982ffd700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe984101940 0x7fe984199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe9837fe700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984100f80 0x7fe9841991d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9740098e0 tx=0x7fe97400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe980ff9700 1 -- 172.21.15.57:0/3197854639 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97401d070 con 0x7fe984100f80 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe980ff9700 1 -- 172.21.15.57:0/3197854639 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97400fb80 con 0x7fe984100f80 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98419e740 con 0x7fe984100f80 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.787+0000 7fe980ff9700 1 -- 172.21.15.57:0/3197854639 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9740173f0 con 0x7fe984100f80 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.788+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9841a6000 con 0x7fe984100f80 2023-10-06T22:13:29.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.789+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98404ebb0 con 0x7fe984100f80 2023-10-06T22:13:29.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.791+0000 7fe980ff9700 1 -- 172.21.15.57:0/3197854639 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe97400fcf0 con 0x7fe984100f80 2023-10-06T22:13:29.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.792+0000 7fe980ff9700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe96c0a1510 0x7fe96c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:29.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.792+0000 7fe980ff9700 1 -- 172.21.15.57:0/3197854639 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fe9740cdba0 con 0x7fe984100f80 2023-10-06T22:13:29.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.792+0000 7fe982ffd700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe96c0a1510 0x7fe96c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:29.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.793+0000 7fe982ffd700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe96c0a1510 0x7fe96c0a39d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe98419a7f0 tx=0x7fe978009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:29.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:29.796+0000 7fe980ff9700 1 -- 172.21.15.57:0/3197854639 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe974097100 con 0x7fe984100f80 2023-10-06T22:13:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.012+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe984110ea0 con 0x7fe984100f80 2023-10-06T22:13:30.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.012+0000 7fe980ff9700 1 -- 172.21.15.57:0/3197854639 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe97409a720 con 0x7fe984100f80 2023-10-06T22:13:30.015 INFO:teuthology.orchestra.run.smithi057.stdout:163208757298 2023-10-06T22:13:30.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.014+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe96c0a1510 msgr2=0x7fe96c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:30.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.014+0000 7fe98a17c700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe96c0a1510 0x7fe96c0a39d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe98419a7f0 tx=0x7fe978009260 comp rx=0 tx=0).stop 2023-10-06T22:13:30.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.014+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984100f80 msgr2=0x7fe9841991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:30.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.014+0000 7fe98a17c700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984100f80 0x7fe9841991d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9740098e0 tx=0x7fe97400f690 comp rx=0 tx=0).stop 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 shutdown_connections 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe96c0a1510 0x7fe96c0a39d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe984106f90 0x7fe98419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe984100f80 0x7fe9841991d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 --2- 172.21.15.57:0/3197854639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe984101940 0x7fe984199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 >> 172.21.15.57:0/3197854639 conn(0x7fe9840780a0 msgr2=0x7fe9840ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 shutdown_connections 2023-10-06T22:13:30.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.015+0000 7fe98a17c700 1 -- 172.21.15.57:0/3197854639 wait complete. 2023-10-06T22:13:30.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.647+0000 7f494e4cc700 1 -- 172.21.15.57:0/2585307419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4948073a00 msgr2=0x7f494810d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:30.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.647+0000 7f494e4cc700 1 --2- 172.21.15.57:0/2585307419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4948073a00 0x7f494810d3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f493c009990 tx=0x7f493c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.648+0000 7f494e4cc700 1 -- 172.21.15.57:0/2585307419 shutdown_connections 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.648+0000 7f494e4cc700 1 --2- 172.21.15.57:0/2585307419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f494810d930 0x7f494810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.648+0000 7f494e4cc700 1 --2- 172.21.15.57:0/2585307419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49480730e0 0x7f49480734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.648+0000 7f494e4cc700 1 --2- 172.21.15.57:0/2585307419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4948073a00 0x7f494810d3f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.648+0000 7f494e4cc700 1 -- 172.21.15.57:0/2585307419 >> 172.21.15.57:0/2585307419 conn(0x7f49480780a0 msgr2=0x7f49480784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.648+0000 7f494e4cc700 1 -- 172.21.15.57:0/2585307419 shutdown_connections 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.648+0000 7f494e4cc700 1 -- 172.21.15.57:0/2585307419 wait complete. 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 Processor -- start 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 -- start start 2023-10-06T22:13:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49480730e0 0x7f4948199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4948073a00 0x7f4948199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494810d930 0x7f494819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494819e210 con 0x7f494810d930 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494819e380 con 0x7f49480730e0 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.649+0000 7f494e4cc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494819e540 con 0x7f4948073a00 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49480730e0 0x7f4948199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49480730e0 0x7f4948199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:56968/0 (socket says 172.21.15.57:56968) 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 -- 172.21.15.57:0/466046083 learned_addr learned my addr 172.21.15.57:0/466046083 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f49477fe700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4948073a00 0x7f4948199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f494ca69700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494810d930 0x7f494819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 -- 172.21.15.57:0/466046083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4948073a00 msgr2=0x7f4948199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4948073a00 0x7f4948199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 -- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494810d930 msgr2=0x7f494819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494810d930 0x7f494819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 -- 172.21.15.57:0/466046083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c0095f0 con 0x7f49480730e0 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f49477fe700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4948073a00 0x7f4948199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.650+0000 7f4947fff700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49480730e0 0x7f4948199250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f493800e9a0 tx=0x7f493800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.651+0000 7f49457fa700 1 -- 172.21.15.57:0/466046083 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938009790 con 0x7f49480730e0 2023-10-06T22:13:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.651+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494819e820 con 0x7f49480730e0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.651+0000 7f49457fa700 1 -- 172.21.15.57:0/466046083 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493800cbd0 con 0x7f49480730e0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.651+0000 7f49457fa700 1 -- 172.21.15.57:0/466046083 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938018680 con 0x7f49480730e0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.651+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49481a6080 con 0x7f49480730e0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.652+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494804ebb0 con 0x7f49480730e0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.654+0000 7f49457fa700 1 -- 172.21.15.57:0/466046083 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f493800cd40 con 0x7f49480730e0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.655+0000 7f49457fa700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f49300a1510 0x7f49300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.656+0000 7f49457fa700 1 -- 172.21.15.57:0/466046083 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f4938014070 con 0x7f49480730e0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.656+0000 7f49477fe700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f49300a1510 0x7f49300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:30.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.657+0000 7f49477fe700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f49300a1510 0x7f49300a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f494819a870 tx=0x7f493c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.658+0000 7f49457fa700 1 -- 172.21.15.57:0/466046083 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f49380968e0 con 0x7f49480730e0 2023-10-06T22:13:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.899+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4948113200 con 0x7f49480730e0 2023-10-06T22:13:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.900+0000 7f49457fa700 1 -- 172.21.15.57:0/466046083 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4938099f00 con 0x7f49480730e0 2023-10-06T22:13:30.903 INFO:teuthology.orchestra.run.smithi057.stdout:141733920821 2023-10-06T22:13:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f49300a1510 msgr2=0x7f49300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f49300a1510 0x7f49300a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f494819a870 tx=0x7f493c005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49480730e0 msgr2=0x7f4948199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49480730e0 0x7f4948199250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f493800e9a0 tx=0x7f493800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 shutdown_connections 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f49300a1510 0x7f49300a39d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f494810d930 0x7f494819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49480730e0 0x7f4948199250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 --2- 172.21.15.57:0/466046083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4948073a00 0x7f4948199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 >> 172.21.15.57:0/466046083 conn(0x7f49480780a0 msgr2=0x7f49480fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 shutdown_connections 2023-10-06T22:13:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:30.903+0000 7f494e4cc700 1 -- 172.21.15.57:0/466046083 wait complete. 2023-10-06T22:13:30.976 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:30 smithi057 ceph-mon[98492]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:30.976 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:30 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3197854639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T22:13:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:30 smithi156 ceph-mon[134358]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:31.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:30 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3197854639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T22:13:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:30 smithi190 ceph-mon[134891]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:30 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3197854639' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T22:13:31.204 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757296 got 163208757298 for osd.5 2023-10-06T22:13:31.205 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 -- 172.21.15.57:0/2344698113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230106f90 msgr2=0x7f5230107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 --2- 172.21.15.57:0/2344698113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230106f90 0x7f5230107370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f522c009990 tx=0x7f522c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 -- 172.21.15.57:0/2344698113 shutdown_connections 2023-10-06T22:13:31.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 --2- 172.21.15.57:0/2344698113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230101940 0x7f523010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 --2- 172.21.15.57:0/2344698113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230106f90 0x7f5230107370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 --2- 172.21.15.57:0/2344698113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5230100f80 0x7f5230101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 -- 172.21.15.57:0/2344698113 >> 172.21.15.57:0/2344698113 conn(0x7f52300780a0 msgr2=0x7f52300784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:31.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.448+0000 7f523766a700 1 -- 172.21.15.57:0/2344698113 shutdown_connections 2023-10-06T22:13:31.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.449+0000 7f523766a700 1 -- 172.21.15.57:0/2344698113 wait complete. 2023-10-06T22:13:31.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 Processor -- start 2023-10-06T22:13:31.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 -- start start 2023-10-06T22:13:31.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5230100f80 0x7f5230199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:31.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230101940 0x7f5230199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230106f90 0x7f523019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523019e170 con 0x7f5230100f80 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523019e2e0 con 0x7f5230106f90 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.450+0000 7f523766a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523019e4a0 con 0x7f5230101940 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230106f90 0x7f523019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5234c05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230101940 0x7f5230199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230106f90 0x7f523019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57000/0 (socket says 172.21.15.57:57000) 2023-10-06T22:13:31.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 -- 172.21.15.57:0/1353418212 learned_addr learned my addr 172.21.15.57:0/1353418212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5234c05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230101940 0x7f5230199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:37856/0 (socket says 172.21.15.57:37856) 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235406700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5230100f80 0x7f5230199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 -- 172.21.15.57:0/1353418212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230101940 msgr2=0x7f5230199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230101940 0x7f5230199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 -- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5230100f80 msgr2=0x7f5230199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5230100f80 0x7f5230199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 -- 172.21.15.57:0/1353418212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5220009520 con 0x7f5230106f90 2023-10-06T22:13:31.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5234c05700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230101940 0x7f5230199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:31.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.451+0000 7f5235c07700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230106f90 0x7f523019db60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f522800efc0 tx=0x7f522800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:31.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.452+0000 7f52267fc700 1 -- 172.21.15.57:0/1353418212 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228018070 con 0x7f5230106f90 2023-10-06T22:13:31.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.452+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c0095f0 con 0x7f5230106f90 2023-10-06T22:13:31.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.452+0000 7f52267fc700 1 -- 172.21.15.57:0/1353418212 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f522800f040 con 0x7f5230106f90 2023-10-06T22:13:31.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.452+0000 7f52267fc700 1 -- 172.21.15.57:0/1353418212 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800cc10 con 0x7f5230106f90 2023-10-06T22:13:31.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.452+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52301a5e10 con 0x7f5230106f90 2023-10-06T22:13:31.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.453+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523004ebb0 con 0x7f5230106f90 2023-10-06T22:13:31.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.456+0000 7f52267fc700 1 -- 172.21.15.57:0/1353418212 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f522800cd90 con 0x7f5230106f90 2023-10-06T22:13:31.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.459+0000 7f52267fc700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f521c0a1510 0x7f521c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:31.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.459+0000 7f52267fc700 1 -- 172.21.15.57:0/1353418212 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f52280cd240 con 0x7f5230106f90 2023-10-06T22:13:31.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.459+0000 7f5235406700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f521c0a1510 0x7f521c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:31.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.460+0000 7f52267fc700 1 -- 172.21.15.57:0/1353418212 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f52280967a0 con 0x7f5230106f90 2023-10-06T22:13:31.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.460+0000 7f5235406700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f521c0a1510 0x7f521c0a39d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f522c00b2e0 tx=0x7f522c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:31.470 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103201 got 249108103202 for osd.8 2023-10-06T22:13:31.471 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:31.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.680+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5230110fa0 con 0x7f5230106f90 2023-10-06T22:13:31.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.681+0000 7f52267fc700 1 -- 172.21.15.57:0/1353418212 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5228099dc0 con 0x7f5230106f90 2023-10-06T22:13:31.683 INFO:teuthology.orchestra.run.smithi057.stdout:107374182458 2023-10-06T22:13:31.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f521c0a1510 msgr2=0x7f521c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:31.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f521c0a1510 0x7f521c0a39d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f522c00b2e0 tx=0x7f522c005fd0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230106f90 msgr2=0x7f523019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230106f90 0x7f523019db60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f522800efc0 tx=0x7f522800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 shutdown_connections 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f521c0a1510 0x7f521c0a39d0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f522c00b2e0 tx=0x7f522c005fd0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5230106f90 0x7f523019db60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5230100f80 0x7f5230199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 --2- 172.21.15.57:0/1353418212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5230101940 0x7f5230199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 >> 172.21.15.57:0/1353418212 conn(0x7f52300780a0 msgr2=0x7f52300ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 shutdown_connections 2023-10-06T22:13:31.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f523766a700 1 -- 172.21.15.57:0/1353418212 wait complete. 2023-10-06T22:13:31.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:31 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/466046083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T22:13:31.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:31 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1353418212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T22:13:32.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:31 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/466046083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T22:13:32.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:31 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1353418212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 -- 172.21.15.57:0/1106809527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 msgr2=0x7f223c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 --2- 172.21.15.57:0/1106809527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c0734c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2230009990 tx=0x7f2230009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 -- 172.21.15.57:0/1106809527 shutdown_connections 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 --2- 172.21.15.57:0/1106809527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f223c10d930 0x7f223c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 --2- 172.21.15.57:0/1106809527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c0734c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 --2- 172.21.15.57:0/1106809527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c073a00 0x7f223c10d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 -- 172.21.15.57:0/1106809527 >> 172.21.15.57:0/1106809527 conn(0x7f223c0780a0 msgr2=0x7f223c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 -- 172.21.15.57:0/1106809527 shutdown_connections 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.679+0000 7f2242f00700 1 -- 172.21.15.57:0/1106809527 wait complete. 2023-10-06T22:13:32.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.681+0000 7f2242f00700 1 Processor -- start 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.681+0000 7f2242f00700 1 -- start start 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2242f00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2242f00700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f223c073a00 0x7f223c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2242f00700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c10d930 0x7f223c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2242f00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c19e1a0 con 0x7f223c0730e0 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2242f00700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c19e310 con 0x7f223c10d930 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2242f00700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c19e4d0 con 0x7f223c073a00 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2240c9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2240c9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52668/0 (socket says 172.21.15.57:52668) 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2240c9c700 1 -- 172.21.15.57:0/3653544235 learned_addr learned my addr 172.21.15.57:0/3653544235 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f224149d700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c10d930 0x7f223c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f223bfff700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f223c073a00 0x7f223c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f224149d700 1 -- 172.21.15.57:0/3653544235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f223c073a00 msgr2=0x7f223c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f224149d700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f223c073a00 0x7f223c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f224149d700 1 -- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 msgr2=0x7f223c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f224149d700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f224149d700 1 -- 172.21.15.57:0/3653544235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300095f0 con 0x7f223c10d930 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.682+0000 7f2240c9c700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.683+0000 7f224149d700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c10d930 0x7f223c19db90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f223400ef90 tx=0x7f223400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.683+0000 7f2239ffb700 1 -- 172.21.15.57:0/3653544235 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234018070 con 0x7f223c10d930 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.683+0000 7f2239ffb700 1 -- 172.21.15.57:0/3653544235 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223400f040 con 0x7f223c10d930 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.683+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c19e750 con 0x7f223c10d930 2023-10-06T22:13:32.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.683+0000 7f2239ffb700 1 -- 172.21.15.57:0/3653544235 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223400cb80 con 0x7f223c10d930 2023-10-06T22:13:32.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.683+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c1a6010 con 0x7f223c10d930 2023-10-06T22:13:32.030 INFO:teuthology.orchestra.run.smithi057.stdout:304942678041 2023-10-06T22:13:32.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.684+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c04ebb0 con 0x7f223c10d930 2023-10-06T22:13:32.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.686+0000 7f2239ffb700 1 -- 172.21.15.57:0/3653544235 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2234003a00 con 0x7f223c10d930 2023-10-06T22:13:32.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.688+0000 7f2239ffb700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f222c0a1590 0x7f222c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.688+0000 7f2239ffb700 1 -- 172.21.15.57:0/3653544235 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f22340cc9c0 con 0x7f223c10d930 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.688+0000 7f2240c9c700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f222c0a1590 0x7f222c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.689+0000 7f2240c9c700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f222c0a1590 0x7f222c0a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2230006010 tx=0x7f223000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.690+0000 7f2239ffb700 1 -- 172.21.15.57:0/3653544235 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2234095f20 con 0x7f223c10d930 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.916+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f223c19a540 con 0x7f223c10d930 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.917+0000 7f2239ffb700 1 -- 172.21.15.57:0/3653544235 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f223401ce80 con 0x7f223c10d930 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f222c0a1590 msgr2=0x7f222c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f222c0a1590 0x7f222c0a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2230006010 tx=0x7f223000b260 comp rx=0 tx=0).stop 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c10d930 msgr2=0x7f223c19db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c10d930 0x7f223c19db90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f223400ef90 tx=0x7f223400c3a0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 shutdown_connections 2023-10-06T22:13:32.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f222c0a1590 0x7f222c0a3a50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c10d930 0x7f223c19db90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f223c0730e0 0x7f223c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 --2- 172.21.15.57:0/3653544235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f223c073a00 0x7f223c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 >> 172.21.15.57:0/3653544235 conn(0x7f223c0780a0 msgr2=0x7f223c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:32.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 shutdown_connections 2023-10-06T22:13:32.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:31.919+0000 7f2242f00700 1 -- 172.21.15.57:0/3653544235 wait complete. 2023-10-06T22:13:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:31 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/466046083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T22:13:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:31 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1353418212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T22:13:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 -- 172.21.15.57:0/3739564343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 msgr2=0x7f11a80730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 --2- 172.21.15.57:0/3739564343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 0x7f11a80730e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009990 tx=0x7f11a4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 -- 172.21.15.57:0/3739564343 shutdown_connections 2023-10-06T22:13:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 --2- 172.21.15.57:0/3739564343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8111bc0 0x7f11a8113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 --2- 172.21.15.57:0/3739564343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 0x7f11a80730e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 --2- 172.21.15.57:0/3739564343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a8073620 0x7f11a8073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 -- 172.21.15.57:0/3739564343 >> 172.21.15.57:0/3739564343 conn(0x7f11a80780a0 msgr2=0x7f11a80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.330+0000 7f11afb3f700 1 -- 172.21.15.57:0/3739564343 shutdown_connections 2023-10-06T22:13:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.331+0000 7f11afb3f700 1 -- 172.21.15.57:0/3739564343 wait complete. 2023-10-06T22:13:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.331+0000 7f11afb3f700 1 Processor -- start 2023-10-06T22:13:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11afb3f700 1 -- start start 2023-10-06T22:13:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11afb3f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8073620 0x7f11a819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11afb3f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 0x7f11a819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11afb3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a8111bc0 0x7f11a81a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11afb3f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81a2620 con 0x7f11a8074ef0 2023-10-06T22:13:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11afb3f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81a2790 con 0x7f11a8111bc0 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11afb3f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81a2950 con 0x7f11a8073620 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11ad8db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8073620 0x7f11a819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.332+0000 7f11ad0da700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 0x7f11a819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8073620 0x7f11a819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45716/0 (socket says 172.21.15.57:45716) 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 -- 172.21.15.57:0/2039925233 learned_addr learned my addr 172.21.15.57:0/2039925233 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ae0dc700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a8111bc0 0x7f11a81a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 -- 172.21.15.57:0/2039925233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a8111bc0 msgr2=0x7f11a81a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a8111bc0 0x7f11a81a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 -- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 msgr2=0x7f11a819dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 0x7f11a819dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 -- 172.21.15.57:0/2039925233 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1198009520 con 0x7f11a8073620 2023-10-06T22:13:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad0da700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 0x7f11a819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f11ad8db700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8073620 0x7f11a819d6b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11a4005910 tx=0x7f11a4005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.333+0000 7f119effd700 1 -- 172.21.15.57:0/2039925233 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a401d070 con 0x7f11a8073620 2023-10-06T22:13:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.334+0000 7f119effd700 1 -- 172.21.15.57:0/2039925233 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11a4022470 con 0x7f11a8073620 2023-10-06T22:13:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.334+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a0009c40 con 0x7f11a8073620 2023-10-06T22:13:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.334+0000 7f119effd700 1 -- 172.21.15.57:0/2039925233 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4017650 con 0x7f11a8073620 2023-10-06T22:13:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.334+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a40095f0 con 0x7f11a8073620 2023-10-06T22:13:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.335+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a804ebb0 con 0x7f11a8073620 2023-10-06T22:13:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.339+0000 7f119effd700 1 -- 172.21.15.57:0/2039925233 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f11a40225e0 con 0x7f11a8073620 2023-10-06T22:13:32.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.340+0000 7f119effd700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f11940a1590 0x7f11940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.340+0000 7f119effd700 1 -- 172.21.15.57:0/2039925233 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f11a40ce7b0 con 0x7f11a8073620 2023-10-06T22:13:32.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.341+0000 7f11ad0da700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f11940a1590 0x7f11940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.341+0000 7f119effd700 1 -- 172.21.15.57:0/2039925233 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f11a4097d70 con 0x7f11a8073620 2023-10-06T22:13:32.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.341+0000 7f11ad0da700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f11940a1590 0x7f11940a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1198009fd0 tx=0x7f1198009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:32.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.581+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f11a819e9c0 con 0x7f11a8073620 2023-10-06T22:13:32.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.581+0000 7f119effd700 1 -- 172.21.15.57:0/2039925233 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11a409b390 con 0x7f11a8073620 2023-10-06T22:13:32.584 INFO:teuthology.orchestra.run.smithi057.stdout:197568495660 2023-10-06T22:13:32.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f11940a1590 msgr2=0x7f11940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f11940a1590 0x7f11940a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1198009fd0 tx=0x7f1198009260 comp rx=0 tx=0).stop 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8073620 msgr2=0x7f11a819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8073620 0x7f11a819d6b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11a4005910 tx=0x7f11a4005720 comp rx=0 tx=0).stop 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 shutdown_connections 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f11940a1590 0x7f11940a3a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a8111bc0 0x7f11a81a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11a8073620 0x7f11a819d6b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 --2- 172.21.15.57:0/2039925233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11a8074ef0 0x7f11a819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 >> 172.21.15.57:0/2039925233 conn(0x7f11a80780a0 msgr2=0x7f11a8110720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 shutdown_connections 2023-10-06T22:13:32.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.584+0000 7f11afb3f700 1 -- 172.21.15.57:0/2039925233 wait complete. 2023-10-06T22:13:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:32 smithi057 ceph-mon[98492]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-10-06T22:13:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:32 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3653544235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T22:13:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:32 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/48398564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T22:13:32.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:32 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2039925233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T22:13:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.264+0000 7ff564713700 1 -- 172.21.15.57:0/3907337711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 msgr2=0x7ff55c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.264+0000 7ff564713700 1 --2- 172.21.15.57:0/3907337711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c107a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0099c0 tx=0x7ff54c009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.264+0000 7ff564713700 1 -- 172.21.15.57:0/3907337711 shutdown_connections 2023-10-06T22:13:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.264+0000 7ff564713700 1 --2- 172.21.15.57:0/3907337711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 0x7ff55c114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.264+0000 7ff564713700 1 --2- 172.21.15.57:0/3907337711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff55c10d630 0x7ff55c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.264+0000 7ff564713700 1 --2- 172.21.15.57:0/3907337711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c107a00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.264+0000 7ff564713700 1 -- 172.21.15.57:0/3907337711 >> 172.21.15.57:0/3907337711 conn(0x7ff55c075540 msgr2=0x7ff55c077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.265+0000 7ff564713700 1 -- 172.21.15.57:0/3907337711 shutdown_connections 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.265+0000 7ff564713700 1 -- 172.21.15.57:0/3907337711 wait complete. 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 Processor -- start 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 -- start start 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 0x7ff55c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff55c10d630 0x7ff55c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1a2520 con 0x7ff55c107580 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1a2690 con 0x7ff55c10d630 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.266+0000 7ff564713700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1a2850 con 0x7ff55c107f40 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff5624af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 0x7ff55c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 0x7ff55c19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45696/0 (socket says 172.21.15.57:45696) 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff5624af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51062/0 (socket says 172.21.15.57:51062) 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 -- 172.21.15.57:0/48398564 learned_addr learned my addr 172.21.15.57:0/48398564 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff562cb0700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff55c10d630 0x7ff55c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 -- 172.21.15.57:0/48398564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff55c10d630 msgr2=0x7ff55c1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff55c10d630 0x7ff55c1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 -- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 msgr2=0x7ff55c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff561cae700 1 -- 172.21.15.57:0/48398564 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0095f0 con 0x7ff55c107f40 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.267+0000 7ff5624af700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.268+0000 7ff561cae700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 0x7ff55c19daa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff54c000c00 tx=0x7ff54c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:32.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.268+0000 7ff5537fe700 1 -- 172.21.15.57:0/48398564 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c01d070 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.268+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c1a2ad0 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.268+0000 7ff5537fe700 1 -- 172.21.15.57:0/48398564 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54c005050 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.268+0000 7ff5537fe700 1 -- 172.21.15.57:0/48398564 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c004ab0 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.268+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c1aa390 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.269+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c04ebb0 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.271+0000 7ff5537fe700 1 -- 172.21.15.57:0/48398564 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff54c0051c0 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.273+0000 7ff5537fe700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5480a1510 0x7ff5480a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.273+0000 7ff5537fe700 1 -- 172.21.15.57:0/48398564 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7ff54c0cdc50 con 0x7ff55c107f40 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.273+0000 7ff5624af700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5480a1510 0x7ff5480a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.273+0000 7ff5624af700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5480a1510 0x7ff5480a39d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff558009fd0 tx=0x7ff558009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:32.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.275+0000 7ff5537fe700 1 -- 172.21.15.57:0/48398564 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff54c0972d0 con 0x7ff55c107f40 2023-10-06T22:13:32.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.510+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7ff55c1174a0 con 0x7ff55c107f40 2023-10-06T22:13:32.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.511+0000 7ff5537fe700 1 -- 172.21.15.57:0/48398564 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff54c09a8f0 con 0x7ff55c107f40 2023-10-06T22:13:32.983 INFO:teuthology.orchestra.run.smithi057.stdout:335007449110 2023-10-06T22:13:32.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5480a1510 msgr2=0x7ff5480a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5480a1510 0x7ff5480a39d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff558009fd0 tx=0x7ff558009260 comp rx=0 tx=0).stop 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 msgr2=0x7ff55c19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 0x7ff55c19daa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff54c000c00 tx=0x7ff54c00bfd0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 shutdown_connections 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff5480a1510 0x7ff5480a39d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff55c10d630 0x7ff55c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff55c107580 0x7ff55c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 --2- 172.21.15.57:0/48398564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff55c107f40 0x7ff55c19daa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.514+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 >> 172.21.15.57:0/48398564 conn(0x7ff55c075540 msgr2=0x7ff55c076a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:32.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.515+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 shutdown_connections 2023-10-06T22:13:32.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.515+0000 7ff564713700 1 -- 172.21.15.57:0/48398564 wait complete. 2023-10-06T22:13:33.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:32 smithi156 ceph-mon[134358]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-10-06T22:13:33.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:32 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3653544235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T22:13:33.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:32 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/48398564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T22:13:33.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:32 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2039925233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T22:13:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:32 smithi190 ceph-mon[134891]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-10-06T22:13:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:32 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3653544235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T22:13:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:32 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/48398564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T22:13:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:32 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2039925233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T22:13:33.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 -- 172.21.15.57:0/2210999541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 msgr2=0x7f0038101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2210999541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038101400 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f002c0099c0 tx=0x7f002c009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 -- 172.21.15.57:0/2210999541 shutdown_connections 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2210999541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0038101940 0x7f003810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2210999541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0038106f90 0x7f0038107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2210999541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038101400 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 -- 172.21.15.57:0/2210999541 >> 172.21.15.57:0/2210999541 conn(0x7f00380780a0 msgr2=0x7f00380784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 -- 172.21.15.57:0/2210999541 shutdown_connections 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.784+0000 7f003ddf2700 1 -- 172.21.15.57:0/2210999541 wait complete. 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.785+0000 7f003ddf2700 1 Processor -- start 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.785+0000 7f003ddf2700 1 -- start start 2023-10-06T22:13:33.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.785+0000 7f003ddf2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f003ddf2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0038101940 0x7f00381974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f003ddf2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0038106f90 0x7f003819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f003ddf2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003819bf70 con 0x7f0038100f80 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f003ddf2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003819c0e0 con 0x7f0038106f90 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f003ddf2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003819c2a0 con 0x7f0038101940 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51094/0 (socket says 172.21.15.57:51094) 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 -- 172.21.15.57:0/2660124576 learned_addr learned my addr 172.21.15.57:0/2660124576 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f0036ffd700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0038101940 0x7f00381974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f0037fff700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0038106f90 0x7f003819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 -- 172.21.15.57:0/2660124576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0038101940 msgr2=0x7f00381974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0038101940 0x7f00381974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 -- 172.21.15.57:0/2660124576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0038106f90 msgr2=0x7f003819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0038106f90 0x7f003819b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 -- 172.21.15.57:0/2660124576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0095f0 con 0x7f0038100f80 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f0037fff700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0038106f90 0x7f003819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.786+0000 7f00377fe700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038196fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f002800e9a0 tx=0x7f002800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.787+0000 7f0034ff9700 1 -- 172.21.15.57:0/2660124576 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028009790 con 0x7f0038100f80 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.787+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003819c580 con 0x7f0038100f80 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.787+0000 7f0034ff9700 1 -- 172.21.15.57:0/2660124576 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f002800cbd0 con 0x7f0038100f80 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.787+0000 7f0034ff9700 1 -- 172.21.15.57:0/2660124576 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028018680 con 0x7f0038100f80 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.787+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00381a3de0 con 0x7f0038100f80 2023-10-06T22:13:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.788+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003804ebb0 con 0x7f0038100f80 2023-10-06T22:13:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.790+0000 7f0034ff9700 1 -- 172.21.15.57:0/2660124576 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f002800cd40 con 0x7f0038100f80 2023-10-06T22:13:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.792+0000 7f0034ff9700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f00200a1590 0x7f00200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.793+0000 7f0034ff9700 1 -- 172.21.15.57:0/2660124576 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f0028014070 con 0x7f0038100f80 2023-10-06T22:13:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.793+0000 7f0036ffd700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f00200a1590 0x7f00200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.793+0000 7f0036ffd700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f00200a1590 0x7f00200a3a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f002c009990 tx=0x7f002c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:33.180 INFO:teuthology.orchestra.run.smithi057.stdout:34359738441 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:32.795+0000 7f0034ff9700 1 -- 172.21.15.57:0/2660124576 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f00280968c0 con 0x7f0038100f80 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.013+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0038110fa0 con 0x7f0038100f80 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.013+0000 7f0034ff9700 1 -- 172.21.15.57:0/2660124576 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0028099ee0 con 0x7f0038100f80 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f00200a1590 msgr2=0x7f00200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f00200a1590 0x7f00200a3a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f002c009990 tx=0x7f002c005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 msgr2=0x7f0038196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038196fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f002800e9a0 tx=0x7f002800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 shutdown_connections 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f00200a1590 0x7f00200a3a50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0038106f90 0x7f003819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0038100f80 0x7f0038196fb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 --2- 172.21.15.57:0/2660124576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0038101940 0x7f00381974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 >> 172.21.15.57:0/2660124576 conn(0x7f00380780a0 msgr2=0x7f00380ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 shutdown_connections 2023-10-06T22:13:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.016+0000 7f003ddf2700 1 -- 172.21.15.57:0/2660124576 wait complete. 2023-10-06T22:13:33.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182456 got 107374182458 for osd.3 2023-10-06T22:13:33.206 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:33.350 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678039 got 304942678041 for osd.10 2023-10-06T22:13:33.350 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:33.689 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495658 got 197568495660 for osd.6 2023-10-06T22:13:33.690 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.889+0000 7f5d63530700 1 -- 172.21.15.57:0/3088747768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c073620 msgr2=0x7f5d5c073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.889+0000 7f5d63530700 1 --2- 172.21.15.57:0/3088747768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c073620 0x7f5d5c073a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009990 tx=0x7f5d4c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.889+0000 7f5d63530700 1 -- 172.21.15.57:0/3088747768 shutdown_connections 2023-10-06T22:13:33.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.889+0000 7f5d63530700 1 --2- 172.21.15.57:0/3088747768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 0x7f5d5c113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.889+0000 7f5d63530700 1 --2- 172.21.15.57:0/3088747768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d5c074ef0 0x7f5d5c0730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.889+0000 7f5d63530700 1 --2- 172.21.15.57:0/3088747768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c073620 0x7f5d5c073a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.889+0000 7f5d63530700 1 -- 172.21.15.57:0/3088747768 >> 172.21.15.57:0/3088747768 conn(0x7f5d5c0780a0 msgr2=0x7f5d5c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.890+0000 7f5d63530700 1 -- 172.21.15.57:0/3088747768 shutdown_connections 2023-10-06T22:13:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.890+0000 7f5d63530700 1 -- 172.21.15.57:0/3088747768 wait complete. 2023-10-06T22:13:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.890+0000 7f5d63530700 1 Processor -- start 2023-10-06T22:13:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d63530700 1 -- start start 2023-10-06T22:13:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d63530700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d5c073620 0x7f5d5c1a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d63530700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c074ef0 0x7f5d5c1a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d63530700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 0x7f5d5c19d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d63530700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c19db40 con 0x7f5d5c111bc0 2023-10-06T22:13:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d63530700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c19dc80 con 0x7f5d5c074ef0 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d63530700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c19de40 con 0x7f5d5c073620 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.891+0000 7f5d61acd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 0x7f5d5c19d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 0x7f5d5c19d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51114/0 (socket says 172.21.15.57:51114) 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 -- 172.21.15.57:0/4096580818 learned_addr learned my addr 172.21.15.57:0/4096580818 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d612cc700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d5c073620 0x7f5d5c1a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d60acb700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c074ef0 0x7f5d5c1a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 -- 172.21.15.57:0/4096580818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d5c073620 msgr2=0x7f5d5c1a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d5c073620 0x7f5d5c1a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 -- 172.21.15.57:0/4096580818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c074ef0 msgr2=0x7f5d5c1a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c074ef0 0x7f5d5c1a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 -- 172.21.15.57:0/4096580818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0095f0 con 0x7f5d5c111bc0 2023-10-06T22:13:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d61acd700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 0x7f5d5c19d4e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ef90 tx=0x7f5d5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.892+0000 7f5d527fc700 1 -- 172.21.15.57:0/4096580818 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54018070 con 0x7f5d5c111bc0 2023-10-06T22:13:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.893+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c19e0c0 con 0x7f5d5c111bc0 2023-10-06T22:13:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.893+0000 7f5d527fc700 1 -- 172.21.15.57:0/4096580818 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5400f040 con 0x7f5d5c111bc0 2023-10-06T22:13:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.893+0000 7f5d527fc700 1 -- 172.21.15.57:0/4096580818 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400ccc0 con 0x7f5d5c111bc0 2023-10-06T22:13:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.893+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1aa4a0 con 0x7f5d5c111bc0 2023-10-06T22:13:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.894+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c04ebb0 con 0x7f5d5c111bc0 2023-10-06T22:13:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.896+0000 7f5d527fc700 1 -- 172.21.15.57:0/4096580818 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5d54003a00 con 0x7f5d5c111bc0 2023-10-06T22:13:33.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.897+0000 7f5d527fc700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d480a1590 0x7f5d480a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.897+0000 7f5d527fc700 1 -- 172.21.15.57:0/4096580818 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f5d540cc570 con 0x7f5d5c111bc0 2023-10-06T22:13:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.897+0000 7f5d612cc700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d480a1590 0x7f5d480a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.898+0000 7f5d612cc700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d480a1590 0x7f5d480a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d58007be0 tx=0x7f5d580073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.901+0000 7f5d527fc700 1 -- 172.21.15.57:0/4096580818 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5d54095ad0 con 0x7f5d5c111bc0 2023-10-06T22:13:33.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:33 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2660124576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T22:13:33.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 -- 172.21.15.57:0/760472271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c106f90 msgr2=0x7f8e9c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 --2- 172.21.15.57:0/760472271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c106f90 0x7f8e9c107370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e980099c0 tx=0x7f8e98009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 -- 172.21.15.57:0/760472271 shutdown_connections 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 --2- 172.21.15.57:0/760472271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e9c101940 0x7f8e9c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 --2- 172.21.15.57:0/760472271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c106f90 0x7f8e9c107370 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 --2- 172.21.15.57:0/760472271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e9c100f80 0x7f8e9c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 -- 172.21.15.57:0/760472271 >> 172.21.15.57:0/760472271 conn(0x7f8e9c0780a0 msgr2=0x7f8e9c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 -- 172.21.15.57:0/760472271 shutdown_connections 2023-10-06T22:13:33.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.957+0000 7f8ea407d700 1 -- 172.21.15.57:0/760472271 wait complete. 2023-10-06T22:13:33.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.958+0000 7f8ea407d700 1 Processor -- start 2023-10-06T22:13:33.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.959+0000 7f8ea407d700 1 -- start start 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.959+0000 7f8ea407d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c100f80 0x7f8e9c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.959+0000 7f8ea407d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e9c101940 0x7f8e9c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.959+0000 7f8ea407d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e9c106f90 0x7f8e9c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.959+0000 7f8ea407d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c19e0e0 con 0x7f8e9c100f80 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.959+0000 7f8ea407d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c19e250 con 0x7f8e9c106f90 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.959+0000 7f8ea407d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c19e410 con 0x7f8e9c101940 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c100f80 0x7f8e9c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1618700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e9c101940 0x7f8e9c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c100f80 0x7f8e9c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51128/0 (socket says 172.21.15.57:51128) 2023-10-06T22:13:33.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 -- 172.21.15.57:0/3374377603 learned_addr learned my addr 172.21.15.57:0/3374377603 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:33.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1618700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e9c101940 0x7f8e9c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45788/0 (socket says 172.21.15.57:45788) 2023-10-06T22:13:33.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea261a700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e9c106f90 0x7f8e9c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 -- 172.21.15.57:0/3374377603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e9c101940 msgr2=0x7f8e9c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e9c101940 0x7f8e9c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 -- 172.21.15.57:0/3374377603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e9c106f90 msgr2=0x7f8e9c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e9c106f90 0x7f8e9c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 -- 172.21.15.57:0/3374377603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c009520 con 0x7f8e9c100f80 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea261a700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e9c106f90 0x7f8e9c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.960+0000 7f8ea1e19700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c100f80 0x7f8e9c199120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e9800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.961+0000 7f8e92ffd700 1 -- 172.21.15.57:0/3374377603 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801d070 con 0x7f8e9c100f80 2023-10-06T22:13:33.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.961+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e980095f0 con 0x7f8e9c100f80 2023-10-06T22:13:33.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.961+0000 7f8e92ffd700 1 -- 172.21.15.57:0/3374377603 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e98005050 con 0x7f8e9c100f80 2023-10-06T22:13:33.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.961+0000 7f8e92ffd700 1 -- 172.21.15.57:0/3374377603 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98004ab0 con 0x7f8e9c100f80 2023-10-06T22:13:33.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.961+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c1a5d80 con 0x7f8e9c100f80 2023-10-06T22:13:33.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.962+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e80005320 con 0x7f8e9c100f80 2023-10-06T22:13:33.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.965+0000 7f8e92ffd700 1 -- 172.21.15.57:0/3374377603 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8e98004c10 con 0x7f8e9c100f80 2023-10-06T22:13:33.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.967+0000 7f8e92ffd700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8e880a1590 0x7f8e880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:33.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.967+0000 7f8e92ffd700 1 -- 172.21.15.57:0/3374377603 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f8e980cda20 con 0x7f8e9c100f80 2023-10-06T22:13:33.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.968+0000 7f8ea1618700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8e880a1590 0x7f8e880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:33.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.968+0000 7f8ea1618700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8e880a1590 0x7f8e880a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c19a740 tx=0x7f8e8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:33.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:33.972+0000 7f8e92ffd700 1 -- 172.21.15.57:0/3374377603 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8e98096f80 con 0x7f8e9c100f80 2023-10-06T22:13:34.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:33 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2660124576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T22:13:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:33 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2660124576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T22:13:34.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.176+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f5d5c117740 con 0x7f5d5c111bc0 2023-10-06T22:13:34.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.176+0000 7f5d527fc700 1 -- 172.21.15.57:0/4096580818 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d540990f0 con 0x7f5d5c111bc0 2023-10-06T22:13:34.179 INFO:teuthology.orchestra.run.smithi057.stdout:283467841567 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d480a1590 msgr2=0x7f5d480a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d480a1590 0x7f5d480a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d58007be0 tx=0x7f5d580073d0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 msgr2=0x7f5d5c19d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 0x7f5d5c19d4e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ef90 tx=0x7f5d5400c3a0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 shutdown_connections 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f5d480a1590 0x7f5d480a3a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c111bc0 0x7f5d5c19d4e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d5c073620 0x7f5d5c1a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 --2- 172.21.15.57:0/4096580818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d5c074ef0 0x7f5d5c1a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.179+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 >> 172.21.15.57:0/4096580818 conn(0x7f5d5c0780a0 msgr2=0x7f5d5c110640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.180+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 shutdown_connections 2023-10-06T22:13:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.180+0000 7f5d63530700 1 -- 172.21.15.57:0/4096580818 wait complete. 2023-10-06T22:13:34.218 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920818 got 141733920821 for osd.4 2023-10-06T22:13:34.218 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.260+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8e80005190 con 0x7f8e9c100f80 2023-10-06T22:13:34.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.260+0000 7f8e92ffd700 1 -- 172.21.15.57:0/3374377603 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e9809a5a0 con 0x7f8e9c100f80 2023-10-06T22:13:34.263 INFO:teuthology.orchestra.run.smithi057.stdout:55834574917 2023-10-06T22:13:34.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8e880a1590 msgr2=0x7f8e880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:34.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8e880a1590 0x7f8e880a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c19a740 tx=0x7f8e8c009260 comp rx=0 tx=0).stop 2023-10-06T22:13:34.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c100f80 msgr2=0x7f8e9c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:34.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c100f80 0x7f8e9c199120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e9800bfd0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 shutdown_connections 2023-10-06T22:13:34.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8e880a1590 0x7f8e880a3a50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e9c106f90 0x7f8e9c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e9c100f80 0x7f8e9c199120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 --2- 172.21.15.57:0/3374377603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e9c101940 0x7f8e9c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:34.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 >> 172.21.15.57:0/3374377603 conn(0x7f8e9c0780a0 msgr2=0x7f8e9c10c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:34.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 shutdown_connections 2023-10-06T22:13:34.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:34.263+0000 7f8ea407d700 1 -- 172.21.15.57:0/3374377603 wait complete. 2023-10-06T22:13:34.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738439 got 34359738441 for osd.0 2023-10-06T22:13:34.906 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:35.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:34 smithi156 ceph-mon[134358]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-10-06T22:13:35.018 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:34 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/4096580818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T22:13:35.018 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:34 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3374377603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T22:13:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:34 smithi190 ceph-mon[134891]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-10-06T22:13:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:34 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/4096580818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T22:13:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:34 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3374377603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T22:13:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:34 smithi057 ceph-mon[98492]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-10-06T22:13:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:34 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/4096580818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T22:13:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:34 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3374377603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T22:13:35.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449107 got 335007449110 for osd.11 2023-10-06T22:13:35.316 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:35.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574913 got 55834574917 for osd.1 2023-10-06T22:13:35.394 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:35.407 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841564 got 283467841567 for osd.9 2023-10-06T22:13:35.407 DEBUG:teuthology.parallel:result is None 2023-10-06T22:13:35.407 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-06T22:13:35.407 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-06T22:13:36.184 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:37.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:36 smithi156 ceph-mon[134358]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:37.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:36 smithi190 ceph-mon[134891]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:37.145 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:36 smithi057 ceph-mon[98492]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:37.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 -- 172.21.15.57:0/3269305527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 msgr2=0x7feddc0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:37.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 --2- 172.21.15.57:0/3269305527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc0734c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009990 tx=0x7fedd8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:37.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 -- 172.21.15.57:0/3269305527 shutdown_connections 2023-10-06T22:13:37.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 --2- 172.21.15.57:0/3269305527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc10d930 0x7feddc10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:37.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 --2- 172.21.15.57:0/3269305527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc0734c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:37.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 --2- 172.21.15.57:0/3269305527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feddc073a00 0x7feddc10d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:37.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 -- 172.21.15.57:0/3269305527 >> 172.21.15.57:0/3269305527 conn(0x7feddc0780a0 msgr2=0x7feddc0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:37.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 -- 172.21.15.57:0/3269305527 shutdown_connections 2023-10-06T22:13:37.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.783+0000 7fede46f3700 1 -- 172.21.15.57:0/3269305527 wait complete. 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.784+0000 7fede46f3700 1 Processor -- start 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.784+0000 7fede46f3700 1 -- start start 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede46f3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede46f3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feddc073a00 0x7feddc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede46f3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc10d930 0x7feddc19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede46f3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc19e280 con 0x7feddc0730e0 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede46f3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc19e3f0 con 0x7feddc073a00 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede46f3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc19e5b0 con 0x7feddc10d930 2023-10-06T22:13:37.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede248f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede248f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51140/0 (socket says 172.21.15.57:51140) 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede248f700 1 -- 172.21.15.57:0/1607945714 learned_addr learned my addr 172.21.15.57:0/1607945714 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede1c8e700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feddc073a00 0x7feddc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.785+0000 7fede2c90700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc10d930 0x7feddc19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede248f700 1 -- 172.21.15.57:0/1607945714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc10d930 msgr2=0x7feddc19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede248f700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc10d930 0x7feddc19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede248f700 1 -- 172.21.15.57:0/1607945714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feddc073a00 msgr2=0x7feddc199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede248f700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feddc073a00 0x7feddc199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede248f700 1 -- 172.21.15.57:0/1607945714 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd80095f0 con 0x7feddc0730e0 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede248f700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc1992c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fedd8005910 tx=0x7fedd8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:37.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fedd37fe700 1 -- 172.21.15.57:0/1607945714 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd801d070 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fedd37fe700 1 -- 172.21.15.57:0/1607945714 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedd8022470 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc19e830 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fedd37fe700 1 -- 172.21.15.57:0/1607945714 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8017650 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.786+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc1a60f0 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.787+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc04ebb0 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.789+0000 7fedd37fe700 1 -- 172.21.15.57:0/1607945714 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fedd80225e0 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.790+0000 7fedd37fe700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedc80a1590 0x7fedc80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.791+0000 7fedd37fe700 1 -- 172.21.15.57:0/1607945714 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fedd80ce110 con 0x7feddc0730e0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.791+0000 7fede1c8e700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedc80a1590 0x7fedc80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.791+0000 7fede1c8e700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedc80a1590 0x7fedc80a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feddc19a8e0 tx=0x7fedcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:37.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:37.795+0000 7fedd37fe700 1 -- 172.21.15.57:0/1607945714 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fedd805d100 con 0x7feddc0730e0 2023-10-06T22:13:38.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.001+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feddc113180 con 0x7fedc80a1590 2023-10-06T22:13:38.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.012+0000 7fedd37fe700 1 -- 172.21.15.57:0/1607945714 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341320 (secure 0 0 0) 0x7feddc113180 con 0x7fedc80a1590 2023-10-06T22:13:38.021 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedc80a1590 msgr2=0x7fedc80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedc80a1590 0x7fedc80a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feddc19a8e0 tx=0x7fedcc009260 comp rx=0 tx=0).stop 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 msgr2=0x7feddc1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc1992c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fedd8005910 tx=0x7fedd8004710 comp rx=0 tx=0).stop 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 shutdown_connections 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fedc80a1590 0x7fedc80a3a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc10d930 0x7feddc19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feddc0730e0 0x7feddc1992c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 --2- 172.21.15.57:0/1607945714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feddc073a00 0x7feddc199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 >> 172.21.15.57:0/1607945714 conn(0x7feddc0780a0 msgr2=0x7feddc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 shutdown_connections 2023-10-06T22:13:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:38.018+0000 7fede46f3700 1 -- 172.21.15.57:0/1607945714 wait complete. 2023-10-06T22:13:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-06T22:13:38.464 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-10-06T22:13:37.628099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655714,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":601,"num_read_kb":418,"num_write":413,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337232,"kb_used_data":12976,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604592,"statfs":{"total":1151940427776,"available":1151595102208,"internally_reserved":0,"allocated":13287424,"data_stored":7281158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002993"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720589+0000","last_change":"2023-10-06T22:12:43.704484+0000","last_active":"2023-10-06T22:13:05.720589+0000","last_peered":"2023-10-06T22:13:05.720589+0000","last_clean":"2023-10-06T22:13:05.720589+0000","last_became_active":"2023-10-06T22:12:43.703740+0000","last_became_peered":"2023-10-06T22:12:43.703740+0000","last_unstale":"2023-10-06T22:13:05.720589+0000","last_undegraded":"2023-10-06T22:13:05.720589+0000","last_fullsized":"2023-10-06T22:13:05.720589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:25.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745169+0000","last_change":"2023-10-06T22:12:43.696968+0000","last_active":"2023-10-06T22:13:05.745169+0000","last_peered":"2023-10-06T22:13:05.745169+0000","last_clean":"2023-10-06T22:13:05.745169+0000","last_became_active":"2023-10-06T22:12:43.696532+0000","last_became_peered":"2023-10-06T22:12:43.696532+0000","last_unstale":"2023-10-06T22:13:05.745169+0000","last_undegraded":"2023-10-06T22:13:05.745169+0000","last_fullsized":"2023-10-06T22:13:05.745169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:59:50.754606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744952+0000","last_change":"2023-10-06T22:12:43.700845+0000","last_active":"2023-10-06T22:13:05.744952+0000","last_peered":"2023-10-06T22:13:05.744952+0000","last_clean":"2023-10-06T22:13:05.744952+0000","last_became_active":"2023-10-06T22:12:43.700531+0000","last_became_peered":"2023-10-06T22:12:43.700531+0000","last_unstale":"2023-10-06T22:13:05.744952+0000","last_undegraded":"2023-10-06T22:13:05.744952+0000","last_fullsized":"2023-10-06T22:13:05.744952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:18.518772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722902+0000","last_change":"2023-10-06T22:12:43.700806+0000","last_active":"2023-10-06T22:13:05.722902+0000","last_peered":"2023-10-06T22:13:05.722902+0000","last_clean":"2023-10-06T22:13:05.722902+0000","last_became_active":"2023-10-06T22:12:43.699901+0000","last_became_peered":"2023-10-06T22:12:43.699901+0000","last_unstale":"2023-10-06T22:13:05.722902+0000","last_undegraded":"2023-10-06T22:13:05.722902+0000","last_fullsized":"2023-10-06T22:13:05.722902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:41:35.906415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724450+0000","last_change":"2023-10-06T22:12:43.694776+0000","last_active":"2023-10-06T22:13:05.724450+0000","last_peered":"2023-10-06T22:13:05.724450+0000","last_clean":"2023-10-06T22:13:05.724450+0000","last_became_active":"2023-10-06T22:12:43.694397+0000","last_became_peered":"2023-10-06T22:12:43.694397+0000","last_unstale":"2023-10-06T22:13:05.724450+0000","last_undegraded":"2023-10-06T22:13:05.724450+0000","last_fullsized":"2023-10-06T22:13:05.724450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:47:43.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.719749+0000","last_change":"2023-10-06T22:12:43.703515+0000","last_active":"2023-10-06T22:13:05.719749+0000","last_peered":"2023-10-06T22:13:05.719749+0000","last_clean":"2023-10-06T22:13:05.719749+0000","last_became_active":"2023-10-06T22:12:43.703094+0000","last_became_peered":"2023-10-06T22:12:43.703094+0000","last_unstale":"2023-10-06T22:13:05.719749+0000","last_undegraded":"2023-10-06T22:13:05.719749+0000","last_fullsized":"2023-10-06T22:13:05.719749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:54:32.130131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727441+0000","last_change":"2023-10-06T22:12:43.697894+0000","last_active":"2023-10-06T22:13:05.727441+0000","last_peered":"2023-10-06T22:13:05.727441+0000","last_clean":"2023-10-06T22:13:05.727441+0000","last_became_active":"2023-10-06T22:12:43.697591+0000","last_became_peered":"2023-10-06T22:12:43.697591+0000","last_unstale":"2023-10-06T22:13:05.727441+0000","last_undegraded":"2023-10-06T22:13:05.727441+0000","last_fullsized":"2023-10-06T22:13:05.727441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:30.313447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.927879+0000","last_change":"2023-10-06T22:13:06.927879+0000","last_active":"2023-10-06T22:13:06.927879+0000","last_peered":"2023-10-06T22:13:06.927879+0000","last_clean":"2023-10-06T22:13:06.927879+0000","last_became_active":"2023-10-06T22:13:06.927559+0000","last_became_peered":"2023-10-06T22:13:06.927559+0000","last_unstale":"2023-10-06T22:13:06.927879+0000","last_undegraded":"2023-10-06T22:13:06.927879+0000","last_fullsized":"2023-10-06T22:13:06.927879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:15.874159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.739167+0000","last_change":"2023-10-06T22:12:43.696995+0000","last_active":"2023-10-06T22:13:05.739167+0000","last_peered":"2023-10-06T22:13:05.739167+0000","last_clean":"2023-10-06T22:13:05.739167+0000","last_became_active":"2023-10-06T22:12:43.696731+0000","last_became_peered":"2023-10-06T22:12:43.696731+0000","last_unstale":"2023-10-06T22:13:05.739167+0000","last_undegraded":"2023-10-06T22:13:05.739167+0000","last_fullsized":"2023-10-06T22:13:05.739167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:43:31.261922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720404+0000","last_change":"2023-10-06T22:12:43.704391+0000","last_active":"2023-10-06T22:13:05.720404+0000","last_peered":"2023-10-06T22:13:05.720404+0000","last_clean":"2023-10-06T22:13:05.720404+0000","last_became_active":"2023-10-06T22:12:43.702954+0000","last_became_peered":"2023-10-06T22:12:43.702954+0000","last_unstale":"2023-10-06T22:13:05.720404+0000","last_undegraded":"2023-10-06T22:13:05.720404+0000","last_fullsized":"2023-10-06T22:13:05.720404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:38.973781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.728368+0000","last_change":"2023-10-06T22:12:43.696265+0000","last_active":"2023-10-06T22:13:05.728368+0000","last_peered":"2023-10-06T22:13:05.728368+0000","last_clean":"2023-10-06T22:13:05.728368+0000","last_became_active":"2023-10-06T22:12:43.695939+0000","last_became_peered":"2023-10-06T22:12:43.695939+0000","last_unstale":"2023-10-06T22:13:05.728368+0000","last_undegraded":"2023-10-06T22:13:05.728368+0000","last_fullsized":"2023-10-06T22:13:05.728368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:32.882083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.812700+0000","last_change":"2023-10-06T22:13:06.812700+0000","last_active":"2023-10-06T22:13:06.812700+0000","last_peered":"2023-10-06T22:13:06.812700+0000","last_clean":"2023-10-06T22:13:06.812700+0000","last_became_active":"2023-10-06T22:13:06.812313+0000","last_became_peered":"2023-10-06T22:13:06.812313+0000","last_unstale":"2023-10-06T22:13:06.812700+0000","last_undegraded":"2023-10-06T22:13:06.812700+0000","last_fullsized":"2023-10-06T22:13:06.812700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:25.010233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422371+0000","last_change":"2023-10-06T22:12:43.694717+0000","last_active":"2023-10-06T22:13:06.422371+0000","last_peered":"2023-10-06T22:13:06.422371+0000","last_clean":"2023-10-06T22:13:06.422371+0000","last_became_active":"2023-10-06T22:12:43.694321+0000","last_became_peered":"2023-10-06T22:12:43.694321+0000","last_unstale":"2023-10-06T22:13:06.422371+0000","last_undegraded":"2023-10-06T22:13:06.422371+0000","last_fullsized":"2023-10-06T22:13:06.422371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:06.334726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744274+0000","last_change":"2023-10-06T22:12:43.703770+0000","last_active":"2023-10-06T22:13:05.744274+0000","last_peered":"2023-10-06T22:13:05.744274+0000","last_clean":"2023-10-06T22:13:05.744274+0000","last_became_active":"2023-10-06T22:12:43.703434+0000","last_became_peered":"2023-10-06T22:12:43.703434+0000","last_unstale":"2023-10-06T22:13:05.744274+0000","last_undegraded":"2023-10-06T22:13:05.744274+0000","last_fullsized":"2023-10-06T22:13:05.744274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:25.894280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744061+0000","last_change":"2023-10-06T22:12:43.700994+0000","last_active":"2023-10-06T22:13:05.744061+0000","last_peered":"2023-10-06T22:13:05.744061+0000","last_clean":"2023-10-06T22:13:05.744061+0000","last_became_active":"2023-10-06T22:12:43.700652+0000","last_became_peered":"2023-10-06T22:12:43.700652+0000","last_unstale":"2023-10-06T22:13:05.744061+0000","last_undegraded":"2023-10-06T22:13:05.744061+0000","last_fullsized":"2023-10-06T22:13:05.744061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:54.855597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738703+0000","last_change":"2023-10-06T22:12:43.702099+0000","last_active":"2023-10-06T22:13:05.738703+0000","last_peered":"2023-10-06T22:13:05.738703+0000","last_clean":"2023-10-06T22:13:05.738703+0000","last_became_active":"2023-10-06T22:12:43.701802+0000","last_became_peered":"2023-10-06T22:12:43.701802+0000","last_unstale":"2023-10-06T22:13:05.738703+0000","last_undegraded":"2023-10-06T22:13:05.738703+0000","last_fullsized":"2023-10-06T22:13:05.738703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:00.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720703+0000","last_change":"2023-10-06T22:12:43.704679+0000","last_active":"2023-10-06T22:13:05.720703+0000","last_peered":"2023-10-06T22:13:05.720703+0000","last_clean":"2023-10-06T22:13:05.720703+0000","last_became_active":"2023-10-06T22:12:43.704186+0000","last_became_peered":"2023-10-06T22:12:43.704186+0000","last_unstale":"2023-10-06T22:13:05.720703+0000","last_undegraded":"2023-10-06T22:13:05.720703+0000","last_fullsized":"2023-10-06T22:13:05.720703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:28.665217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.735809+0000","last_change":"2023-10-06T22:12:43.695699+0000","last_active":"2023-10-06T22:13:05.735809+0000","last_peered":"2023-10-06T22:13:05.735809+0000","last_clean":"2023-10-06T22:13:05.735809+0000","last_became_active":"2023-10-06T22:12:43.695359+0000","last_became_peered":"2023-10-06T22:12:43.695359+0000","last_unstale":"2023-10-06T22:13:05.735809+0000","last_undegraded":"2023-10-06T22:13:05.735809+0000","last_fullsized":"2023-10-06T22:13:05.735809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:46.296768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727882+0000","last_change":"2023-10-06T22:12:43.707170+0000","last_active":"2023-10-06T22:13:05.727882+0000","last_peered":"2023-10-06T22:13:05.727882+0000","last_clean":"2023-10-06T22:13:05.727882+0000","last_became_active":"2023-10-06T22:12:43.706875+0000","last_became_peered":"2023-10-06T22:12:43.706875+0000","last_unstale":"2023-10-06T22:13:05.727882+0000","last_undegraded":"2023-10-06T22:13:05.727882+0000","last_fullsized":"2023-10-06T22:13:05.727882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:46.548458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":97,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:34.765129+0000","last_change":"2023-10-06T22:13:06.736382+0000","last_active":"2023-10-06T22:13:34.765129+0000","last_peered":"2023-10-06T22:13:34.765129+0000","last_clean":"2023-10-06T22:13:34.765129+0000","last_became_active":"2023-10-06T22:13:06.725703+0000","last_became_peered":"2023-10-06T22:13:06.725703+0000","last_unstale":"2023-10-06T22:13:34.765129+0000","last_undegraded":"2023-10-06T22:13:34.765129+0000","last_fullsized":"2023-10-06T22:13:34.765129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:10.056384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752688+0000","last_change":"2023-10-06T22:12:43.702574+0000","last_active":"2023-10-06T22:13:05.752688+0000","last_peered":"2023-10-06T22:13:05.752688+0000","last_clean":"2023-10-06T22:13:05.752688+0000","last_became_active":"2023-10-06T22:12:43.702238+0000","last_became_peered":"2023-10-06T22:12:43.702238+0000","last_unstale":"2023-10-06T22:13:05.752688+0000","last_undegraded":"2023-10-06T22:13:05.752688+0000","last_fullsized":"2023-10-06T22:13:05.752688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:34:09.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752842+0000","last_change":"2023-10-06T22:12:42.697256+0000","last_active":"2023-10-06T22:13:05.752842+0000","last_peered":"2023-10-06T22:13:05.752842+0000","last_clean":"2023-10-06T22:13:05.752842+0000","last_became_active":"2023-10-06T22:12:42.696724+0000","last_became_peered":"2023-10-06T22:12:42.696724+0000","last_unstale":"2023-10-06T22:13:05.752842+0000","last_undegraded":"2023-10-06T22:13:05.752842+0000","last_fullsized":"2023-10-06T22:13:05.752842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:50:01.378292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745682+0000","last_change":"2023-10-06T22:12:43.702050+0000","last_active":"2023-10-06T22:13:05.745682+0000","last_peered":"2023-10-06T22:13:05.745682+0000","last_clean":"2023-10-06T22:13:05.745682+0000","last_became_active":"2023-10-06T22:12:43.701432+0000","last_became_peered":"2023-10-06T22:12:43.701432+0000","last_unstale":"2023-10-06T22:13:05.745682+0000","last_undegraded":"2023-10-06T22:13:05.745682+0000","last_fullsized":"2023-10-06T22:13:05.745682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:30:37.909775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.948160+0000","last_change":"2023-10-06T22:13:06.948160+0000","last_active":"2023-10-06T22:13:06.948160+0000","last_peered":"2023-10-06T22:13:06.948160+0000","last_clean":"2023-10-06T22:13:06.948160+0000","last_became_active":"2023-10-06T22:13:06.927995+0000","last_became_peered":"2023-10-06T22:13:06.927995+0000","last_unstale":"2023-10-06T22:13:06.948160+0000","last_undegraded":"2023-10-06T22:13:06.948160+0000","last_fullsized":"2023-10-06T22:13:06.948160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:33:27.974776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738816+0000","last_change":"2023-10-06T22:12:43.696511+0000","last_active":"2023-10-06T22:13:05.738816+0000","last_peered":"2023-10-06T22:13:05.738816+0000","last_clean":"2023-10-06T22:13:05.738816+0000","last_became_active":"2023-10-06T22:12:43.696087+0000","last_became_peered":"2023-10-06T22:12:43.696087+0000","last_unstale":"2023-10-06T22:13:05.738816+0000","last_undegraded":"2023-10-06T22:13:05.738816+0000","last_fullsized":"2023-10-06T22:13:05.738816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:12:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724948+0000","last_change":"2023-10-06T22:12:42.694963+0000","last_active":"2023-10-06T22:13:05.724948+0000","last_peered":"2023-10-06T22:13:05.724948+0000","last_clean":"2023-10-06T22:13:05.724948+0000","last_became_active":"2023-10-06T22:12:42.694517+0000","last_became_peered":"2023-10-06T22:12:42.694517+0000","last_unstale":"2023-10-06T22:13:05.724948+0000","last_undegraded":"2023-10-06T22:13:05.724948+0000","last_fullsized":"2023-10-06T22:13:05.724948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:07:33.404328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723715+0000","last_change":"2023-10-06T22:12:43.698077+0000","last_active":"2023-10-06T22:13:05.723715+0000","last_peered":"2023-10-06T22:13:05.723715+0000","last_clean":"2023-10-06T22:13:05.723715+0000","last_became_active":"2023-10-06T22:12:43.697676+0000","last_became_peered":"2023-10-06T22:12:43.697676+0000","last_unstale":"2023-10-06T22:13:05.723715+0000","last_undegraded":"2023-10-06T22:13:05.723715+0000","last_fullsized":"2023-10-06T22:13:05.723715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:59:56.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:33.731417+0000","last_change":"2023-10-06T22:12:42.698067+0000","last_active":"2023-10-06T22:13:33.731417+0000","last_peered":"2023-10-06T22:13:33.731417+0000","last_clean":"2023-10-06T22:13:33.731417+0000","last_became_active":"2023-10-06T22:12:42.697671+0000","last_became_peered":"2023-10-06T22:12:42.697671+0000","last_unstale":"2023-10-06T22:13:33.731417+0000","last_undegraded":"2023-10-06T22:13:33.731417+0000","last_fullsized":"2023-10-06T22:13:33.731417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:33.607513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724160+0000","last_change":"2023-10-06T22:12:43.696213+0000","last_active":"2023-10-06T22:13:05.724160+0000","last_peered":"2023-10-06T22:13:05.724160+0000","last_clean":"2023-10-06T22:13:05.724160+0000","last_became_active":"2023-10-06T22:12:43.695925+0000","last_became_peered":"2023-10-06T22:12:43.695925+0000","last_unstale":"2023-10-06T22:13:05.724160+0000","last_undegraded":"2023-10-06T22:13:05.724160+0000","last_fullsized":"2023-10-06T22:13:05.724160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:24.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.719695+0000","last_change":"2023-10-06T22:12:42.687188+0000","last_active":"2023-10-06T22:13:05.719695+0000","last_peered":"2023-10-06T22:13:05.719695+0000","last_clean":"2023-10-06T22:13:05.719695+0000","last_became_active":"2023-10-06T22:12:42.686747+0000","last_became_peered":"2023-10-06T22:12:42.686747+0000","last_unstale":"2023-10-06T22:13:05.719695+0000","last_undegraded":"2023-10-06T22:13:05.719695+0000","last_fullsized":"2023-10-06T22:13:05.719695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:21.326911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723338+0000","last_change":"2023-10-06T22:12:43.697988+0000","last_active":"2023-10-06T22:13:05.723338+0000","last_peered":"2023-10-06T22:13:05.723338+0000","last_clean":"2023-10-06T22:13:05.723338+0000","last_became_active":"2023-10-06T22:12:43.697641+0000","last_became_peered":"2023-10-06T22:12:43.697641+0000","last_unstale":"2023-10-06T22:13:05.723338+0000","last_undegraded":"2023-10-06T22:13:05.723338+0000","last_fullsized":"2023-10-06T22:13:05.723338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:33.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.423261+0000","last_change":"2023-10-06T22:12:42.688882+0000","last_active":"2023-10-06T22:13:06.423261+0000","last_peered":"2023-10-06T22:13:06.423261+0000","last_clean":"2023-10-06T22:13:06.423261+0000","last_became_active":"2023-10-06T22:12:42.687653+0000","last_became_peered":"2023-10-06T22:12:42.687653+0000","last_unstale":"2023-10-06T22:13:06.423261+0000","last_undegraded":"2023-10-06T22:13:06.423261+0000","last_fullsized":"2023-10-06T22:13:06.423261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:18.146081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.753938+0000","last_change":"2023-10-06T22:12:43.696559+0000","last_active":"2023-10-06T22:13:05.753938+0000","last_peered":"2023-10-06T22:13:05.753938+0000","last_clean":"2023-10-06T22:13:05.753938+0000","last_became_active":"2023-10-06T22:12:43.696252+0000","last_became_peered":"2023-10-06T22:12:43.696252+0000","last_unstale":"2023-10-06T22:13:05.753938+0000","last_undegraded":"2023-10-06T22:13:05.753938+0000","last_fullsized":"2023-10-06T22:13:05.753938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:31:36.097000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722785+0000","last_change":"2023-10-06T22:12:42.694299+0000","last_active":"2023-10-06T22:13:05.722785+0000","last_peered":"2023-10-06T22:13:05.722785+0000","last_clean":"2023-10-06T22:13:05.722785+0000","last_became_active":"2023-10-06T22:12:42.693820+0000","last_became_peered":"2023-10-06T22:12:42.693820+0000","last_unstale":"2023-10-06T22:13:05.722785+0000","last_undegraded":"2023-10-06T22:13:05.722785+0000","last_fullsized":"2023-10-06T22:13:05.722785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:13.837379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.734798+0000","last_change":"2023-10-06T22:12:43.695234+0000","last_active":"2023-10-06T22:13:05.734798+0000","last_peered":"2023-10-06T22:13:05.734798+0000","last_clean":"2023-10-06T22:13:05.734798+0000","last_became_active":"2023-10-06T22:12:43.694889+0000","last_became_peered":"2023-10-06T22:12:43.694889+0000","last_unstale":"2023-10-06T22:13:05.734798+0000","last_undegraded":"2023-10-06T22:13:05.734798+0000","last_fullsized":"2023-10-06T22:13:05.734798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:51.039264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":105,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:32.729337+0000","last_change":"2023-10-06T22:12:42.695438+0000","last_active":"2023-10-06T22:13:32.729337+0000","last_peered":"2023-10-06T22:13:32.729337+0000","last_clean":"2023-10-06T22:13:32.729337+0000","last_became_active":"2023-10-06T22:12:42.695090+0000","last_became_peered":"2023-10-06T22:12:42.695090+0000","last_unstale":"2023-10-06T22:13:32.729337+0000","last_undegraded":"2023-10-06T22:13:32.729337+0000","last_fullsized":"2023-10-06T22:13:32.729337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:25:34.385961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.739124+0000","last_change":"2023-10-06T22:12:43.697459+0000","last_active":"2023-10-06T22:13:05.739124+0000","last_peered":"2023-10-06T22:13:05.739124+0000","last_clean":"2023-10-06T22:13:05.739124+0000","last_became_active":"2023-10-06T22:12:43.696405+0000","last_became_peered":"2023-10-06T22:12:43.696405+0000","last_unstale":"2023-10-06T22:13:05.739124+0000","last_undegraded":"2023-10-06T22:13:05.739124+0000","last_fullsized":"2023-10-06T22:13:05.739124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:30.928083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727798+0000","last_change":"2023-10-06T22:12:42.692151+0000","last_active":"2023-10-06T22:13:05.727798+0000","last_peered":"2023-10-06T22:13:05.727798+0000","last_clean":"2023-10-06T22:13:05.727798+0000","last_became_active":"2023-10-06T22:12:42.691780+0000","last_became_peered":"2023-10-06T22:12:42.691780+0000","last_unstale":"2023-10-06T22:13:05.727798+0000","last_undegraded":"2023-10-06T22:13:05.727798+0000","last_fullsized":"2023-10-06T22:13:05.727798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:49:35.697733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745577+0000","last_change":"2023-10-06T22:12:43.701754+0000","last_active":"2023-10-06T22:13:05.745577+0000","last_peered":"2023-10-06T22:13:05.745577+0000","last_clean":"2023-10-06T22:13:05.745577+0000","last_became_active":"2023-10-06T22:12:43.700881+0000","last_became_peered":"2023-10-06T22:12:43.700881+0000","last_unstale":"2023-10-06T22:13:05.745577+0000","last_undegraded":"2023-10-06T22:13:05.745577+0000","last_fullsized":"2023-10-06T22:13:05.745577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:35:21.988307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723824+0000","last_change":"2023-10-06T22:12:42.699751+0000","last_active":"2023-10-06T22:13:05.723824+0000","last_peered":"2023-10-06T22:13:05.723824+0000","last_clean":"2023-10-06T22:13:05.723824+0000","last_became_active":"2023-10-06T22:12:42.699321+0000","last_became_peered":"2023-10-06T22:12:42.699321+0000","last_unstale":"2023-10-06T22:13:05.723824+0000","last_undegraded":"2023-10-06T22:13:05.723824+0000","last_fullsized":"2023-10-06T22:13:05.723824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:35:25.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738929+0000","last_change":"2023-10-06T22:12:43.702426+0000","last_active":"2023-10-06T22:13:05.738929+0000","last_peered":"2023-10-06T22:13:05.738929+0000","last_clean":"2023-10-06T22:13:05.738929+0000","last_became_active":"2023-10-06T22:12:43.702101+0000","last_became_peered":"2023-10-06T22:12:43.702101+0000","last_unstale":"2023-10-06T22:13:05.738929+0000","last_undegraded":"2023-10-06T22:13:05.738929+0000","last_fullsized":"2023-10-06T22:13:05.738929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:31:09.133941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":135,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:32.729366+0000","last_change":"2023-10-06T22:12:42.688815+0000","last_active":"2023-10-06T22:13:32.729366+0000","last_peered":"2023-10-06T22:13:32.729366+0000","last_clean":"2023-10-06T22:13:32.729366+0000","last_became_active":"2023-10-06T22:12:42.688447+0000","last_became_peered":"2023-10-06T22:12:42.688447+0000","last_unstale":"2023-10-06T22:13:32.729366+0000","last_undegraded":"2023-10-06T22:13:32.729366+0000","last_fullsized":"2023-10-06T22:13:32.729366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:26:25.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57508,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":92,"num_read_kb":102,"num_write":30,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724127+0000","last_change":"2023-10-06T22:12:43.698406+0000","last_active":"2023-10-06T22:13:05.724127+0000","last_peered":"2023-10-06T22:13:05.724127+0000","last_clean":"2023-10-06T22:13:05.724127+0000","last_became_active":"2023-10-06T22:12:43.697204+0000","last_became_peered":"2023-10-06T22:12:43.697204+0000","last_unstale":"2023-10-06T22:13:05.724127+0000","last_undegraded":"2023-10-06T22:13:05.724127+0000","last_fullsized":"2023-10-06T22:13:05.724127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:59.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727727+0000","last_change":"2023-10-06T22:12:42.691007+0000","last_active":"2023-10-06T22:13:05.727727+0000","last_peered":"2023-10-06T22:13:05.727727+0000","last_clean":"2023-10-06T22:13:05.727727+0000","last_became_active":"2023-10-06T22:12:42.690693+0000","last_became_peered":"2023-10-06T22:12:42.690693+0000","last_unstale":"2023-10-06T22:13:05.727727+0000","last_undegraded":"2023-10-06T22:13:05.727727+0000","last_fullsized":"2023-10-06T22:13:05.727727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:47.670091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.421929+0000","last_change":"2023-10-06T22:12:43.705131+0000","last_active":"2023-10-06T22:13:06.421929+0000","last_peered":"2023-10-06T22:13:06.421929+0000","last_clean":"2023-10-06T22:13:06.421929+0000","last_became_active":"2023-10-06T22:12:43.704469+0000","last_became_peered":"2023-10-06T22:12:43.704469+0000","last_unstale":"2023-10-06T22:13:06.421929+0000","last_undegraded":"2023-10-06T22:13:06.421929+0000","last_fullsized":"2023-10-06T22:13:06.421929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:36:39.392274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723093+0000","last_change":"2023-10-06T22:12:42.698392+0000","last_active":"2023-10-06T22:13:05.723093+0000","last_peered":"2023-10-06T22:13:05.723093+0000","last_clean":"2023-10-06T22:13:05.723093+0000","last_became_active":"2023-10-06T22:12:42.698019+0000","last_became_peered":"2023-10-06T22:12:42.698019+0000","last_unstale":"2023-10-06T22:13:05.723093+0000","last_undegraded":"2023-10-06T22:13:05.723093+0000","last_fullsized":"2023-10-06T22:13:05.723093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:00.345300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727760+0000","last_change":"2023-10-06T22:12:43.693238+0000","last_active":"2023-10-06T22:13:05.727760+0000","last_peered":"2023-10-06T22:13:05.727760+0000","last_clean":"2023-10-06T22:13:05.727760+0000","last_became_active":"2023-10-06T22:12:43.692948+0000","last_became_peered":"2023-10-06T22:12:43.692948+0000","last_unstale":"2023-10-06T22:13:05.727760+0000","last_undegraded":"2023-10-06T22:13:05.727760+0000","last_fullsized":"2023-10-06T22:13:05.727760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:24:30.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:31.733392+0000","last_change":"2023-10-06T22:12:42.693213+0000","last_active":"2023-10-06T22:13:31.733392+0000","last_peered":"2023-10-06T22:13:31.733392+0000","last_clean":"2023-10-06T22:13:31.733392+0000","last_became_active":"2023-10-06T22:12:42.692894+0000","last_became_peered":"2023-10-06T22:12:42.692894+0000","last_unstale":"2023-10-06T22:13:31.733392+0000","last_undegraded":"2023-10-06T22:13:31.733392+0000","last_fullsized":"2023-10-06T22:13:31.733392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:40:21.159102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727338+0000","last_change":"2023-10-06T22:12:43.695527+0000","last_active":"2023-10-06T22:13:05.727338+0000","last_peered":"2023-10-06T22:13:05.727338+0000","last_clean":"2023-10-06T22:13:05.727338+0000","last_became_active":"2023-10-06T22:12:43.694364+0000","last_became_peered":"2023-10-06T22:12:43.694364+0000","last_unstale":"2023-10-06T22:13:05.727338+0000","last_undegraded":"2023-10-06T22:13:05.727338+0000","last_fullsized":"2023-10-06T22:13:05.727338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723938+0000","last_change":"2023-10-06T22:12:42.697854+0000","last_active":"2023-10-06T22:13:05.723938+0000","last_peered":"2023-10-06T22:13:05.723938+0000","last_clean":"2023-10-06T22:13:05.723938+0000","last_became_active":"2023-10-06T22:12:42.697542+0000","last_became_peered":"2023-10-06T22:12:42.697542+0000","last_unstale":"2023-10-06T22:13:05.723938+0000","last_undegraded":"2023-10-06T22:13:05.723938+0000","last_fullsized":"2023-10-06T22:13:05.723938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:12:11.181398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722884+0000","last_change":"2023-10-06T22:12:43.698070+0000","last_active":"2023-10-06T22:13:05.722884+0000","last_peered":"2023-10-06T22:13:05.722884+0000","last_clean":"2023-10-06T22:13:05.722884+0000","last_became_active":"2023-10-06T22:12:43.697682+0000","last_became_peered":"2023-10-06T22:12:43.697682+0000","last_unstale":"2023-10-06T22:13:05.722884+0000","last_undegraded":"2023-10-06T22:13:05.722884+0000","last_fullsized":"2023-10-06T22:13:05.722884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:43.808866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724573+0000","last_change":"2023-10-06T22:12:42.691691+0000","last_active":"2023-10-06T22:13:05.724573+0000","last_peered":"2023-10-06T22:13:05.724573+0000","last_clean":"2023-10-06T22:13:05.724573+0000","last_became_active":"2023-10-06T22:12:42.691001+0000","last_became_peered":"2023-10-06T22:12:42.691001+0000","last_unstale":"2023-10-06T22:13:05.724573+0000","last_undegraded":"2023-10-06T22:13:05.724573+0000","last_fullsized":"2023-10-06T22:13:05.724573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:11:18.559570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.927620+0000","last_change":"2023-10-06T22:13:06.927620+0000","last_active":"2023-10-06T22:13:06.927620+0000","last_peered":"2023-10-06T22:13:06.927620+0000","last_clean":"2023-10-06T22:13:06.927620+0000","last_became_active":"2023-10-06T22:13:06.927206+0000","last_became_peered":"2023-10-06T22:13:06.927206+0000","last_unstale":"2023-10-06T22:13:06.927620+0000","last_undegraded":"2023-10-06T22:13:06.927620+0000","last_fullsized":"2023-10-06T22:13:06.927620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:50:30.780174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.726783+0000","last_change":"2023-10-06T22:13:06.726783+0000","last_active":"2023-10-06T22:13:06.726783+0000","last_peered":"2023-10-06T22:13:06.726783+0000","last_clean":"2023-10-06T22:13:06.726783+0000","last_became_active":"2023-10-06T22:13:06.726342+0000","last_became_peered":"2023-10-06T22:13:06.726342+0000","last_unstale":"2023-10-06T22:13:06.726783+0000","last_undegraded":"2023-10-06T22:13:06.726783+0000","last_fullsized":"2023-10-06T22:13:06.726783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:42.359819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720194+0000","last_change":"2023-10-06T22:12:43.702870+0000","last_active":"2023-10-06T22:13:05.720194+0000","last_peered":"2023-10-06T22:13:05.720194+0000","last_clean":"2023-10-06T22:13:05.720194+0000","last_became_active":"2023-10-06T22:12:43.702564+0000","last_became_peered":"2023-10-06T22:12:43.702564+0000","last_unstale":"2023-10-06T22:13:05.720194+0000","last_undegraded":"2023-10-06T22:13:05.720194+0000","last_fullsized":"2023-10-06T22:13:05.720194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:23:32.238268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:32.762998+0000","last_change":"2023-10-06T22:12:42.696603+0000","last_active":"2023-10-06T22:13:32.762998+0000","last_peered":"2023-10-06T22:13:32.762998+0000","last_clean":"2023-10-06T22:13:32.762998+0000","last_became_active":"2023-10-06T22:12:42.696313+0000","last_became_peered":"2023-10-06T22:12:42.696313+0000","last_unstale":"2023-10-06T22:13:32.762998+0000","last_undegraded":"2023-10-06T22:13:32.762998+0000","last_fullsized":"2023-10-06T22:13:32.762998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:15.719749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720013+0000","last_change":"2023-10-06T22:12:43.703356+0000","last_active":"2023-10-06T22:13:05.720013+0000","last_peered":"2023-10-06T22:13:05.720013+0000","last_clean":"2023-10-06T22:13:05.720013+0000","last_became_active":"2023-10-06T22:12:43.702670+0000","last_became_peered":"2023-10-06T22:12:43.702670+0000","last_unstale":"2023-10-06T22:13:05.720013+0000","last_undegraded":"2023-10-06T22:13:05.720013+0000","last_fullsized":"2023-10-06T22:13:05.720013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:15:05.681491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722900+0000","last_change":"2023-10-06T22:12:43.699083+0000","last_active":"2023-10-06T22:13:05.722900+0000","last_peered":"2023-10-06T22:13:05.722900+0000","last_clean":"2023-10-06T22:13:05.722900+0000","last_became_active":"2023-10-06T22:12:43.698826+0000","last_became_peered":"2023-10-06T22:12:43.698826+0000","last_unstale":"2023-10-06T22:13:05.722900+0000","last_undegraded":"2023-10-06T22:13:05.722900+0000","last_fullsized":"2023-10-06T22:13:05.722900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:04.225060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:30.731177+0000","last_change":"2023-10-06T22:12:42.699873+0000","last_active":"2023-10-06T22:13:30.731177+0000","last_peered":"2023-10-06T22:13:30.731177+0000","last_clean":"2023-10-06T22:13:30.731177+0000","last_became_active":"2023-10-06T22:12:42.699476+0000","last_became_peered":"2023-10-06T22:12:42.699476+0000","last_unstale":"2023-10-06T22:13:30.731177+0000","last_undegraded":"2023-10-06T22:13:30.731177+0000","last_fullsized":"2023-10-06T22:13:30.731177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:13:27.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751789+0000","last_change":"2023-10-06T22:12:43.697720+0000","last_active":"2023-10-06T22:13:05.751789+0000","last_peered":"2023-10-06T22:13:05.751789+0000","last_clean":"2023-10-06T22:13:05.751789+0000","last_became_active":"2023-10-06T22:12:43.697430+0000","last_became_peered":"2023-10-06T22:12:43.697430+0000","last_unstale":"2023-10-06T22:13:05.751789+0000","last_undegraded":"2023-10-06T22:13:05.751789+0000","last_fullsized":"2023-10-06T22:13:05.751789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:29.633012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722760+0000","last_change":"2023-10-06T22:12:42.690781+0000","last_active":"2023-10-06T22:13:05.722760+0000","last_peered":"2023-10-06T22:13:05.722760+0000","last_clean":"2023-10-06T22:13:05.722760+0000","last_became_active":"2023-10-06T22:12:42.690274+0000","last_became_peered":"2023-10-06T22:12:42.690274+0000","last_unstale":"2023-10-06T22:13:05.722760+0000","last_undegraded":"2023-10-06T22:13:05.722760+0000","last_fullsized":"2023-10-06T22:13:05.722760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:26.653182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727428+0000","last_change":"2023-10-06T22:11:36.409669+0000","last_active":"2023-10-06T22:13:05.727428+0000","last_peered":"2023-10-06T22:13:05.727428+0000","last_clean":"2023-10-06T22:13:05.727428+0000","last_became_active":"2023-10-06T22:11:36.409197+0000","last_became_peered":"2023-10-06T22:11:36.409197+0000","last_unstale":"2023-10-06T22:13:05.727428+0000","last_undegraded":"2023-10-06T22:13:05.727428+0000","last_fullsized":"2023-10-06T22:13:05.727428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_clean_scrub_stamp":"2023-10-06T22:08:24.598743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:50:55.982571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.423067+0000","last_change":"2023-10-06T22:12:43.698617+0000","last_active":"2023-10-06T22:13:06.423067+0000","last_peered":"2023-10-06T22:13:06.423067+0000","last_clean":"2023-10-06T22:13:06.423067+0000","last_became_active":"2023-10-06T22:12:43.697942+0000","last_became_peered":"2023-10-06T22:12:43.697942+0000","last_unstale":"2023-10-06T22:13:06.423067+0000","last_undegraded":"2023-10-06T22:13:06.423067+0000","last_fullsized":"2023-10-06T22:13:06.423067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:38.075730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727545+0000","last_change":"2023-10-06T22:12:42.695553+0000","last_active":"2023-10-06T22:13:05.727545+0000","last_peered":"2023-10-06T22:13:05.727545+0000","last_clean":"2023-10-06T22:13:05.727545+0000","last_became_active":"2023-10-06T22:12:42.695209+0000","last_became_peered":"2023-10-06T22:12:42.695209+0000","last_unstale":"2023-10-06T22:13:05.727545+0000","last_undegraded":"2023-10-06T22:13:05.727545+0000","last_fullsized":"2023-10-06T22:13:05.727545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:10.387710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724270+0000","last_change":"2023-10-06T22:12:42.699606+0000","last_active":"2023-10-06T22:13:05.724270+0000","last_peered":"2023-10-06T22:13:05.724270+0000","last_clean":"2023-10-06T22:13:05.724270+0000","last_became_active":"2023-10-06T22:12:42.699222+0000","last_became_peered":"2023-10-06T22:12:42.699222+0000","last_unstale":"2023-10-06T22:13:05.724270+0000","last_undegraded":"2023-10-06T22:13:05.724270+0000","last_fullsized":"2023-10-06T22:13:05.724270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:52.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422535+0000","last_change":"2023-10-06T22:12:43.697783+0000","last_active":"2023-10-06T22:13:06.422535+0000","last_peered":"2023-10-06T22:13:06.422535+0000","last_clean":"2023-10-06T22:13:06.422535+0000","last_became_active":"2023-10-06T22:12:43.697388+0000","last_became_peered":"2023-10-06T22:12:43.697388+0000","last_unstale":"2023-10-06T22:13:06.422535+0000","last_undegraded":"2023-10-06T22:13:06.422535+0000","last_fullsized":"2023-10-06T22:13:06.422535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:50.113458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.735903+0000","last_change":"2023-10-06T22:12:42.694180+0000","last_active":"2023-10-06T22:13:05.735903+0000","last_peered":"2023-10-06T22:13:05.735903+0000","last_clean":"2023-10-06T22:13:05.735903+0000","last_became_active":"2023-10-06T22:12:42.693861+0000","last_became_peered":"2023-10-06T22:12:42.693861+0000","last_unstale":"2023-10-06T22:13:05.735903+0000","last_undegraded":"2023-10-06T22:13:05.735903+0000","last_fullsized":"2023-10-06T22:13:05.735903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:28:02.511117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.753037+0000","last_change":"2023-10-06T22:12:43.699118+0000","last_active":"2023-10-06T22:13:05.753037+0000","last_peered":"2023-10-06T22:13:05.753037+0000","last_clean":"2023-10-06T22:13:05.753037+0000","last_became_active":"2023-10-06T22:12:43.698540+0000","last_became_peered":"2023-10-06T22:12:43.698540+0000","last_unstale":"2023-10-06T22:13:05.753037+0000","last_undegraded":"2023-10-06T22:13:05.753037+0000","last_fullsized":"2023-10-06T22:13:05.753037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:31.192762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722961+0000","last_change":"2023-10-06T22:12:42.690617+0000","last_active":"2023-10-06T22:13:05.722961+0000","last_peered":"2023-10-06T22:13:05.722961+0000","last_clean":"2023-10-06T22:13:05.722961+0000","last_became_active":"2023-10-06T22:12:42.690238+0000","last_became_peered":"2023-10-06T22:12:42.690238+0000","last_unstale":"2023-10-06T22:13:05.722961+0000","last_undegraded":"2023-10-06T22:13:05.722961+0000","last_fullsized":"2023-10-06T22:13:05.722961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:09.819570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745288+0000","last_change":"2023-10-06T22:12:43.699064+0000","last_active":"2023-10-06T22:13:05.745288+0000","last_peered":"2023-10-06T22:13:05.745288+0000","last_clean":"2023-10-06T22:13:05.745288+0000","last_became_active":"2023-10-06T22:12:43.698727+0000","last_became_peered":"2023-10-06T22:12:43.698727+0000","last_unstale":"2023-10-06T22:13:05.745288+0000","last_undegraded":"2023-10-06T22:13:05.745288+0000","last_fullsized":"2023-10-06T22:13:05.745288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:48:14.493376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738711+0000","last_change":"2023-10-06T22:12:42.693779+0000","last_active":"2023-10-06T22:13:05.738711+0000","last_peered":"2023-10-06T22:13:05.738711+0000","last_clean":"2023-10-06T22:13:05.738711+0000","last_became_active":"2023-10-06T22:12:42.693257+0000","last_became_peered":"2023-10-06T22:12:42.693257+0000","last_unstale":"2023-10-06T22:13:05.738711+0000","last_undegraded":"2023-10-06T22:13:05.738711+0000","last_fullsized":"2023-10-06T22:13:05.738711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:56:48.945078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723955+0000","last_change":"2023-10-06T22:12:43.700620+0000","last_active":"2023-10-06T22:13:05.723955+0000","last_peered":"2023-10-06T22:13:05.723955+0000","last_clean":"2023-10-06T22:13:05.723955+0000","last_became_active":"2023-10-06T22:12:43.700093+0000","last_became_peered":"2023-10-06T22:12:43.700093+0000","last_unstale":"2023-10-06T22:13:05.723955+0000","last_undegraded":"2023-10-06T22:13:05.723955+0000","last_fullsized":"2023-10-06T22:13:05.723955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:29.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751425+0000","last_change":"2023-10-06T22:12:42.689640+0000","last_active":"2023-10-06T22:13:05.751425+0000","last_peered":"2023-10-06T22:13:05.751425+0000","last_clean":"2023-10-06T22:13:05.751425+0000","last_became_active":"2023-10-06T22:12:42.688584+0000","last_became_peered":"2023-10-06T22:12:42.688584+0000","last_unstale":"2023-10-06T22:13:05.751425+0000","last_undegraded":"2023-10-06T22:13:05.751425+0000","last_fullsized":"2023-10-06T22:13:05.751425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:43.066396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720226+0000","last_change":"2023-10-06T22:12:43.700084+0000","last_active":"2023-10-06T22:13:05.720226+0000","last_peered":"2023-10-06T22:13:05.720226+0000","last_clean":"2023-10-06T22:13:05.720226+0000","last_became_active":"2023-10-06T22:12:43.699787+0000","last_became_peered":"2023-10-06T22:12:43.699787+0000","last_unstale":"2023-10-06T22:13:05.720226+0000","last_undegraded":"2023-10-06T22:13:05.720226+0000","last_fullsized":"2023-10-06T22:13:05.720226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:07.219800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727627+0000","last_change":"2023-10-06T22:12:42.688750+0000","last_active":"2023-10-06T22:13:05.727627+0000","last_peered":"2023-10-06T22:13:05.727627+0000","last_clean":"2023-10-06T22:13:05.727627+0000","last_became_active":"2023-10-06T22:12:42.688387+0000","last_became_peered":"2023-10-06T22:12:42.688387+0000","last_unstale":"2023-10-06T22:13:05.727627+0000","last_undegraded":"2023-10-06T22:13:05.727627+0000","last_fullsized":"2023-10-06T22:13:05.727627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:32.980631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752151+0000","last_change":"2023-10-06T22:12:43.706741+0000","last_active":"2023-10-06T22:13:05.752151+0000","last_peered":"2023-10-06T22:13:05.752151+0000","last_clean":"2023-10-06T22:13:05.752151+0000","last_became_active":"2023-10-06T22:12:43.706416+0000","last_became_peered":"2023-10-06T22:12:43.706416+0000","last_unstale":"2023-10-06T22:13:05.752151+0000","last_undegraded":"2023-10-06T22:13:05.752151+0000","last_fullsized":"2023-10-06T22:13:05.752151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:55:16.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724994+0000","last_change":"2023-10-06T22:12:42.692850+0000","last_active":"2023-10-06T22:13:05.724994+0000","last_peered":"2023-10-06T22:13:05.724994+0000","last_clean":"2023-10-06T22:13:05.724994+0000","last_became_active":"2023-10-06T22:12:42.692287+0000","last_became_peered":"2023-10-06T22:12:42.692287+0000","last_unstale":"2023-10-06T22:13:05.724994+0000","last_undegraded":"2023-10-06T22:13:05.724994+0000","last_fullsized":"2023-10-06T22:13:05.724994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:37:59.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727726+0000","last_change":"2023-10-06T22:12:43.700667+0000","last_active":"2023-10-06T22:13:05.727726+0000","last_peered":"2023-10-06T22:13:05.727726+0000","last_clean":"2023-10-06T22:13:05.727726+0000","last_became_active":"2023-10-06T22:12:43.698758+0000","last_became_peered":"2023-10-06T22:12:43.698758+0000","last_unstale":"2023-10-06T22:13:05.727726+0000","last_undegraded":"2023-10-06T22:13:05.727726+0000","last_fullsized":"2023-10-06T22:13:05.727726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:49:53.091578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:32.730010+0000","last_change":"2023-10-06T22:12:42.695964+0000","last_active":"2023-10-06T22:13:32.730010+0000","last_peered":"2023-10-06T22:13:32.730010+0000","last_clean":"2023-10-06T22:13:32.730010+0000","last_became_active":"2023-10-06T22:12:42.695431+0000","last_became_peered":"2023-10-06T22:12:42.695431+0000","last_unstale":"2023-10-06T22:13:32.730010+0000","last_undegraded":"2023-10-06T22:13:32.730010+0000","last_fullsized":"2023-10-06T22:13:32.730010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:21:42.761421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.728382+0000","last_change":"2023-10-06T22:12:43.698088+0000","last_active":"2023-10-06T22:13:05.728382+0000","last_peered":"2023-10-06T22:13:05.728382+0000","last_clean":"2023-10-06T22:13:05.728382+0000","last_became_active":"2023-10-06T22:12:43.697765+0000","last_became_peered":"2023-10-06T22:12:43.697765+0000","last_unstale":"2023-10-06T22:13:05.728382+0000","last_undegraded":"2023-10-06T22:13:05.728382+0000","last_fullsized":"2023-10-06T22:13:05.728382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:26:22.958348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723709+0000","last_change":"2023-10-06T22:12:42.698586+0000","last_active":"2023-10-06T22:13:05.723709+0000","last_peered":"2023-10-06T22:13:05.723709+0000","last_clean":"2023-10-06T22:13:05.723709+0000","last_became_active":"2023-10-06T22:12:42.698198+0000","last_became_peered":"2023-10-06T22:12:42.698198+0000","last_unstale":"2023-10-06T22:13:05.723709+0000","last_undegraded":"2023-10-06T22:13:05.723709+0000","last_fullsized":"2023-10-06T22:13:05.723709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:03:43.898987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727662+0000","last_change":"2023-10-06T22:12:43.698615+0000","last_active":"2023-10-06T22:13:05.727662+0000","last_peered":"2023-10-06T22:13:05.727662+0000","last_clean":"2023-10-06T22:13:05.727662+0000","last_became_active":"2023-10-06T22:12:43.698302+0000","last_became_peered":"2023-10-06T22:12:43.698302+0000","last_unstale":"2023-10-06T22:13:05.727662+0000","last_undegraded":"2023-10-06T22:13:05.727662+0000","last_fullsized":"2023-10-06T22:13:05.727662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:35.973119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422877+0000","last_change":"2023-10-06T22:12:42.700325+0000","last_active":"2023-10-06T22:13:06.422877+0000","last_peered":"2023-10-06T22:13:06.422877+0000","last_clean":"2023-10-06T22:13:06.422877+0000","last_became_active":"2023-10-06T22:12:42.699426+0000","last_became_peered":"2023-10-06T22:12:42.699426+0000","last_unstale":"2023-10-06T22:13:06.422877+0000","last_undegraded":"2023-10-06T22:13:06.422877+0000","last_fullsized":"2023-10-06T22:13:06.422877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:09.649737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727916+0000","last_change":"2023-10-06T22:12:43.698620+0000","last_active":"2023-10-06T22:13:05.727916+0000","last_peered":"2023-10-06T22:13:05.727916+0000","last_clean":"2023-10-06T22:13:05.727916+0000","last_became_active":"2023-10-06T22:12:43.698335+0000","last_became_peered":"2023-10-06T22:12:43.698335+0000","last_unstale":"2023-10-06T22:13:05.727916+0000","last_undegraded":"2023-10-06T22:13:05.727916+0000","last_fullsized":"2023-10-06T22:13:05.727916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:49.222848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720263+0000","last_change":"2023-10-06T22:12:43.700477+0000","last_active":"2023-10-06T22:13:05.720263+0000","last_peered":"2023-10-06T22:13:05.720263+0000","last_clean":"2023-10-06T22:13:05.720263+0000","last_became_active":"2023-10-06T22:12:43.700177+0000","last_became_peered":"2023-10-06T22:12:43.700177+0000","last_unstale":"2023-10-06T22:13:05.720263+0000","last_undegraded":"2023-10-06T22:13:05.720263+0000","last_fullsized":"2023-10-06T22:13:05.720263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:11.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727515+0000","last_change":"2023-10-06T22:12:43.701792+0000","last_active":"2023-10-06T22:13:05.727515+0000","last_peered":"2023-10-06T22:13:05.727515+0000","last_clean":"2023-10-06T22:13:05.727515+0000","last_became_active":"2023-10-06T22:12:43.701316+0000","last_became_peered":"2023-10-06T22:12:43.701316+0000","last_unstale":"2023-10-06T22:13:05.727515+0000","last_undegraded":"2023-10-06T22:13:05.727515+0000","last_fullsized":"2023-10-06T22:13:05.727515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:03.085219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727616+0000","last_change":"2023-10-06T22:12:43.707871+0000","last_active":"2023-10-06T22:13:05.727616+0000","last_peered":"2023-10-06T22:13:05.727616+0000","last_clean":"2023-10-06T22:13:05.727616+0000","last_became_active":"2023-10-06T22:12:43.707659+0000","last_became_peered":"2023-10-06T22:12:43.707659+0000","last_unstale":"2023-10-06T22:13:05.727616+0000","last_undegraded":"2023-10-06T22:13:05.727616+0000","last_fullsized":"2023-10-06T22:13:05.727616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:43.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744632+0000","last_change":"2023-10-06T22:12:43.697701+0000","last_active":"2023-10-06T22:13:05.744632+0000","last_peered":"2023-10-06T22:13:05.744632+0000","last_clean":"2023-10-06T22:13:05.744632+0000","last_became_active":"2023-10-06T22:12:43.697361+0000","last_became_peered":"2023-10-06T22:12:43.697361+0000","last_unstale":"2023-10-06T22:13:05.744632+0000","last_undegraded":"2023-10-06T22:13:05.744632+0000","last_fullsized":"2023-10-06T22:13:05.744632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:14:53.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422617+0000","last_change":"2023-10-06T22:12:43.700058+0000","last_active":"2023-10-06T22:13:06.422617+0000","last_peered":"2023-10-06T22:13:06.422617+0000","last_clean":"2023-10-06T22:13:06.422617+0000","last_became_active":"2023-10-06T22:12:43.697893+0000","last_became_peered":"2023-10-06T22:12:43.697893+0000","last_unstale":"2023-10-06T22:13:06.422617+0000","last_undegraded":"2023-10-06T22:13:06.422617+0000","last_fullsized":"2023-10-06T22:13:06.422617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:01:37.518096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.729307+0000","last_change":"2023-10-06T22:13:06.729307+0000","last_active":"2023-10-06T22:13:06.729307+0000","last_peered":"2023-10-06T22:13:06.729307+0000","last_clean":"2023-10-06T22:13:06.729307+0000","last_became_active":"2023-10-06T22:13:06.728877+0000","last_became_peered":"2023-10-06T22:13:06.728877+0000","last_unstale":"2023-10-06T22:13:06.729307+0000","last_undegraded":"2023-10-06T22:13:06.729307+0000","last_fullsized":"2023-10-06T22:13:06.729307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:17.629046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.728370+0000","last_change":"2023-10-06T22:12:43.700668+0000","last_active":"2023-10-06T22:13:05.728370+0000","last_peered":"2023-10-06T22:13:05.728370+0000","last_clean":"2023-10-06T22:13:05.728370+0000","last_became_active":"2023-10-06T22:12:43.700387+0000","last_became_peered":"2023-10-06T22:12:43.700387+0000","last_unstale":"2023-10-06T22:13:05.728370+0000","last_undegraded":"2023-10-06T22:13:05.728370+0000","last_fullsized":"2023-10-06T22:13:05.728370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:36:00.053496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752951+0000","last_change":"2023-10-06T22:12:43.702658+0000","last_active":"2023-10-06T22:13:05.752951+0000","last_peered":"2023-10-06T22:13:05.752951+0000","last_clean":"2023-10-06T22:13:05.752951+0000","last_became_active":"2023-10-06T22:12:43.702264+0000","last_became_peered":"2023-10-06T22:12:43.702264+0000","last_unstale":"2023-10-06T22:13:05.752951+0000","last_undegraded":"2023-10-06T22:13:05.752951+0000","last_fullsized":"2023-10-06T22:13:05.752951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:56.472029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722942+0000","last_change":"2023-10-06T22:12:43.705718+0000","last_active":"2023-10-06T22:13:05.722942+0000","last_peered":"2023-10-06T22:13:05.722942+0000","last_clean":"2023-10-06T22:13:05.722942+0000","last_became_active":"2023-10-06T22:12:43.705479+0000","last_became_peered":"2023-10-06T22:12:43.705479+0000","last_unstale":"2023-10-06T22:13:05.722942+0000","last_undegraded":"2023-10-06T22:13:05.722942+0000","last_fullsized":"2023-10-06T22:13:05.722942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:55:36.393577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751324+0000","last_change":"2023-10-06T22:12:43.701891+0000","last_active":"2023-10-06T22:13:05.751324+0000","last_peered":"2023-10-06T22:13:05.751324+0000","last_clean":"2023-10-06T22:13:05.751324+0000","last_became_active":"2023-10-06T22:12:43.701535+0000","last_became_peered":"2023-10-06T22:12:43.701535+0000","last_unstale":"2023-10-06T22:13:05.751324+0000","last_undegraded":"2023-10-06T22:13:05.751324+0000","last_fullsized":"2023-10-06T22:13:05.751324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:17:40.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738839+0000","last_change":"2023-10-06T22:12:43.707075+0000","last_active":"2023-10-06T22:13:05.738839+0000","last_peered":"2023-10-06T22:13:05.738839+0000","last_clean":"2023-10-06T22:13:05.738839+0000","last_became_active":"2023-10-06T22:12:43.706723+0000","last_became_peered":"2023-10-06T22:12:43.706723+0000","last_unstale":"2023-10-06T22:13:05.738839+0000","last_undegraded":"2023-10-06T22:13:05.738839+0000","last_fullsized":"2023-10-06T22:13:05.738839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:18.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422288+0000","last_change":"2023-10-06T22:12:43.704656+0000","last_active":"2023-10-06T22:13:06.422288+0000","last_peered":"2023-10-06T22:13:06.422288+0000","last_clean":"2023-10-06T22:13:06.422288+0000","last_became_active":"2023-10-06T22:12:43.704312+0000","last_became_peered":"2023-10-06T22:12:43.704312+0000","last_unstale":"2023-10-06T22:13:06.422288+0000","last_undegraded":"2023-10-06T22:13:06.422288+0000","last_fullsized":"2023-10-06T22:13:06.422288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:21:37.951180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720341+0000","last_change":"2023-10-06T22:12:43.702239+0000","last_active":"2023-10-06T22:13:05.720341+0000","last_peered":"2023-10-06T22:13:05.720341+0000","last_clean":"2023-10-06T22:13:05.720341+0000","last_became_active":"2023-10-06T22:12:43.701851+0000","last_became_peered":"2023-10-06T22:12:43.701851+0000","last_unstale":"2023-10-06T22:13:05.720341+0000","last_undegraded":"2023-10-06T22:13:05.720341+0000","last_fullsized":"2023-10-06T22:13:05.720341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:40:53.661587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65100,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":465,"num_read_kb":299,"num_write":230,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":195300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449111,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":444618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42399999999999999}]},{"osd":1,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.499},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47599999999999998}]},{"osd":6,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.45300000000000001}]},{"osd":7,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.62},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.63500000000000001}]},{"osd":10,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.65200000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1092830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.442}]},{"osd":5,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.376},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.505}]},{"osd":8,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.73599999999999999}]},{"osd":9,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.78700000000000003}]},{"osd":11,"last update":"Fri Oct 6 22:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66700000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841568,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":444158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60999999999999999,"15min":0.621},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.99399999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97199999999999998,"5min":1.036,"15min":1.036},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.83099999999999996,"15min":0.84699999999999998},"min":{"1min":0.53500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95699999999999996,"5min":1.486,"15min":1.486},"last":0.73699999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78900000000000003,"15min":0.80500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0349999999999999,"5min":1.548,"15min":1.548},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91500000000000004,"15min":0.93100000000000005},"min":{"1min":0.66500000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.99099999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.95399999999999996},"min":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.0369999999999999,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.0149999999999999,"15min":1.0369999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1879999999999999,"5min":1.585,"15min":1.585},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2090000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.49299999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.998,"15min":1.016},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3129999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":0.91200000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2769999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.194,"5min":1.268,"15min":1.268},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.997,"15min":1.018},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.60599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":443533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.496},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.30199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.623,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.80500000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.50900000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90300000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.86299999999999999}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.96299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1037176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.33600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44,"15min":0.436},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.627,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.626}]},{"osd":3,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65700000000000003,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.875,"15min":0.875},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.51500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.751}]},{"osd":9,"last update":"Fri Oct 6 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86899999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":443608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.45700000000000002},"min":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60899999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70599999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.443}]},{"osd":1,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.83099999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.54200000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88300000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85799999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82399999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.022,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.66600000000000004},"min":{"1min":0.61299999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89200000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.54300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.107,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.05,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.626}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79300000000000004,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99399999999999999,"5min":1.173,"15min":1.173},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83899999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97699999999999998,"5min":1.145,"15min":1.145},"last":0.65600000000000003}]},{"osd":8,"last update":"Fri Oct 6 22:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.876,"15min":0.88400000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.014,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.86199999999999999,"15min":0.86499999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.88600000000000001,"15min":0.90400000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.90800000000000003,"15min":0.92500000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":0.96099999999999997,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.82299999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.90700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":502968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.39700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.504}]},{"osd":2,"last update":"Fri Oct 6 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86099999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Oct 6 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60899999999999999,"15min":0.57299999999999995},"min":{"1min":0.48299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Oct 6 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.41499999999999998,"15min":0.38700000000000001},"min":{"1min":0.318,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.433},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.42199999999999999},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.873,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.621},{"interface":"front","average":{"1min":0.625,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65300000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.623,"15min":0.59099999999999997},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.441,"5min":0.375,"15min":0.375},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Oct 6 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55700000000000005,"15min":0.53400000000000003},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52800000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89100000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.79900000000000004,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.026,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63700000000000001}]},{"osd":10,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.71399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":445000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60699999999999998,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96499999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.57899999999999996,"15min":0.51700000000000002},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0600000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.56799999999999995},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.77000000000000002}]},{"osd":3,"last update":"Fri Oct 6 22:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.67200000000000004,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82999999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.622},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.65300000000000002},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.86899999999999999},"min":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.121,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.103,"15min":1.103},"last":0.749},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Oct 6 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76000000000000001,"15min":0.77100000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92200000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89300000000000002,"15min":0.90100000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.71599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":502965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.435,"5min":0.307,"15min":0.307},"max":{"1min":0.84299999999999997,"5min":1.125,"15min":1.125},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.503},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72799999999999998,"5min":0.873,"15min":0.873},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57699999999999996,"15min":0.50700000000000001},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Oct 6 22:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.495},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.45400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Oct 6 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.433,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.46200000000000002,"15min":0.45000000000000001},"min":{"1min":0.32200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72399999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Oct 6 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.48599999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.47499999999999998}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":1.002,"15min":1.002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89100000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.629},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.38}]},{"osd":9,"last update":"Fri Oct 6 22:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0409999999999999,"5min":1.633,"15min":1.633},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73799999999999999,"15min":0.73899999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91400000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.53300000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.75800000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":443439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.435,"15min":0.39100000000000001},"min":{"1min":0.30599999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.60799999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.45700000000000002,"15min":0.40699999999999997},"min":{"1min":0.33300000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.82199999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66700000000000004,"15min":0.60099999999999998},"min":{"1min":0.51800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84299999999999997,"5min":1.069,"15min":1.069},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66300000000000003,"15min":0.60799999999999998},"min":{"1min":0.57099999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92700000000000005,"5min":1.089,"15min":1.089},"last":0.88100000000000001}]},{"osd":2,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.54400000000000004,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88400000000000001,"5min":1.046,"15min":1.046},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70499999999999996,"15min":0.64300000000000002},"min":{"1min":0.55300000000000005,"5min":0.434,"15min":0.434},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.52800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 22:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.64200000000000002},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.95699999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.751},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95499999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.62},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76000000000000001,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0509999999999999,"15min":1.133},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.778,"5min":7.0529999999999999,"15min":7.0529999999999999},"last":0.93100000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.91700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.077,"5min":1.214,"15min":1.214},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.77000000000000002}]},{"osd":10,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.94899999999999995,"15min":0.95399999999999996},"min":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":444126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.36199999999999999},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54300000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53500000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.48699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.60899999999999999,"15min":0.57299999999999995},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.54500000000000004},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.53700000000000003},"min":{"1min":0.49399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64200000000000002,"15min":0.61699999999999999},"min":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.86899999999999999,"5min":1,"15min":1},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55300000000000005,"15min":0.52300000000000002},"min":{"1min":0.374,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83499999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61699999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.47799999999999998}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57299999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70499999999999996,"15min":0.71399999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.81699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95699999999999996,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77100000000000002}]},{"osd":10,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757300,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1556480,"data_stored":1036737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.39800000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.70999999999999996}]},{"osd":4,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.60199999999999998,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65000000000000002,"15min":0.624},"min":{"1min":0.42599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86599999999999999,"5min":1.274,"15min":1.274},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.60299999999999998},"min":{"1min":0.59299999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96599999999999997,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89900000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.77600000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63400000000000001}]},{"osd":9,"last update":"Fri Oct 6 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0249999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61399999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.89500000000000002}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T22:13:38.617 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-06T22:13:39.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:38 smithi156 ceph-mon[134358]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:39.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:38 smithi190 ceph-mon[134891]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:39.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:38 smithi057 ceph-mon[98492]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:39.359 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:39.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:39 smithi057 ceph-mon[98492]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:13:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:39 smithi190 ceph-mon[134891]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:13:40.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:39 smithi156 ceph-mon[134358]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:13:40.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.948+0000 7fa31bbd5700 1 -- 172.21.15.57:0/2392350791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa31410d930 msgr2=0x7fa31410fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:40.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.948+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/2392350791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa31410d930 0x7fa31410fd20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00b0a0 tx=0x7fa30c00b3b0 comp rx=0 tx=0).stop 2023-10-06T22:13:40.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.948+0000 7fa31bbd5700 1 -- 172.21.15.57:0/2392350791 shutdown_connections 2023-10-06T22:13:40.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.948+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/2392350791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa31410d930 0x7fa31410fd20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:40.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.948+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/2392350791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3140730e0 0x7fa3140734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:40.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.948+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/2392350791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa314073a00 0x7fa31410d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:40.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.948+0000 7fa31bbd5700 1 -- 172.21.15.57:0/2392350791 >> 172.21.15.57:0/2392350791 conn(0x7fa3140780a0 msgr2=0x7fa3140784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:40.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.949+0000 7fa31bbd5700 1 -- 172.21.15.57:0/2392350791 shutdown_connections 2023-10-06T22:13:40.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.949+0000 7fa31bbd5700 1 -- 172.21.15.57:0/2392350791 wait complete. 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.949+0000 7fa31bbd5700 1 Processor -- start 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa31bbd5700 1 -- start start 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa31bbd5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3140730e0 0x7fa314199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa31bbd5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa314073a00 0x7fa3141998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa31bbd5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31410d930 0x7fa31419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa31bbd5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31419e350 con 0x7fa314073a00 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa31bbd5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31419e4c0 con 0x7fa31410d930 2023-10-06T22:13:40.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa31bbd5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31419e680 con 0x7fa3140730e0 2023-10-06T22:13:40.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa319971700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3140730e0 0x7fa314199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:40.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa319170700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa314073a00 0x7fa3141998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:40.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa319971700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3140730e0 0x7fa314199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45834/0 (socket says 172.21.15.57:45834) 2023-10-06T22:13:40.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.950+0000 7fa319971700 1 -- 172.21.15.57:0/1939215885 learned_addr learned my addr 172.21.15.57:0/1939215885 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:40.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa31a172700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31410d930 0x7fa31419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:40.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa319971700 1 -- 172.21.15.57:0/1939215885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31410d930 msgr2=0x7fa31419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:40.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa319971700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31410d930 0x7fa31419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:40.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa319971700 1 -- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa314073a00 msgr2=0x7fa3141998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:40.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa319971700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa314073a00 0x7fa3141998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:40.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa319971700 1 -- 172.21.15.57:0/1939215885 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009d00 con 0x7fa3140730e0 2023-10-06T22:13:40.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa319170700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa314073a00 0x7fa3141998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:40.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa319971700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3140730e0 0x7fa314199390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa31000e9a0 tx=0x7fa31000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:40.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.951+0000 7fa30affd700 1 -- 172.21.15.57:0/1939215885 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310009790 con 0x7fa3140730e0 2023-10-06T22:13:40.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.952+0000 7fa30affd700 1 -- 172.21.15.57:0/1939215885 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa310010430 con 0x7fa3140730e0 2023-10-06T22:13:40.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.952+0000 7fa30affd700 1 -- 172.21.15.57:0/1939215885 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310018680 con 0x7fa3140730e0 2023-10-06T22:13:40.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.952+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31419e960 con 0x7fa3140730e0 2023-10-06T22:13:40.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.952+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3141a61c0 con 0x7fa3140730e0 2023-10-06T22:13:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.953+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31419eb80 con 0x7fa3140730e0 2023-10-06T22:13:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.955+0000 7fa30affd700 1 -- 172.21.15.57:0/1939215885 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa3100105a0 con 0x7fa3140730e0 2023-10-06T22:13:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.956+0000 7fa30affd700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa3000a1510 0x7fa3000a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.956+0000 7fa30affd700 1 -- 172.21.15.57:0/1939215885 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fa310014070 con 0x7fa3140730e0 2023-10-06T22:13:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.956+0000 7fa319170700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa3000a1510 0x7fa3000a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.957+0000 7fa319170700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa3000a1510 0x7fa3000a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa31419a9b0 tx=0x7fa30400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:40.959+0000 7fa30affd700 1 -- 172.21.15.57:0/1939215885 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa3100964b0 con 0x7fa3140730e0 2023-10-06T22:13:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:40 smithi190 ceph-mon[134891]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:41.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.164+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa314113250 con 0x7fa3000a1510 2023-10-06T22:13:41.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:40 smithi057 ceph-mon[98492]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:41.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.177+0000 7fa30affd700 1 -- 172.21.15.57:0/1939215885 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341497 (secure 0 0 0) 0x7fa314113250 con 0x7fa3000a1510 2023-10-06T22:13:41.185 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa3000a1510 msgr2=0x7fa3000a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa3000a1510 0x7fa3000a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa31419a9b0 tx=0x7fa30400b410 comp rx=0 tx=0).stop 2023-10-06T22:13:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3140730e0 msgr2=0x7fa314199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3140730e0 0x7fa314199390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa31000e9a0 tx=0x7fa31000ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 shutdown_connections 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa3000a1510 0x7fa3000a39d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa31410d930 0x7fa31419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3140730e0 0x7fa314199390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 --2- 172.21.15.57:0/1939215885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa314073a00 0x7fa3141998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 >> 172.21.15.57:0/1939215885 conn(0x7fa3140780a0 msgr2=0x7fa3140fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 shutdown_connections 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:41.183+0000 7fa31bbd5700 1 -- 172.21.15.57:0/1939215885 wait complete. 2023-10-06T22:13:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-06T22:13:41.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:40 smithi156 ceph-mon[134358]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:41.640 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2023-10-06T22:13:39.628518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655714,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":641,"num_read_kb":438,"num_write":413,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337232,"kb_used_data":12976,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604592,"statfs":{"total":1151940427776,"available":1151595102208,"internally_reserved":0,"allocated":13287424,"data_stored":7281158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720589+0000","last_change":"2023-10-06T22:12:43.704484+0000","last_active":"2023-10-06T22:13:05.720589+0000","last_peered":"2023-10-06T22:13:05.720589+0000","last_clean":"2023-10-06T22:13:05.720589+0000","last_became_active":"2023-10-06T22:12:43.703740+0000","last_became_peered":"2023-10-06T22:12:43.703740+0000","last_unstale":"2023-10-06T22:13:05.720589+0000","last_undegraded":"2023-10-06T22:13:05.720589+0000","last_fullsized":"2023-10-06T22:13:05.720589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:25.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745169+0000","last_change":"2023-10-06T22:12:43.696968+0000","last_active":"2023-10-06T22:13:05.745169+0000","last_peered":"2023-10-06T22:13:05.745169+0000","last_clean":"2023-10-06T22:13:05.745169+0000","last_became_active":"2023-10-06T22:12:43.696532+0000","last_became_peered":"2023-10-06T22:12:43.696532+0000","last_unstale":"2023-10-06T22:13:05.745169+0000","last_undegraded":"2023-10-06T22:13:05.745169+0000","last_fullsized":"2023-10-06T22:13:05.745169+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:59:50.754606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744952+0000","last_change":"2023-10-06T22:12:43.700845+0000","last_active":"2023-10-06T22:13:05.744952+0000","last_peered":"2023-10-06T22:13:05.744952+0000","last_clean":"2023-10-06T22:13:05.744952+0000","last_became_active":"2023-10-06T22:12:43.700531+0000","last_became_peered":"2023-10-06T22:12:43.700531+0000","last_unstale":"2023-10-06T22:13:05.744952+0000","last_undegraded":"2023-10-06T22:13:05.744952+0000","last_fullsized":"2023-10-06T22:13:05.744952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:18.518772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722902+0000","last_change":"2023-10-06T22:12:43.700806+0000","last_active":"2023-10-06T22:13:05.722902+0000","last_peered":"2023-10-06T22:13:05.722902+0000","last_clean":"2023-10-06T22:13:05.722902+0000","last_became_active":"2023-10-06T22:12:43.699901+0000","last_became_peered":"2023-10-06T22:12:43.699901+0000","last_unstale":"2023-10-06T22:13:05.722902+0000","last_undegraded":"2023-10-06T22:13:05.722902+0000","last_fullsized":"2023-10-06T22:13:05.722902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:41:35.906415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724450+0000","last_change":"2023-10-06T22:12:43.694776+0000","last_active":"2023-10-06T22:13:05.724450+0000","last_peered":"2023-10-06T22:13:05.724450+0000","last_clean":"2023-10-06T22:13:05.724450+0000","last_became_active":"2023-10-06T22:12:43.694397+0000","last_became_peered":"2023-10-06T22:12:43.694397+0000","last_unstale":"2023-10-06T22:13:05.724450+0000","last_undegraded":"2023-10-06T22:13:05.724450+0000","last_fullsized":"2023-10-06T22:13:05.724450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:47:43.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.719749+0000","last_change":"2023-10-06T22:12:43.703515+0000","last_active":"2023-10-06T22:13:05.719749+0000","last_peered":"2023-10-06T22:13:05.719749+0000","last_clean":"2023-10-06T22:13:05.719749+0000","last_became_active":"2023-10-06T22:12:43.703094+0000","last_became_peered":"2023-10-06T22:12:43.703094+0000","last_unstale":"2023-10-06T22:13:05.719749+0000","last_undegraded":"2023-10-06T22:13:05.719749+0000","last_fullsized":"2023-10-06T22:13:05.719749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:54:32.130131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727441+0000","last_change":"2023-10-06T22:12:43.697894+0000","last_active":"2023-10-06T22:13:05.727441+0000","last_peered":"2023-10-06T22:13:05.727441+0000","last_clean":"2023-10-06T22:13:05.727441+0000","last_became_active":"2023-10-06T22:12:43.697591+0000","last_became_peered":"2023-10-06T22:12:43.697591+0000","last_unstale":"2023-10-06T22:13:05.727441+0000","last_undegraded":"2023-10-06T22:13:05.727441+0000","last_fullsized":"2023-10-06T22:13:05.727441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:30.313447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.927879+0000","last_change":"2023-10-06T22:13:06.927879+0000","last_active":"2023-10-06T22:13:06.927879+0000","last_peered":"2023-10-06T22:13:06.927879+0000","last_clean":"2023-10-06T22:13:06.927879+0000","last_became_active":"2023-10-06T22:13:06.927559+0000","last_became_peered":"2023-10-06T22:13:06.927559+0000","last_unstale":"2023-10-06T22:13:06.927879+0000","last_undegraded":"2023-10-06T22:13:06.927879+0000","last_fullsized":"2023-10-06T22:13:06.927879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:15.874159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.739167+0000","last_change":"2023-10-06T22:12:43.696995+0000","last_active":"2023-10-06T22:13:05.739167+0000","last_peered":"2023-10-06T22:13:05.739167+0000","last_clean":"2023-10-06T22:13:05.739167+0000","last_became_active":"2023-10-06T22:12:43.696731+0000","last_became_peered":"2023-10-06T22:12:43.696731+0000","last_unstale":"2023-10-06T22:13:05.739167+0000","last_undegraded":"2023-10-06T22:13:05.739167+0000","last_fullsized":"2023-10-06T22:13:05.739167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:43:31.261922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720404+0000","last_change":"2023-10-06T22:12:43.704391+0000","last_active":"2023-10-06T22:13:05.720404+0000","last_peered":"2023-10-06T22:13:05.720404+0000","last_clean":"2023-10-06T22:13:05.720404+0000","last_became_active":"2023-10-06T22:12:43.702954+0000","last_became_peered":"2023-10-06T22:12:43.702954+0000","last_unstale":"2023-10-06T22:13:05.720404+0000","last_undegraded":"2023-10-06T22:13:05.720404+0000","last_fullsized":"2023-10-06T22:13:05.720404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:38.973781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.728368+0000","last_change":"2023-10-06T22:12:43.696265+0000","last_active":"2023-10-06T22:13:05.728368+0000","last_peered":"2023-10-06T22:13:05.728368+0000","last_clean":"2023-10-06T22:13:05.728368+0000","last_became_active":"2023-10-06T22:12:43.695939+0000","last_became_peered":"2023-10-06T22:12:43.695939+0000","last_unstale":"2023-10-06T22:13:05.728368+0000","last_undegraded":"2023-10-06T22:13:05.728368+0000","last_fullsized":"2023-10-06T22:13:05.728368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:32.882083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.812700+0000","last_change":"2023-10-06T22:13:06.812700+0000","last_active":"2023-10-06T22:13:06.812700+0000","last_peered":"2023-10-06T22:13:06.812700+0000","last_clean":"2023-10-06T22:13:06.812700+0000","last_became_active":"2023-10-06T22:13:06.812313+0000","last_became_peered":"2023-10-06T22:13:06.812313+0000","last_unstale":"2023-10-06T22:13:06.812700+0000","last_undegraded":"2023-10-06T22:13:06.812700+0000","last_fullsized":"2023-10-06T22:13:06.812700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:25.010233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422371+0000","last_change":"2023-10-06T22:12:43.694717+0000","last_active":"2023-10-06T22:13:06.422371+0000","last_peered":"2023-10-06T22:13:06.422371+0000","last_clean":"2023-10-06T22:13:06.422371+0000","last_became_active":"2023-10-06T22:12:43.694321+0000","last_became_peered":"2023-10-06T22:12:43.694321+0000","last_unstale":"2023-10-06T22:13:06.422371+0000","last_undegraded":"2023-10-06T22:13:06.422371+0000","last_fullsized":"2023-10-06T22:13:06.422371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:06.334726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744274+0000","last_change":"2023-10-06T22:12:43.703770+0000","last_active":"2023-10-06T22:13:05.744274+0000","last_peered":"2023-10-06T22:13:05.744274+0000","last_clean":"2023-10-06T22:13:05.744274+0000","last_became_active":"2023-10-06T22:12:43.703434+0000","last_became_peered":"2023-10-06T22:12:43.703434+0000","last_unstale":"2023-10-06T22:13:05.744274+0000","last_undegraded":"2023-10-06T22:13:05.744274+0000","last_fullsized":"2023-10-06T22:13:05.744274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:25.894280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744061+0000","last_change":"2023-10-06T22:12:43.700994+0000","last_active":"2023-10-06T22:13:05.744061+0000","last_peered":"2023-10-06T22:13:05.744061+0000","last_clean":"2023-10-06T22:13:05.744061+0000","last_became_active":"2023-10-06T22:12:43.700652+0000","last_became_peered":"2023-10-06T22:12:43.700652+0000","last_unstale":"2023-10-06T22:13:05.744061+0000","last_undegraded":"2023-10-06T22:13:05.744061+0000","last_fullsized":"2023-10-06T22:13:05.744061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:54.855597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738703+0000","last_change":"2023-10-06T22:12:43.702099+0000","last_active":"2023-10-06T22:13:05.738703+0000","last_peered":"2023-10-06T22:13:05.738703+0000","last_clean":"2023-10-06T22:13:05.738703+0000","last_became_active":"2023-10-06T22:12:43.701802+0000","last_became_peered":"2023-10-06T22:12:43.701802+0000","last_unstale":"2023-10-06T22:13:05.738703+0000","last_undegraded":"2023-10-06T22:13:05.738703+0000","last_fullsized":"2023-10-06T22:13:05.738703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:00.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720703+0000","last_change":"2023-10-06T22:12:43.704679+0000","last_active":"2023-10-06T22:13:05.720703+0000","last_peered":"2023-10-06T22:13:05.720703+0000","last_clean":"2023-10-06T22:13:05.720703+0000","last_became_active":"2023-10-06T22:12:43.704186+0000","last_became_peered":"2023-10-06T22:12:43.704186+0000","last_unstale":"2023-10-06T22:13:05.720703+0000","last_undegraded":"2023-10-06T22:13:05.720703+0000","last_fullsized":"2023-10-06T22:13:05.720703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:28.665217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.735809+0000","last_change":"2023-10-06T22:12:43.695699+0000","last_active":"2023-10-06T22:13:05.735809+0000","last_peered":"2023-10-06T22:13:05.735809+0000","last_clean":"2023-10-06T22:13:05.735809+0000","last_became_active":"2023-10-06T22:12:43.695359+0000","last_became_peered":"2023-10-06T22:12:43.695359+0000","last_unstale":"2023-10-06T22:13:05.735809+0000","last_undegraded":"2023-10-06T22:13:05.735809+0000","last_fullsized":"2023-10-06T22:13:05.735809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:46.296768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727882+0000","last_change":"2023-10-06T22:12:43.707170+0000","last_active":"2023-10-06T22:13:05.727882+0000","last_peered":"2023-10-06T22:13:05.727882+0000","last_clean":"2023-10-06T22:13:05.727882+0000","last_became_active":"2023-10-06T22:12:43.706875+0000","last_became_peered":"2023-10-06T22:12:43.706875+0000","last_unstale":"2023-10-06T22:13:05.727882+0000","last_undegraded":"2023-10-06T22:13:05.727882+0000","last_fullsized":"2023-10-06T22:13:05.727882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:46.548458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":97,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:34.765129+0000","last_change":"2023-10-06T22:13:06.736382+0000","last_active":"2023-10-06T22:13:34.765129+0000","last_peered":"2023-10-06T22:13:34.765129+0000","last_clean":"2023-10-06T22:13:34.765129+0000","last_became_active":"2023-10-06T22:13:06.725703+0000","last_became_peered":"2023-10-06T22:13:06.725703+0000","last_unstale":"2023-10-06T22:13:34.765129+0000","last_undegraded":"2023-10-06T22:13:34.765129+0000","last_fullsized":"2023-10-06T22:13:34.765129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:10.056384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752688+0000","last_change":"2023-10-06T22:12:43.702574+0000","last_active":"2023-10-06T22:13:05.752688+0000","last_peered":"2023-10-06T22:13:05.752688+0000","last_clean":"2023-10-06T22:13:05.752688+0000","last_became_active":"2023-10-06T22:12:43.702238+0000","last_became_peered":"2023-10-06T22:12:43.702238+0000","last_unstale":"2023-10-06T22:13:05.752688+0000","last_undegraded":"2023-10-06T22:13:05.752688+0000","last_fullsized":"2023-10-06T22:13:05.752688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:34:09.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752842+0000","last_change":"2023-10-06T22:12:42.697256+0000","last_active":"2023-10-06T22:13:05.752842+0000","last_peered":"2023-10-06T22:13:05.752842+0000","last_clean":"2023-10-06T22:13:05.752842+0000","last_became_active":"2023-10-06T22:12:42.696724+0000","last_became_peered":"2023-10-06T22:12:42.696724+0000","last_unstale":"2023-10-06T22:13:05.752842+0000","last_undegraded":"2023-10-06T22:13:05.752842+0000","last_fullsized":"2023-10-06T22:13:05.752842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:50:01.378292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745682+0000","last_change":"2023-10-06T22:12:43.702050+0000","last_active":"2023-10-06T22:13:05.745682+0000","last_peered":"2023-10-06T22:13:05.745682+0000","last_clean":"2023-10-06T22:13:05.745682+0000","last_became_active":"2023-10-06T22:12:43.701432+0000","last_became_peered":"2023-10-06T22:12:43.701432+0000","last_unstale":"2023-10-06T22:13:05.745682+0000","last_undegraded":"2023-10-06T22:13:05.745682+0000","last_fullsized":"2023-10-06T22:13:05.745682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:30:37.909775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.948160+0000","last_change":"2023-10-06T22:13:06.948160+0000","last_active":"2023-10-06T22:13:06.948160+0000","last_peered":"2023-10-06T22:13:06.948160+0000","last_clean":"2023-10-06T22:13:06.948160+0000","last_became_active":"2023-10-06T22:13:06.927995+0000","last_became_peered":"2023-10-06T22:13:06.927995+0000","last_unstale":"2023-10-06T22:13:06.948160+0000","last_undegraded":"2023-10-06T22:13:06.948160+0000","last_fullsized":"2023-10-06T22:13:06.948160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:33:27.974776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738816+0000","last_change":"2023-10-06T22:12:43.696511+0000","last_active":"2023-10-06T22:13:05.738816+0000","last_peered":"2023-10-06T22:13:05.738816+0000","last_clean":"2023-10-06T22:13:05.738816+0000","last_became_active":"2023-10-06T22:12:43.696087+0000","last_became_peered":"2023-10-06T22:12:43.696087+0000","last_unstale":"2023-10-06T22:13:05.738816+0000","last_undegraded":"2023-10-06T22:13:05.738816+0000","last_fullsized":"2023-10-06T22:13:05.738816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:12:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724948+0000","last_change":"2023-10-06T22:12:42.694963+0000","last_active":"2023-10-06T22:13:05.724948+0000","last_peered":"2023-10-06T22:13:05.724948+0000","last_clean":"2023-10-06T22:13:05.724948+0000","last_became_active":"2023-10-06T22:12:42.694517+0000","last_became_peered":"2023-10-06T22:12:42.694517+0000","last_unstale":"2023-10-06T22:13:05.724948+0000","last_undegraded":"2023-10-06T22:13:05.724948+0000","last_fullsized":"2023-10-06T22:13:05.724948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:07:33.404328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723715+0000","last_change":"2023-10-06T22:12:43.698077+0000","last_active":"2023-10-06T22:13:05.723715+0000","last_peered":"2023-10-06T22:13:05.723715+0000","last_clean":"2023-10-06T22:13:05.723715+0000","last_became_active":"2023-10-06T22:12:43.697676+0000","last_became_peered":"2023-10-06T22:12:43.697676+0000","last_unstale":"2023-10-06T22:13:05.723715+0000","last_undegraded":"2023-10-06T22:13:05.723715+0000","last_fullsized":"2023-10-06T22:13:05.723715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:59:56.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:33.731417+0000","last_change":"2023-10-06T22:12:42.698067+0000","last_active":"2023-10-06T22:13:33.731417+0000","last_peered":"2023-10-06T22:13:33.731417+0000","last_clean":"2023-10-06T22:13:33.731417+0000","last_became_active":"2023-10-06T22:12:42.697671+0000","last_became_peered":"2023-10-06T22:12:42.697671+0000","last_unstale":"2023-10-06T22:13:33.731417+0000","last_undegraded":"2023-10-06T22:13:33.731417+0000","last_fullsized":"2023-10-06T22:13:33.731417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:33.607513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724160+0000","last_change":"2023-10-06T22:12:43.696213+0000","last_active":"2023-10-06T22:13:05.724160+0000","last_peered":"2023-10-06T22:13:05.724160+0000","last_clean":"2023-10-06T22:13:05.724160+0000","last_became_active":"2023-10-06T22:12:43.695925+0000","last_became_peered":"2023-10-06T22:12:43.695925+0000","last_unstale":"2023-10-06T22:13:05.724160+0000","last_undegraded":"2023-10-06T22:13:05.724160+0000","last_fullsized":"2023-10-06T22:13:05.724160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:24.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.719695+0000","last_change":"2023-10-06T22:12:42.687188+0000","last_active":"2023-10-06T22:13:05.719695+0000","last_peered":"2023-10-06T22:13:05.719695+0000","last_clean":"2023-10-06T22:13:05.719695+0000","last_became_active":"2023-10-06T22:12:42.686747+0000","last_became_peered":"2023-10-06T22:12:42.686747+0000","last_unstale":"2023-10-06T22:13:05.719695+0000","last_undegraded":"2023-10-06T22:13:05.719695+0000","last_fullsized":"2023-10-06T22:13:05.719695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:21.326911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723338+0000","last_change":"2023-10-06T22:12:43.697988+0000","last_active":"2023-10-06T22:13:05.723338+0000","last_peered":"2023-10-06T22:13:05.723338+0000","last_clean":"2023-10-06T22:13:05.723338+0000","last_became_active":"2023-10-06T22:12:43.697641+0000","last_became_peered":"2023-10-06T22:12:43.697641+0000","last_unstale":"2023-10-06T22:13:05.723338+0000","last_undegraded":"2023-10-06T22:13:05.723338+0000","last_fullsized":"2023-10-06T22:13:05.723338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:33.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.423261+0000","last_change":"2023-10-06T22:12:42.688882+0000","last_active":"2023-10-06T22:13:06.423261+0000","last_peered":"2023-10-06T22:13:06.423261+0000","last_clean":"2023-10-06T22:13:06.423261+0000","last_became_active":"2023-10-06T22:12:42.687653+0000","last_became_peered":"2023-10-06T22:12:42.687653+0000","last_unstale":"2023-10-06T22:13:06.423261+0000","last_undegraded":"2023-10-06T22:13:06.423261+0000","last_fullsized":"2023-10-06T22:13:06.423261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:18.146081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.753938+0000","last_change":"2023-10-06T22:12:43.696559+0000","last_active":"2023-10-06T22:13:05.753938+0000","last_peered":"2023-10-06T22:13:05.753938+0000","last_clean":"2023-10-06T22:13:05.753938+0000","last_became_active":"2023-10-06T22:12:43.696252+0000","last_became_peered":"2023-10-06T22:12:43.696252+0000","last_unstale":"2023-10-06T22:13:05.753938+0000","last_undegraded":"2023-10-06T22:13:05.753938+0000","last_fullsized":"2023-10-06T22:13:05.753938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:31:36.097000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722785+0000","last_change":"2023-10-06T22:12:42.694299+0000","last_active":"2023-10-06T22:13:05.722785+0000","last_peered":"2023-10-06T22:13:05.722785+0000","last_clean":"2023-10-06T22:13:05.722785+0000","last_became_active":"2023-10-06T22:12:42.693820+0000","last_became_peered":"2023-10-06T22:12:42.693820+0000","last_unstale":"2023-10-06T22:13:05.722785+0000","last_undegraded":"2023-10-06T22:13:05.722785+0000","last_fullsized":"2023-10-06T22:13:05.722785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:13.837379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.734798+0000","last_change":"2023-10-06T22:12:43.695234+0000","last_active":"2023-10-06T22:13:05.734798+0000","last_peered":"2023-10-06T22:13:05.734798+0000","last_clean":"2023-10-06T22:13:05.734798+0000","last_became_active":"2023-10-06T22:12:43.694889+0000","last_became_peered":"2023-10-06T22:12:43.694889+0000","last_unstale":"2023-10-06T22:13:05.734798+0000","last_undegraded":"2023-10-06T22:13:05.734798+0000","last_fullsized":"2023-10-06T22:13:05.734798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:51.039264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":115,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:37.736251+0000","last_change":"2023-10-06T22:12:42.695438+0000","last_active":"2023-10-06T22:13:37.736251+0000","last_peered":"2023-10-06T22:13:37.736251+0000","last_clean":"2023-10-06T22:13:37.736251+0000","last_became_active":"2023-10-06T22:12:42.695090+0000","last_became_peered":"2023-10-06T22:12:42.695090+0000","last_unstale":"2023-10-06T22:13:37.736251+0000","last_undegraded":"2023-10-06T22:13:37.736251+0000","last_fullsized":"2023-10-06T22:13:37.736251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:25:34.385961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.739124+0000","last_change":"2023-10-06T22:12:43.697459+0000","last_active":"2023-10-06T22:13:05.739124+0000","last_peered":"2023-10-06T22:13:05.739124+0000","last_clean":"2023-10-06T22:13:05.739124+0000","last_became_active":"2023-10-06T22:12:43.696405+0000","last_became_peered":"2023-10-06T22:12:43.696405+0000","last_unstale":"2023-10-06T22:13:05.739124+0000","last_undegraded":"2023-10-06T22:13:05.739124+0000","last_fullsized":"2023-10-06T22:13:05.739124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:30.928083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727798+0000","last_change":"2023-10-06T22:12:42.692151+0000","last_active":"2023-10-06T22:13:05.727798+0000","last_peered":"2023-10-06T22:13:05.727798+0000","last_clean":"2023-10-06T22:13:05.727798+0000","last_became_active":"2023-10-06T22:12:42.691780+0000","last_became_peered":"2023-10-06T22:12:42.691780+0000","last_unstale":"2023-10-06T22:13:05.727798+0000","last_undegraded":"2023-10-06T22:13:05.727798+0000","last_fullsized":"2023-10-06T22:13:05.727798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:49:35.697733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745577+0000","last_change":"2023-10-06T22:12:43.701754+0000","last_active":"2023-10-06T22:13:05.745577+0000","last_peered":"2023-10-06T22:13:05.745577+0000","last_clean":"2023-10-06T22:13:05.745577+0000","last_became_active":"2023-10-06T22:12:43.700881+0000","last_became_peered":"2023-10-06T22:12:43.700881+0000","last_unstale":"2023-10-06T22:13:05.745577+0000","last_undegraded":"2023-10-06T22:13:05.745577+0000","last_fullsized":"2023-10-06T22:13:05.745577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:35:21.988307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723824+0000","last_change":"2023-10-06T22:12:42.699751+0000","last_active":"2023-10-06T22:13:05.723824+0000","last_peered":"2023-10-06T22:13:05.723824+0000","last_clean":"2023-10-06T22:13:05.723824+0000","last_became_active":"2023-10-06T22:12:42.699321+0000","last_became_peered":"2023-10-06T22:12:42.699321+0000","last_unstale":"2023-10-06T22:13:05.723824+0000","last_undegraded":"2023-10-06T22:13:05.723824+0000","last_fullsized":"2023-10-06T22:13:05.723824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:35:25.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738929+0000","last_change":"2023-10-06T22:12:43.702426+0000","last_active":"2023-10-06T22:13:05.738929+0000","last_peered":"2023-10-06T22:13:05.738929+0000","last_clean":"2023-10-06T22:13:05.738929+0000","last_became_active":"2023-10-06T22:12:43.702101+0000","last_became_peered":"2023-10-06T22:12:43.702101+0000","last_unstale":"2023-10-06T22:13:05.738929+0000","last_undegraded":"2023-10-06T22:13:05.738929+0000","last_fullsized":"2023-10-06T22:13:05.738929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:31:09.133941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:37.736193+0000","last_change":"2023-10-06T22:12:42.688815+0000","last_active":"2023-10-06T22:13:37.736193+0000","last_peered":"2023-10-06T22:13:37.736193+0000","last_clean":"2023-10-06T22:13:37.736193+0000","last_became_active":"2023-10-06T22:12:42.688447+0000","last_became_peered":"2023-10-06T22:12:42.688447+0000","last_unstale":"2023-10-06T22:13:37.736193+0000","last_undegraded":"2023-10-06T22:13:37.736193+0000","last_fullsized":"2023-10-06T22:13:37.736193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:26:25.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57508,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":107,"num_write":30,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724127+0000","last_change":"2023-10-06T22:12:43.698406+0000","last_active":"2023-10-06T22:13:05.724127+0000","last_peered":"2023-10-06T22:13:05.724127+0000","last_clean":"2023-10-06T22:13:05.724127+0000","last_became_active":"2023-10-06T22:12:43.697204+0000","last_became_peered":"2023-10-06T22:12:43.697204+0000","last_unstale":"2023-10-06T22:13:05.724127+0000","last_undegraded":"2023-10-06T22:13:05.724127+0000","last_fullsized":"2023-10-06T22:13:05.724127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:59.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727727+0000","last_change":"2023-10-06T22:12:42.691007+0000","last_active":"2023-10-06T22:13:05.727727+0000","last_peered":"2023-10-06T22:13:05.727727+0000","last_clean":"2023-10-06T22:13:05.727727+0000","last_became_active":"2023-10-06T22:12:42.690693+0000","last_became_peered":"2023-10-06T22:12:42.690693+0000","last_unstale":"2023-10-06T22:13:05.727727+0000","last_undegraded":"2023-10-06T22:13:05.727727+0000","last_fullsized":"2023-10-06T22:13:05.727727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:47.670091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.421929+0000","last_change":"2023-10-06T22:12:43.705131+0000","last_active":"2023-10-06T22:13:06.421929+0000","last_peered":"2023-10-06T22:13:06.421929+0000","last_clean":"2023-10-06T22:13:06.421929+0000","last_became_active":"2023-10-06T22:12:43.704469+0000","last_became_peered":"2023-10-06T22:12:43.704469+0000","last_unstale":"2023-10-06T22:13:06.421929+0000","last_undegraded":"2023-10-06T22:13:06.421929+0000","last_fullsized":"2023-10-06T22:13:06.421929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:36:39.392274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723093+0000","last_change":"2023-10-06T22:12:42.698392+0000","last_active":"2023-10-06T22:13:05.723093+0000","last_peered":"2023-10-06T22:13:05.723093+0000","last_clean":"2023-10-06T22:13:05.723093+0000","last_became_active":"2023-10-06T22:12:42.698019+0000","last_became_peered":"2023-10-06T22:12:42.698019+0000","last_unstale":"2023-10-06T22:13:05.723093+0000","last_undegraded":"2023-10-06T22:13:05.723093+0000","last_fullsized":"2023-10-06T22:13:05.723093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:00.345300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727760+0000","last_change":"2023-10-06T22:12:43.693238+0000","last_active":"2023-10-06T22:13:05.727760+0000","last_peered":"2023-10-06T22:13:05.727760+0000","last_clean":"2023-10-06T22:13:05.727760+0000","last_became_active":"2023-10-06T22:12:43.692948+0000","last_became_peered":"2023-10-06T22:12:43.692948+0000","last_unstale":"2023-10-06T22:13:05.727760+0000","last_undegraded":"2023-10-06T22:13:05.727760+0000","last_fullsized":"2023-10-06T22:13:05.727760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:24:30.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:36.740565+0000","last_change":"2023-10-06T22:12:42.693213+0000","last_active":"2023-10-06T22:13:36.740565+0000","last_peered":"2023-10-06T22:13:36.740565+0000","last_clean":"2023-10-06T22:13:36.740565+0000","last_became_active":"2023-10-06T22:12:42.692894+0000","last_became_peered":"2023-10-06T22:12:42.692894+0000","last_unstale":"2023-10-06T22:13:36.740565+0000","last_undegraded":"2023-10-06T22:13:36.740565+0000","last_fullsized":"2023-10-06T22:13:36.740565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:40:21.159102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727338+0000","last_change":"2023-10-06T22:12:43.695527+0000","last_active":"2023-10-06T22:13:05.727338+0000","last_peered":"2023-10-06T22:13:05.727338+0000","last_clean":"2023-10-06T22:13:05.727338+0000","last_became_active":"2023-10-06T22:12:43.694364+0000","last_became_peered":"2023-10-06T22:12:43.694364+0000","last_unstale":"2023-10-06T22:13:05.727338+0000","last_undegraded":"2023-10-06T22:13:05.727338+0000","last_fullsized":"2023-10-06T22:13:05.727338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723938+0000","last_change":"2023-10-06T22:12:42.697854+0000","last_active":"2023-10-06T22:13:05.723938+0000","last_peered":"2023-10-06T22:13:05.723938+0000","last_clean":"2023-10-06T22:13:05.723938+0000","last_became_active":"2023-10-06T22:12:42.697542+0000","last_became_peered":"2023-10-06T22:12:42.697542+0000","last_unstale":"2023-10-06T22:13:05.723938+0000","last_undegraded":"2023-10-06T22:13:05.723938+0000","last_fullsized":"2023-10-06T22:13:05.723938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:12:11.181398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722884+0000","last_change":"2023-10-06T22:12:43.698070+0000","last_active":"2023-10-06T22:13:05.722884+0000","last_peered":"2023-10-06T22:13:05.722884+0000","last_clean":"2023-10-06T22:13:05.722884+0000","last_became_active":"2023-10-06T22:12:43.697682+0000","last_became_peered":"2023-10-06T22:12:43.697682+0000","last_unstale":"2023-10-06T22:13:05.722884+0000","last_undegraded":"2023-10-06T22:13:05.722884+0000","last_fullsized":"2023-10-06T22:13:05.722884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:43.808866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724573+0000","last_change":"2023-10-06T22:12:42.691691+0000","last_active":"2023-10-06T22:13:05.724573+0000","last_peered":"2023-10-06T22:13:05.724573+0000","last_clean":"2023-10-06T22:13:05.724573+0000","last_became_active":"2023-10-06T22:12:42.691001+0000","last_became_peered":"2023-10-06T22:12:42.691001+0000","last_unstale":"2023-10-06T22:13:05.724573+0000","last_undegraded":"2023-10-06T22:13:05.724573+0000","last_fullsized":"2023-10-06T22:13:05.724573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:11:18.559570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.927620+0000","last_change":"2023-10-06T22:13:06.927620+0000","last_active":"2023-10-06T22:13:06.927620+0000","last_peered":"2023-10-06T22:13:06.927620+0000","last_clean":"2023-10-06T22:13:06.927620+0000","last_became_active":"2023-10-06T22:13:06.927206+0000","last_became_peered":"2023-10-06T22:13:06.927206+0000","last_unstale":"2023-10-06T22:13:06.927620+0000","last_undegraded":"2023-10-06T22:13:06.927620+0000","last_fullsized":"2023-10-06T22:13:06.927620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:50:30.780174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.726783+0000","last_change":"2023-10-06T22:13:06.726783+0000","last_active":"2023-10-06T22:13:06.726783+0000","last_peered":"2023-10-06T22:13:06.726783+0000","last_clean":"2023-10-06T22:13:06.726783+0000","last_became_active":"2023-10-06T22:13:06.726342+0000","last_became_peered":"2023-10-06T22:13:06.726342+0000","last_unstale":"2023-10-06T22:13:06.726783+0000","last_undegraded":"2023-10-06T22:13:06.726783+0000","last_fullsized":"2023-10-06T22:13:06.726783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:42.359819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720194+0000","last_change":"2023-10-06T22:12:43.702870+0000","last_active":"2023-10-06T22:13:05.720194+0000","last_peered":"2023-10-06T22:13:05.720194+0000","last_clean":"2023-10-06T22:13:05.720194+0000","last_became_active":"2023-10-06T22:12:43.702564+0000","last_became_peered":"2023-10-06T22:12:43.702564+0000","last_unstale":"2023-10-06T22:13:05.720194+0000","last_undegraded":"2023-10-06T22:13:05.720194+0000","last_fullsized":"2023-10-06T22:13:05.720194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:23:32.238268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:37.770582+0000","last_change":"2023-10-06T22:12:42.696603+0000","last_active":"2023-10-06T22:13:37.770582+0000","last_peered":"2023-10-06T22:13:37.770582+0000","last_clean":"2023-10-06T22:13:37.770582+0000","last_became_active":"2023-10-06T22:12:42.696313+0000","last_became_peered":"2023-10-06T22:12:42.696313+0000","last_unstale":"2023-10-06T22:13:37.770582+0000","last_undegraded":"2023-10-06T22:13:37.770582+0000","last_fullsized":"2023-10-06T22:13:37.770582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:15.719749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720013+0000","last_change":"2023-10-06T22:12:43.703356+0000","last_active":"2023-10-06T22:13:05.720013+0000","last_peered":"2023-10-06T22:13:05.720013+0000","last_clean":"2023-10-06T22:13:05.720013+0000","last_became_active":"2023-10-06T22:12:43.702670+0000","last_became_peered":"2023-10-06T22:12:43.702670+0000","last_unstale":"2023-10-06T22:13:05.720013+0000","last_undegraded":"2023-10-06T22:13:05.720013+0000","last_fullsized":"2023-10-06T22:13:05.720013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:15:05.681491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722900+0000","last_change":"2023-10-06T22:12:43.699083+0000","last_active":"2023-10-06T22:13:05.722900+0000","last_peered":"2023-10-06T22:13:05.722900+0000","last_clean":"2023-10-06T22:13:05.722900+0000","last_became_active":"2023-10-06T22:12:43.698826+0000","last_became_peered":"2023-10-06T22:12:43.698826+0000","last_unstale":"2023-10-06T22:13:05.722900+0000","last_undegraded":"2023-10-06T22:13:05.722900+0000","last_fullsized":"2023-10-06T22:13:05.722900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:04.225060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:35.738547+0000","last_change":"2023-10-06T22:12:42.699873+0000","last_active":"2023-10-06T22:13:35.738547+0000","last_peered":"2023-10-06T22:13:35.738547+0000","last_clean":"2023-10-06T22:13:35.738547+0000","last_became_active":"2023-10-06T22:12:42.699476+0000","last_became_peered":"2023-10-06T22:12:42.699476+0000","last_unstale":"2023-10-06T22:13:35.738547+0000","last_undegraded":"2023-10-06T22:13:35.738547+0000","last_fullsized":"2023-10-06T22:13:35.738547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:13:27.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751789+0000","last_change":"2023-10-06T22:12:43.697720+0000","last_active":"2023-10-06T22:13:05.751789+0000","last_peered":"2023-10-06T22:13:05.751789+0000","last_clean":"2023-10-06T22:13:05.751789+0000","last_became_active":"2023-10-06T22:12:43.697430+0000","last_became_peered":"2023-10-06T22:12:43.697430+0000","last_unstale":"2023-10-06T22:13:05.751789+0000","last_undegraded":"2023-10-06T22:13:05.751789+0000","last_fullsized":"2023-10-06T22:13:05.751789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:29.633012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722760+0000","last_change":"2023-10-06T22:12:42.690781+0000","last_active":"2023-10-06T22:13:05.722760+0000","last_peered":"2023-10-06T22:13:05.722760+0000","last_clean":"2023-10-06T22:13:05.722760+0000","last_became_active":"2023-10-06T22:12:42.690274+0000","last_became_peered":"2023-10-06T22:12:42.690274+0000","last_unstale":"2023-10-06T22:13:05.722760+0000","last_undegraded":"2023-10-06T22:13:05.722760+0000","last_fullsized":"2023-10-06T22:13:05.722760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:26.653182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727428+0000","last_change":"2023-10-06T22:11:36.409669+0000","last_active":"2023-10-06T22:13:05.727428+0000","last_peered":"2023-10-06T22:13:05.727428+0000","last_clean":"2023-10-06T22:13:05.727428+0000","last_became_active":"2023-10-06T22:11:36.409197+0000","last_became_peered":"2023-10-06T22:11:36.409197+0000","last_unstale":"2023-10-06T22:13:05.727428+0000","last_undegraded":"2023-10-06T22:13:05.727428+0000","last_fullsized":"2023-10-06T22:13:05.727428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_clean_scrub_stamp":"2023-10-06T22:08:24.598743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:50:55.982571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.423067+0000","last_change":"2023-10-06T22:12:43.698617+0000","last_active":"2023-10-06T22:13:06.423067+0000","last_peered":"2023-10-06T22:13:06.423067+0000","last_clean":"2023-10-06T22:13:06.423067+0000","last_became_active":"2023-10-06T22:12:43.697942+0000","last_became_peered":"2023-10-06T22:12:43.697942+0000","last_unstale":"2023-10-06T22:13:06.423067+0000","last_undegraded":"2023-10-06T22:13:06.423067+0000","last_fullsized":"2023-10-06T22:13:06.423067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:38.075730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727545+0000","last_change":"2023-10-06T22:12:42.695553+0000","last_active":"2023-10-06T22:13:05.727545+0000","last_peered":"2023-10-06T22:13:05.727545+0000","last_clean":"2023-10-06T22:13:05.727545+0000","last_became_active":"2023-10-06T22:12:42.695209+0000","last_became_peered":"2023-10-06T22:12:42.695209+0000","last_unstale":"2023-10-06T22:13:05.727545+0000","last_undegraded":"2023-10-06T22:13:05.727545+0000","last_fullsized":"2023-10-06T22:13:05.727545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:10.387710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724270+0000","last_change":"2023-10-06T22:12:42.699606+0000","last_active":"2023-10-06T22:13:05.724270+0000","last_peered":"2023-10-06T22:13:05.724270+0000","last_clean":"2023-10-06T22:13:05.724270+0000","last_became_active":"2023-10-06T22:12:42.699222+0000","last_became_peered":"2023-10-06T22:12:42.699222+0000","last_unstale":"2023-10-06T22:13:05.724270+0000","last_undegraded":"2023-10-06T22:13:05.724270+0000","last_fullsized":"2023-10-06T22:13:05.724270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:52.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422535+0000","last_change":"2023-10-06T22:12:43.697783+0000","last_active":"2023-10-06T22:13:06.422535+0000","last_peered":"2023-10-06T22:13:06.422535+0000","last_clean":"2023-10-06T22:13:06.422535+0000","last_became_active":"2023-10-06T22:12:43.697388+0000","last_became_peered":"2023-10-06T22:12:43.697388+0000","last_unstale":"2023-10-06T22:13:06.422535+0000","last_undegraded":"2023-10-06T22:13:06.422535+0000","last_fullsized":"2023-10-06T22:13:06.422535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:50.113458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.735903+0000","last_change":"2023-10-06T22:12:42.694180+0000","last_active":"2023-10-06T22:13:05.735903+0000","last_peered":"2023-10-06T22:13:05.735903+0000","last_clean":"2023-10-06T22:13:05.735903+0000","last_became_active":"2023-10-06T22:12:42.693861+0000","last_became_peered":"2023-10-06T22:12:42.693861+0000","last_unstale":"2023-10-06T22:13:05.735903+0000","last_undegraded":"2023-10-06T22:13:05.735903+0000","last_fullsized":"2023-10-06T22:13:05.735903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:28:02.511117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.753037+0000","last_change":"2023-10-06T22:12:43.699118+0000","last_active":"2023-10-06T22:13:05.753037+0000","last_peered":"2023-10-06T22:13:05.753037+0000","last_clean":"2023-10-06T22:13:05.753037+0000","last_became_active":"2023-10-06T22:12:43.698540+0000","last_became_peered":"2023-10-06T22:12:43.698540+0000","last_unstale":"2023-10-06T22:13:05.753037+0000","last_undegraded":"2023-10-06T22:13:05.753037+0000","last_fullsized":"2023-10-06T22:13:05.753037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:31.192762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722961+0000","last_change":"2023-10-06T22:12:42.690617+0000","last_active":"2023-10-06T22:13:05.722961+0000","last_peered":"2023-10-06T22:13:05.722961+0000","last_clean":"2023-10-06T22:13:05.722961+0000","last_became_active":"2023-10-06T22:12:42.690238+0000","last_became_peered":"2023-10-06T22:12:42.690238+0000","last_unstale":"2023-10-06T22:13:05.722961+0000","last_undegraded":"2023-10-06T22:13:05.722961+0000","last_fullsized":"2023-10-06T22:13:05.722961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:09.819570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.745288+0000","last_change":"2023-10-06T22:12:43.699064+0000","last_active":"2023-10-06T22:13:05.745288+0000","last_peered":"2023-10-06T22:13:05.745288+0000","last_clean":"2023-10-06T22:13:05.745288+0000","last_became_active":"2023-10-06T22:12:43.698727+0000","last_became_peered":"2023-10-06T22:12:43.698727+0000","last_unstale":"2023-10-06T22:13:05.745288+0000","last_undegraded":"2023-10-06T22:13:05.745288+0000","last_fullsized":"2023-10-06T22:13:05.745288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:48:14.493376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738711+0000","last_change":"2023-10-06T22:12:42.693779+0000","last_active":"2023-10-06T22:13:05.738711+0000","last_peered":"2023-10-06T22:13:05.738711+0000","last_clean":"2023-10-06T22:13:05.738711+0000","last_became_active":"2023-10-06T22:12:42.693257+0000","last_became_peered":"2023-10-06T22:12:42.693257+0000","last_unstale":"2023-10-06T22:13:05.738711+0000","last_undegraded":"2023-10-06T22:13:05.738711+0000","last_fullsized":"2023-10-06T22:13:05.738711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:56:48.945078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723955+0000","last_change":"2023-10-06T22:12:43.700620+0000","last_active":"2023-10-06T22:13:05.723955+0000","last_peered":"2023-10-06T22:13:05.723955+0000","last_clean":"2023-10-06T22:13:05.723955+0000","last_became_active":"2023-10-06T22:12:43.700093+0000","last_became_peered":"2023-10-06T22:12:43.700093+0000","last_unstale":"2023-10-06T22:13:05.723955+0000","last_undegraded":"2023-10-06T22:13:05.723955+0000","last_fullsized":"2023-10-06T22:13:05.723955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:29.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751425+0000","last_change":"2023-10-06T22:12:42.689640+0000","last_active":"2023-10-06T22:13:05.751425+0000","last_peered":"2023-10-06T22:13:05.751425+0000","last_clean":"2023-10-06T22:13:05.751425+0000","last_became_active":"2023-10-06T22:12:42.688584+0000","last_became_peered":"2023-10-06T22:12:42.688584+0000","last_unstale":"2023-10-06T22:13:05.751425+0000","last_undegraded":"2023-10-06T22:13:05.751425+0000","last_fullsized":"2023-10-06T22:13:05.751425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:43.066396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720226+0000","last_change":"2023-10-06T22:12:43.700084+0000","last_active":"2023-10-06T22:13:05.720226+0000","last_peered":"2023-10-06T22:13:05.720226+0000","last_clean":"2023-10-06T22:13:05.720226+0000","last_became_active":"2023-10-06T22:12:43.699787+0000","last_became_peered":"2023-10-06T22:12:43.699787+0000","last_unstale":"2023-10-06T22:13:05.720226+0000","last_undegraded":"2023-10-06T22:13:05.720226+0000","last_fullsized":"2023-10-06T22:13:05.720226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:07.219800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727627+0000","last_change":"2023-10-06T22:12:42.688750+0000","last_active":"2023-10-06T22:13:05.727627+0000","last_peered":"2023-10-06T22:13:05.727627+0000","last_clean":"2023-10-06T22:13:05.727627+0000","last_became_active":"2023-10-06T22:12:42.688387+0000","last_became_peered":"2023-10-06T22:12:42.688387+0000","last_unstale":"2023-10-06T22:13:05.727627+0000","last_undegraded":"2023-10-06T22:13:05.727627+0000","last_fullsized":"2023-10-06T22:13:05.727627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:32.980631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752151+0000","last_change":"2023-10-06T22:12:43.706741+0000","last_active":"2023-10-06T22:13:05.752151+0000","last_peered":"2023-10-06T22:13:05.752151+0000","last_clean":"2023-10-06T22:13:05.752151+0000","last_became_active":"2023-10-06T22:12:43.706416+0000","last_became_peered":"2023-10-06T22:12:43.706416+0000","last_unstale":"2023-10-06T22:13:05.752151+0000","last_undegraded":"2023-10-06T22:13:05.752151+0000","last_fullsized":"2023-10-06T22:13:05.752151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:55:16.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724994+0000","last_change":"2023-10-06T22:12:42.692850+0000","last_active":"2023-10-06T22:13:05.724994+0000","last_peered":"2023-10-06T22:13:05.724994+0000","last_clean":"2023-10-06T22:13:05.724994+0000","last_became_active":"2023-10-06T22:12:42.692287+0000","last_became_peered":"2023-10-06T22:12:42.692287+0000","last_unstale":"2023-10-06T22:13:05.724994+0000","last_undegraded":"2023-10-06T22:13:05.724994+0000","last_fullsized":"2023-10-06T22:13:05.724994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:37:59.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727726+0000","last_change":"2023-10-06T22:12:43.700667+0000","last_active":"2023-10-06T22:13:05.727726+0000","last_peered":"2023-10-06T22:13:05.727726+0000","last_clean":"2023-10-06T22:13:05.727726+0000","last_became_active":"2023-10-06T22:12:43.698758+0000","last_became_peered":"2023-10-06T22:12:43.698758+0000","last_unstale":"2023-10-06T22:13:05.727726+0000","last_undegraded":"2023-10-06T22:13:05.727726+0000","last_fullsized":"2023-10-06T22:13:05.727726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:49:53.091578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:37.737023+0000","last_change":"2023-10-06T22:12:42.695964+0000","last_active":"2023-10-06T22:13:37.737023+0000","last_peered":"2023-10-06T22:13:37.737023+0000","last_clean":"2023-10-06T22:13:37.737023+0000","last_became_active":"2023-10-06T22:12:42.695431+0000","last_became_peered":"2023-10-06T22:12:42.695431+0000","last_unstale":"2023-10-06T22:13:37.737023+0000","last_undegraded":"2023-10-06T22:13:37.737023+0000","last_fullsized":"2023-10-06T22:13:37.737023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:21:42.761421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.728382+0000","last_change":"2023-10-06T22:12:43.698088+0000","last_active":"2023-10-06T22:13:05.728382+0000","last_peered":"2023-10-06T22:13:05.728382+0000","last_clean":"2023-10-06T22:13:05.728382+0000","last_became_active":"2023-10-06T22:12:43.697765+0000","last_became_peered":"2023-10-06T22:12:43.697765+0000","last_unstale":"2023-10-06T22:13:05.728382+0000","last_undegraded":"2023-10-06T22:13:05.728382+0000","last_fullsized":"2023-10-06T22:13:05.728382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:26:22.958348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723709+0000","last_change":"2023-10-06T22:12:42.698586+0000","last_active":"2023-10-06T22:13:05.723709+0000","last_peered":"2023-10-06T22:13:05.723709+0000","last_clean":"2023-10-06T22:13:05.723709+0000","last_became_active":"2023-10-06T22:12:42.698198+0000","last_became_peered":"2023-10-06T22:12:42.698198+0000","last_unstale":"2023-10-06T22:13:05.723709+0000","last_undegraded":"2023-10-06T22:13:05.723709+0000","last_fullsized":"2023-10-06T22:13:05.723709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:03:43.898987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727662+0000","last_change":"2023-10-06T22:12:43.698615+0000","last_active":"2023-10-06T22:13:05.727662+0000","last_peered":"2023-10-06T22:13:05.727662+0000","last_clean":"2023-10-06T22:13:05.727662+0000","last_became_active":"2023-10-06T22:12:43.698302+0000","last_became_peered":"2023-10-06T22:12:43.698302+0000","last_unstale":"2023-10-06T22:13:05.727662+0000","last_undegraded":"2023-10-06T22:13:05.727662+0000","last_fullsized":"2023-10-06T22:13:05.727662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:35.973119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422877+0000","last_change":"2023-10-06T22:12:42.700325+0000","last_active":"2023-10-06T22:13:06.422877+0000","last_peered":"2023-10-06T22:13:06.422877+0000","last_clean":"2023-10-06T22:13:06.422877+0000","last_became_active":"2023-10-06T22:12:42.699426+0000","last_became_peered":"2023-10-06T22:12:42.699426+0000","last_unstale":"2023-10-06T22:13:06.422877+0000","last_undegraded":"2023-10-06T22:13:06.422877+0000","last_fullsized":"2023-10-06T22:13:06.422877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:09.649737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727916+0000","last_change":"2023-10-06T22:12:43.698620+0000","last_active":"2023-10-06T22:13:05.727916+0000","last_peered":"2023-10-06T22:13:05.727916+0000","last_clean":"2023-10-06T22:13:05.727916+0000","last_became_active":"2023-10-06T22:12:43.698335+0000","last_became_peered":"2023-10-06T22:12:43.698335+0000","last_unstale":"2023-10-06T22:13:05.727916+0000","last_undegraded":"2023-10-06T22:13:05.727916+0000","last_fullsized":"2023-10-06T22:13:05.727916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:49.222848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720263+0000","last_change":"2023-10-06T22:12:43.700477+0000","last_active":"2023-10-06T22:13:05.720263+0000","last_peered":"2023-10-06T22:13:05.720263+0000","last_clean":"2023-10-06T22:13:05.720263+0000","last_became_active":"2023-10-06T22:12:43.700177+0000","last_became_peered":"2023-10-06T22:12:43.700177+0000","last_unstale":"2023-10-06T22:13:05.720263+0000","last_undegraded":"2023-10-06T22:13:05.720263+0000","last_fullsized":"2023-10-06T22:13:05.720263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:11.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727515+0000","last_change":"2023-10-06T22:12:43.701792+0000","last_active":"2023-10-06T22:13:05.727515+0000","last_peered":"2023-10-06T22:13:05.727515+0000","last_clean":"2023-10-06T22:13:05.727515+0000","last_became_active":"2023-10-06T22:12:43.701316+0000","last_became_peered":"2023-10-06T22:12:43.701316+0000","last_unstale":"2023-10-06T22:13:05.727515+0000","last_undegraded":"2023-10-06T22:13:05.727515+0000","last_fullsized":"2023-10-06T22:13:05.727515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:03.085219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727616+0000","last_change":"2023-10-06T22:12:43.707871+0000","last_active":"2023-10-06T22:13:05.727616+0000","last_peered":"2023-10-06T22:13:05.727616+0000","last_clean":"2023-10-06T22:13:05.727616+0000","last_became_active":"2023-10-06T22:12:43.707659+0000","last_became_peered":"2023-10-06T22:12:43.707659+0000","last_unstale":"2023-10-06T22:13:05.727616+0000","last_undegraded":"2023-10-06T22:13:05.727616+0000","last_fullsized":"2023-10-06T22:13:05.727616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:43.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.744632+0000","last_change":"2023-10-06T22:12:43.697701+0000","last_active":"2023-10-06T22:13:05.744632+0000","last_peered":"2023-10-06T22:13:05.744632+0000","last_clean":"2023-10-06T22:13:05.744632+0000","last_became_active":"2023-10-06T22:12:43.697361+0000","last_became_peered":"2023-10-06T22:12:43.697361+0000","last_unstale":"2023-10-06T22:13:05.744632+0000","last_undegraded":"2023-10-06T22:13:05.744632+0000","last_fullsized":"2023-10-06T22:13:05.744632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:14:53.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422617+0000","last_change":"2023-10-06T22:12:43.700058+0000","last_active":"2023-10-06T22:13:06.422617+0000","last_peered":"2023-10-06T22:13:06.422617+0000","last_clean":"2023-10-06T22:13:06.422617+0000","last_became_active":"2023-10-06T22:12:43.697893+0000","last_became_peered":"2023-10-06T22:12:43.697893+0000","last_unstale":"2023-10-06T22:13:06.422617+0000","last_undegraded":"2023-10-06T22:13:06.422617+0000","last_fullsized":"2023-10-06T22:13:06.422617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:01:37.518096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.729307+0000","last_change":"2023-10-06T22:13:06.729307+0000","last_active":"2023-10-06T22:13:06.729307+0000","last_peered":"2023-10-06T22:13:06.729307+0000","last_clean":"2023-10-06T22:13:06.729307+0000","last_became_active":"2023-10-06T22:13:06.728877+0000","last_became_peered":"2023-10-06T22:13:06.728877+0000","last_unstale":"2023-10-06T22:13:06.729307+0000","last_undegraded":"2023-10-06T22:13:06.729307+0000","last_fullsized":"2023-10-06T22:13:06.729307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:17.629046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.728370+0000","last_change":"2023-10-06T22:12:43.700668+0000","last_active":"2023-10-06T22:13:05.728370+0000","last_peered":"2023-10-06T22:13:05.728370+0000","last_clean":"2023-10-06T22:13:05.728370+0000","last_became_active":"2023-10-06T22:12:43.700387+0000","last_became_peered":"2023-10-06T22:12:43.700387+0000","last_unstale":"2023-10-06T22:13:05.728370+0000","last_undegraded":"2023-10-06T22:13:05.728370+0000","last_fullsized":"2023-10-06T22:13:05.728370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:36:00.053496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752951+0000","last_change":"2023-10-06T22:12:43.702658+0000","last_active":"2023-10-06T22:13:05.752951+0000","last_peered":"2023-10-06T22:13:05.752951+0000","last_clean":"2023-10-06T22:13:05.752951+0000","last_became_active":"2023-10-06T22:12:43.702264+0000","last_became_peered":"2023-10-06T22:12:43.702264+0000","last_unstale":"2023-10-06T22:13:05.752951+0000","last_undegraded":"2023-10-06T22:13:05.752951+0000","last_fullsized":"2023-10-06T22:13:05.752951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:56.472029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722942+0000","last_change":"2023-10-06T22:12:43.705718+0000","last_active":"2023-10-06T22:13:05.722942+0000","last_peered":"2023-10-06T22:13:05.722942+0000","last_clean":"2023-10-06T22:13:05.722942+0000","last_became_active":"2023-10-06T22:12:43.705479+0000","last_became_peered":"2023-10-06T22:12:43.705479+0000","last_unstale":"2023-10-06T22:13:05.722942+0000","last_undegraded":"2023-10-06T22:13:05.722942+0000","last_fullsized":"2023-10-06T22:13:05.722942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:55:36.393577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751324+0000","last_change":"2023-10-06T22:12:43.701891+0000","last_active":"2023-10-06T22:13:05.751324+0000","last_peered":"2023-10-06T22:13:05.751324+0000","last_clean":"2023-10-06T22:13:05.751324+0000","last_became_active":"2023-10-06T22:12:43.701535+0000","last_became_peered":"2023-10-06T22:12:43.701535+0000","last_unstale":"2023-10-06T22:13:05.751324+0000","last_undegraded":"2023-10-06T22:13:05.751324+0000","last_fullsized":"2023-10-06T22:13:05.751324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:17:40.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738839+0000","last_change":"2023-10-06T22:12:43.707075+0000","last_active":"2023-10-06T22:13:05.738839+0000","last_peered":"2023-10-06T22:13:05.738839+0000","last_clean":"2023-10-06T22:13:05.738839+0000","last_became_active":"2023-10-06T22:12:43.706723+0000","last_became_peered":"2023-10-06T22:12:43.706723+0000","last_unstale":"2023-10-06T22:13:05.738839+0000","last_undegraded":"2023-10-06T22:13:05.738839+0000","last_fullsized":"2023-10-06T22:13:05.738839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:18.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422288+0000","last_change":"2023-10-06T22:12:43.704656+0000","last_active":"2023-10-06T22:13:06.422288+0000","last_peered":"2023-10-06T22:13:06.422288+0000","last_clean":"2023-10-06T22:13:06.422288+0000","last_became_active":"2023-10-06T22:12:43.704312+0000","last_became_peered":"2023-10-06T22:12:43.704312+0000","last_unstale":"2023-10-06T22:13:06.422288+0000","last_undegraded":"2023-10-06T22:13:06.422288+0000","last_fullsized":"2023-10-06T22:13:06.422288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:21:37.951180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.720341+0000","last_change":"2023-10-06T22:12:43.702239+0000","last_active":"2023-10-06T22:13:05.720341+0000","last_peered":"2023-10-06T22:13:05.720341+0000","last_clean":"2023-10-06T22:13:05.720341+0000","last_became_active":"2023-10-06T22:12:43.701851+0000","last_became_peered":"2023-10-06T22:12:43.701851+0000","last_unstale":"2023-10-06T22:13:05.720341+0000","last_undegraded":"2023-10-06T22:13:05.720341+0000","last_fullsized":"2023-10-06T22:13:05.720341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:40:53.661587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65100,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":505,"num_read_kb":319,"num_write":230,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":195300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449111,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":444618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42399999999999999}]},{"osd":1,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.499},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47599999999999998}]},{"osd":6,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.45300000000000001}]},{"osd":7,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.62},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.63500000000000001}]},{"osd":10,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.65200000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678043,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1092830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.501},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59699999999999998,"5min":0.748,"15min":0.748},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":1.026,"15min":1.026},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96099999999999997,"5min":1.093,"15min":1.093},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.624},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78000000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81699999999999995,"5min":1.079,"15min":1.079},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.266,"15min":1.266},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81200000000000006},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.376},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.505}]},{"osd":8,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90200000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95199999999999996,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.80400000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.88800000000000001,"15min":0.88400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.85299999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.79200000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841568,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":444158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60999999999999999,"15min":0.621},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.99399999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.97199999999999998,"5min":1.036,"15min":1.036},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.83099999999999996,"15min":0.84699999999999998},"min":{"1min":0.53500000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95699999999999996,"5min":1.486,"15min":1.486},"last":0.73699999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78900000000000003,"15min":0.80500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0349999999999999,"5min":1.548,"15min":1.548},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91500000000000004,"15min":0.93100000000000005},"min":{"1min":0.66500000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.99099999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.95399999999999996},"min":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.0369999999999999,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.0149999999999999,"15min":1.0369999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1879999999999999,"5min":1.585,"15min":1.585},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2090000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.49299999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.998,"15min":1.016},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3129999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":0.91200000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2769999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.194,"5min":1.268,"15min":1.268},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.997,"15min":1.018},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.60599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103206,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":443533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45200000000000001},"min":{"1min":0.30199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.623,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.50900000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90300000000000002,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.92700000000000005}]},{"osd":10,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.63800000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.754,"15min":0.72999999999999998},"min":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.89500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1037176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.33600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44,"15min":0.436},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.627,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.626}]},{"osd":3,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65700000000000003,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.875,"15min":0.875},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.51500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.751}]},{"osd":9,"last update":"Fri Oct 6 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86899999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":6,"up_from":46,"seq":197568495662,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":443608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.45700000000000002},"min":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60899999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.441},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70599999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.83099999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.54200000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88300000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66400000000000003}]},{"osd":2,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85799999999999998,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82399999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79600000000000004,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.022,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.66600000000000004},"min":{"1min":0.61299999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89200000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.54300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.107,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.05,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79300000000000004,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99399999999999999,"5min":1.173,"15min":1.173},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83899999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97699999999999998,"5min":1.145,"15min":1.145},"last":0.80200000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.876,"15min":0.88400000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.014,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.86199999999999999,"15min":0.86499999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.73099999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.88600000000000001,"15min":0.90400000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.90800000000000003,"15min":0.92500000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":0.96099999999999997,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.61299999999999999}]},{"osd":10,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":502968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.39700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55900000000000005,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Oct 6 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60899999999999999,"15min":0.57299999999999995},"min":{"1min":0.48299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Oct 6 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.41499999999999998,"15min":0.38700000000000001},"min":{"1min":0.318,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.42199999999999999},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.873,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.87}]},{"osd":6,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.623,"15min":0.59099999999999997},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61499999999999999,"15min":0.58899999999999997},"min":{"1min":0.441,"5min":0.375,"15min":0.375},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55700000000000005,"15min":0.53400000000000003},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri Oct 6 22:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89100000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.94199999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.79900000000000004,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.026,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79800000000000004}]},{"osd":10,"last update":"Fri Oct 6 22:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":1.0369999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":445000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60699999999999998,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96499999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.57899999999999996,"15min":0.51700000000000002},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0600000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.56799999999999995},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.77000000000000002}]},{"osd":3,"last update":"Fri Oct 6 22:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.67200000000000004,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82999999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.622},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.65300000000000002},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.86899999999999999},"min":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.121,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.103,"15min":1.103},"last":0.749},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Oct 6 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76000000000000001,"15min":0.77100000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92200000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89300000000000002,"15min":0.90100000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Oct 6 22:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.71599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":502965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.432},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.435,"5min":0.307,"15min":0.307},"max":{"1min":0.84299999999999997,"5min":1.125,"15min":1.125},"last":0.46300000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.503},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72799999999999998,"5min":0.873,"15min":0.873},"last":0.498},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57699999999999996,"15min":0.50700000000000001},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.45400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44900000000000001,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.62},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Oct 6 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.48599999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":1.002,"15min":1.002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89100000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.629},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79000000000000004}]},{"osd":9,"last update":"Fri Oct 6 22:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0409999999999999,"5min":1.633,"15min":1.633},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73799999999999999,"15min":0.73899999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91400000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.88800000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":443439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.435,"15min":0.39100000000000001},"min":{"1min":0.30599999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.60799999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.45700000000000002,"15min":0.40699999999999997},"min":{"1min":0.33300000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.82199999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66700000000000004,"15min":0.60099999999999998},"min":{"1min":0.51800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84299999999999997,"5min":1.069,"15min":1.069},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66300000000000003,"15min":0.60799999999999998},"min":{"1min":0.57099999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92700000000000005,"5min":1.089,"15min":1.089},"last":0.88100000000000001}]},{"osd":2,"last update":"Fri Oct 6 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.54400000000000004,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88400000000000001,"5min":1.046,"15min":1.046},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70499999999999996,"15min":0.64300000000000002},"min":{"1min":0.55300000000000005,"5min":0.434,"15min":0.434},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.52800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 22:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.64200000000000002},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.95699999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.751},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95499999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.62},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76000000000000001,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0509999999999999,"15min":1.133},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.778,"5min":7.0529999999999999,"15min":7.0529999999999999},"last":0.93100000000000005}]},{"osd":8,"last update":"Fri Oct 6 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.91700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.077,"5min":1.214,"15min":1.214},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.77000000000000002}]},{"osd":10,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.94899999999999995,"15min":0.95399999999999996},"min":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":444126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.36199999999999999},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54300000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53500000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.48699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.60899999999999999,"15min":0.57299999999999995},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.54500000000000004},"min":{"1min":0.47799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.53700000000000003},"min":{"1min":0.49399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64200000000000002,"15min":0.61699999999999999},"min":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.86899999999999999,"5min":1,"15min":1},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55300000000000005,"15min":0.52300000000000002},"min":{"1min":0.374,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83499999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61699999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.47799999999999998}]},{"osd":8,"last update":"Fri Oct 6 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.56299999999999994,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57299999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70499999999999996,"15min":0.71399999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.81699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95699999999999996,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77100000000000002}]},{"osd":10,"last update":"Fri Oct 6 22:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757300,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1556480,"data_stored":1036737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.39800000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.70999999999999996}]},{"osd":4,"last update":"Fri Oct 6 22:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.60199999999999998,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Oct 6 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65000000000000002,"15min":0.624},"min":{"1min":0.42599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86599999999999999,"5min":1.274,"15min":1.274},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.60299999999999998},"min":{"1min":0.59299999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96599999999999997,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65200000000000002,"15min":0.66200000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89900000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.77600000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63400000000000001}]},{"osd":9,"last update":"Fri Oct 6 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0249999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61399999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.98499999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.89500000000000002}]},{"osd":10,"last update":"Fri Oct 6 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T22:13:41.649 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-06T22:13:41.649 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T22:13:41.649 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-06T22:13:41.649 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-06T22:13:42.389 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:42.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:42 smithi057 ceph-mon[98492]: from='client.24886 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:13:42.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:42 smithi057 ceph-mon[98492]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:13:43.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:42 smithi190 ceph-mon[134891]: from='client.24886 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:13:43.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:42 smithi190 ceph-mon[134891]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:13:43.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:42 smithi156 ceph-mon[134358]: from='client.24886 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:13:43.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:42 smithi156 ceph-mon[134358]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:13:43.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 -- 172.21.15.57:0/2124046057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 msgr2=0x7f8334110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:43.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 --2- 172.21.15.57:0/2124046057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 0x7f8334110030 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b0a0 tx=0x7f832c00b3b0 comp rx=0 tx=0).stop 2023-10-06T22:13:43.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 -- 172.21.15.57:0/2124046057 shutdown_connections 2023-10-06T22:13:43.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 --2- 172.21.15.57:0/2124046057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 0x7f8334110030 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:43.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 --2- 172.21.15.57:0/2124046057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103380 0x7f8334103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:43.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 --2- 172.21.15.57:0/2124046057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8334103d30 0x7f8334107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:43.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 -- 172.21.15.57:0/2124046057 >> 172.21.15.57:0/2124046057 conn(0x7f83340fec10 msgr2=0x7f8334101030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:43.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 -- 172.21.15.57:0/2124046057 shutdown_connections 2023-10-06T22:13:43.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.884+0000 7f833ad00700 1 -- 172.21.15.57:0/2124046057 wait complete. 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.885+0000 7f833ad00700 1 Processor -- start 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.885+0000 7f833ad00700 1 -- start start 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f833ad00700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8334103380 0x7f8334199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f833ad00700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103d30 0x7f83341998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f833ad00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 0x7f833419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f833ad00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419e330 con 0x7f8334108460 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f833ad00700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419e4a0 con 0x7f8334103380 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f833ad00700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419e660 con 0x7f8334103d30 2023-10-06T22:13:43.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f8333fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103d30 0x7f83341998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f833929d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 0x7f833419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f8333fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103d30 0x7f83341998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56658/0 (socket says 172.21.15.57:56658) 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f8333fff700 1 -- 172.21.15.57:0/4224825447 learned_addr learned my addr 172.21.15.57:0/4224825447 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.886+0000 7f8338a9c700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8334103380 0x7f8334199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8333fff700 1 -- 172.21.15.57:0/4224825447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8334103380 msgr2=0x7f8334199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8333fff700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8334103380 0x7f8334199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8333fff700 1 -- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 msgr2=0x7f833419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8333fff700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 0x7f833419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8333fff700 1 -- 172.21.15.57:0/4224825447 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009d00 con 0x7f8334103d30 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f833929d700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 0x7f833419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:43.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8333fff700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103d30 0x7f83341998b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f832000e970 tx=0x7f832000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:43.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8331ffb700 1 -- 172.21.15.57:0/4224825447 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320009790 con 0x7f8334103d30 2023-10-06T22:13:43.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8331ffb700 1 -- 172.21.15.57:0/4224825447 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f832000c7e0 con 0x7f8334103d30 2023-10-06T22:13:43.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833419e940 con 0x7f8334103d30 2023-10-06T22:13:43.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f8331ffb700 1 -- 172.21.15.57:0/4224825447 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83200054e0 con 0x7f8334103d30 2023-10-06T22:13:43.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.887+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83341a61a0 con 0x7f8334103d30 2023-10-06T22:13:43.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.888+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833404ebb0 con 0x7f8334103d30 2023-10-06T22:13:43.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.890+0000 7f8331ffb700 1 -- 172.21.15.57:0/4224825447 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f832000c950 con 0x7f8334103d30 2023-10-06T22:13:43.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.892+0000 7f8331ffb700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f83240a1590 0x7f83240a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:43.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.892+0000 7f8331ffb700 1 -- 172.21.15.57:0/4224825447 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f8320014070 con 0x7f8334103d30 2023-10-06T22:13:43.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.892+0000 7f8338a9c700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f83240a1590 0x7f83240a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:43.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.892+0000 7f8338a9c700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f83240a1590 0x7f83240a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8328007be0 tx=0x7f83280073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:43.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:43.894+0000 7f8331ffb700 1 -- 172.21.15.57:0/4224825447 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f83200964e0 con 0x7f8334103d30 2023-10-06T22:13:43.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:43 smithi057 ceph-mon[98492]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:13:44.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:43 smithi190 ceph-mon[134891]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:13:44.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.148+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f833419ead0 con 0x7f8334103d30 2023-10-06T22:13:44.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.149+0000 7f8331ffb700 1 -- 172.21.15.57:0/4224825447 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f833419ead0 con 0x7f8334103d30 2023-10-06T22:13:44.150 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:44.150 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T22:13:44.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f83240a1590 msgr2=0x7f83240a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:44.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f83240a1590 0x7f83240a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8328007be0 tx=0x7f83280073d0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103d30 msgr2=0x7f83341998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103d30 0x7f83341998b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f832000e970 tx=0x7f832000ec80 comp rx=0 tx=0).stop 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 shutdown_connections 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f83240a1590 0x7f83240a3a50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8334108460 0x7f833419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8334103380 0x7f8334199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 --2- 172.21.15.57:0/4224825447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8334103d30 0x7f83341998b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 >> 172.21.15.57:0/4224825447 conn(0x7f83340fec10 msgr2=0x7f83340ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 shutdown_connections 2023-10-06T22:13:44.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.151+0000 7f833ad00700 1 -- 172.21.15.57:0/4224825447 wait complete. 2023-10-06T22:13:44.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:43 smithi156 ceph-mon[134358]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:13:44.667 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-06T22:13:44.668 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:13:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.875+0000 7f8fc85ff700 1 -- 172.21.15.57:0/3198631437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 msgr2=0x7f8fc01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.875+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/3198631437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc01012c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009990 tx=0x7f8fbc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.876+0000 7f8fc85ff700 1 -- 172.21.15.57:0/3198631437 shutdown_connections 2023-10-06T22:13:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.876+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/3198631437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0102a40 0x7f8fc0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.876+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/3198631437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc01012c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.876+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/3198631437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc0102080 0x7f8fc0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.876+0000 7f8fc85ff700 1 -- 172.21.15.57:0/3198631437 >> 172.21.15.57:0/3198631437 conn(0x7f8fc00fc420 msgr2=0x7f8fc00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.876+0000 7f8fc85ff700 1 -- 172.21.15.57:0/3198631437 shutdown_connections 2023-10-06T22:13:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.876+0000 7f8fc85ff700 1 -- 172.21.15.57:0/3198631437 wait complete. 2023-10-06T22:13:44.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 Processor -- start 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 -- start start 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc0102080 0x7f8fc00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0102a40 0x7f8fc00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc0073f80 con 0x7f8fc0100ea0 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00740f0 con 0x7f8fc0102a40 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc85ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc01a0c60 con 0x7f8fc0102080 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.877+0000 7f8fc639b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc0074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42874/0 (socket says 172.21.15.57:42874) 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 -- 172.21.15.57:0/4255225770 learned_addr learned my addr 172.21.15.57:0/4255225770 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc5b9a700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc0102080 0x7f8fc00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:44.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc6b9c700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0102a40 0x7f8fc00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 -- 172.21.15.57:0/4255225770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc0102080 msgr2=0x7f8fc00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc0102080 0x7f8fc00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 -- 172.21.15.57:0/4255225770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0102a40 msgr2=0x7f8fc00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0102a40 0x7f8fc00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 -- 172.21.15.57:0/4255225770 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0095f0 con 0x7f8fc0100ea0 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc639b700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc0074c60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0098e0 tx=0x7f8fbc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fb77fe700 1 -- 172.21.15.57:0/4255225770 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc01d070 con 0x7f8fc0100ea0 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fb77fe700 1 -- 172.21.15.57:0/4255225770 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fbc00be40 con 0x7f8fc0100ea0 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc01a0ee0 con 0x7f8fc0100ea0 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.878+0000 7f8fb77fe700 1 -- 172.21.15.57:0/4255225770 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0173f0 con 0x7f8fc0100ea0 2023-10-06T22:13:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.879+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc01a1250 con 0x7f8fc0100ea0 2023-10-06T22:13:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.880+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc0108510 con 0x7f8fc0100ea0 2023-10-06T22:13:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.881+0000 7f8fb77fe700 1 -- 172.21.15.57:0/4255225770 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8fbc00fcc0 con 0x7f8fc0100ea0 2023-10-06T22:13:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.883+0000 7f8fb77fe700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8fac0a1550 0x7f8fac0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.883+0000 7f8fb77fe700 1 -- 172.21.15.57:0/4255225770 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f8fbc0ce120 con 0x7f8fc0100ea0 2023-10-06T22:13:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.883+0000 7f8fc5b9a700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8fac0a1550 0x7f8fac0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:44.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.884+0000 7f8fc5b9a700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8fac0a1550 0x7f8fac0a3a10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0100bc0 tx=0x7f8fb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:44.887+0000 7f8fb77fe700 1 -- 172.21.15.57:0/4255225770 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8fbc097680 con 0x7f8fc0100ea0 2023-10-06T22:13:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:44 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/4224825447' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T22:13:45.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:44 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/4224825447' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T22:13:45.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.100+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fc010c760 con 0x7f8fc0100ea0 2023-10-06T22:13:45.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.102+0000 7f8fb77fe700 1 -- 172.21.15.57:0/4255225770 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7f8fbc09aca0 con 0x7f8fc0100ea0 2023-10-06T22:13:45.103 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:45.103 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8fac0a1550 msgr2=0x7f8fac0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8fac0a1550 0x7f8fac0a3a10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0100bc0 tx=0x7f8fb0009260 comp rx=0 tx=0).stop 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 msgr2=0x7f8fc0074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc0074c60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0098e0 tx=0x7f8fbc009580 comp rx=0 tx=0).stop 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 shutdown_connections 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8fac0a1550 0x7f8fac0a3a10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fc0102a40 0x7f8fc00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc0100ea0 0x7f8fc0074c60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 --2- 172.21.15.57:0/4255225770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fc0102080 0x7f8fc00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 >> 172.21.15.57:0/4255225770 conn(0x7f8fc00fc420 msgr2=0x7f8fc0105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:45.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 shutdown_connections 2023-10-06T22:13:45.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.104+0000 7f8fc85ff700 1 -- 172.21.15.57:0/4255225770 wait complete. 2023-10-06T22:13:45.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:13:45.196 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:13:45.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:44 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/4224825447' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 -- 172.21.15.57:0/956252670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14101950 msgr2=0x7f0f14108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 --2- 172.21.15.57:0/956252670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14101950 0x7f0f14108200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00b0d0 tx=0x7f0f0c00b3e0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 -- 172.21.15.57:0/956252670 shutdown_connections 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 --2- 172.21.15.57:0/956252670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14101950 0x7f0f14108200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 --2- 172.21.15.57:0/956252670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f140ffdc0 0x7f0f141001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 --2- 172.21.15.57:0/956252670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f14100fb0 0x7f0f14101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 -- 172.21.15.57:0/956252670 >> 172.21.15.57:0/956252670 conn(0x7f0f140fb330 msgr2=0x7f0f140fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 -- 172.21.15.57:0/956252670 shutdown_connections 2023-10-06T22:13:45.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.403+0000 7f0f19fff700 1 -- 172.21.15.57:0/956252670 wait complete. 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.404+0000 7f0f19fff700 1 Processor -- start 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f19fff700 1 -- start start 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f19fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f140ffdc0 0x7f0f14198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f19fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14100fb0 0x7f0f14199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f19fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f14101950 0x7f0f141a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f19fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a0a70 con 0x7f0f14100fb0 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f19fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a0bb0 con 0x7f0f14101950 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f19fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a0d40 con 0x7f0f140ffdc0 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14100fb0 0x7f0f14199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14100fb0 0x7f0f14199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42878/0 (socket says 172.21.15.57:42878) 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f12ffd700 1 -- 172.21.15.57:0/2573493378 learned_addr learned my addr 172.21.15.57:0/2573493378 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f137fe700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f140ffdc0 0x7f0f14198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.405+0000 7f0f13fff700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f14101950 0x7f0f141a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f12ffd700 1 -- 172.21.15.57:0/2573493378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f140ffdc0 msgr2=0x7f0f14198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f12ffd700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f140ffdc0 0x7f0f14198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f12ffd700 1 -- 172.21.15.57:0/2573493378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f14101950 msgr2=0x7f0f141a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f12ffd700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f14101950 0x7f0f141a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f12ffd700 1 -- 172.21.15.57:0/2573493378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c009d00 con 0x7f0f14100fb0 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f137fe700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f140ffdc0 0x7f0f14198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f12ffd700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14100fb0 0x7f0f14199430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e9a0 tx=0x7f0f0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:45.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f10ff9700 1 -- 172.21.15.57:0/2573493378 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08009790 con 0x7f0f14100fb0 2023-10-06T22:13:45.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f10ff9700 1 -- 172.21.15.57:0/2573493378 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f0800cbd0 con 0x7f0f14100fb0 2023-10-06T22:13:45.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f141a0fc0 con 0x7f0f14100fb0 2023-10-06T22:13:45.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f10ff9700 1 -- 172.21.15.57:0/2573493378 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08018680 con 0x7f0f14100fb0 2023-10-06T22:13:45.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.406+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f141a14f0 con 0x7f0f14100fb0 2023-10-06T22:13:45.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.407+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f14193510 con 0x7f0f14100fb0 2023-10-06T22:13:45.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.409+0000 7f0f10ff9700 1 -- 172.21.15.57:0/2573493378 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0f0800cd40 con 0x7f0f14100fb0 2023-10-06T22:13:45.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.411+0000 7f0f10ff9700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0efc0a1590 0x7f0efc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.411+0000 7f0f10ff9700 1 -- 172.21.15.57:0/2573493378 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f0f08014070 con 0x7f0f14100fb0 2023-10-06T22:13:45.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.411+0000 7f0f137fe700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0efc0a1590 0x7f0efc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.412+0000 7f0f137fe700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0efc0a1590 0x7f0efc0a3a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005950 tx=0x7f0f040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:45.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.415+0000 7f0f10ff9700 1 -- 172.21.15.57:0/2573493378 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0f08052100 con 0x7f0f14100fb0 2023-10-06T22:13:45.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.627+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f1410c940 con 0x7f0f14100fb0 2023-10-06T22:13:45.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.627+0000 7f0f10ff9700 1 -- 172.21.15.57:0/2573493378 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f08096560 con 0x7f0f14100fb0 2023-10-06T22:13:45.629 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:13:45.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0efc0a1590 msgr2=0x7f0efc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0efc0a1590 0x7f0efc0a3a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005950 tx=0x7f0f040058e0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14100fb0 msgr2=0x7f0f14199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14100fb0 0x7f0f14199430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e9a0 tx=0x7f0f0800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 shutdown_connections 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f0efc0a1590 0x7f0efc0a3a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f14101950 0x7f0f141a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f140ffdc0 0x7f0f14198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 --2- 172.21.15.57:0/2573493378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f14100fb0 0x7f0f14199430 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 >> 172.21.15.57:0/2573493378 conn(0x7f0f140fb330 msgr2=0x7f0f14106890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 shutdown_connections 2023-10-06T22:13:45.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.630+0000 7f0f19fff700 1 -- 172.21.15.57:0/2573493378 wait complete. 2023-10-06T22:13:45.650 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:13:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 -- 172.21.15.57:0/1519566286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102a30 msgr2=0x7fb930109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 --2- 172.21.15.57:0/1519566286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102a30 0x7fb930109350 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb92800b0a0 tx=0x7fb92800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 -- 172.21.15.57:0/1519566286 shutdown_connections 2023-10-06T22:13:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 --2- 172.21.15.57:0/1519566286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102a30 0x7fb930109350 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 --2- 172.21.15.57:0/1519566286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb930100e90 0x7fb9301012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 --2- 172.21.15.57:0/1519566286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb930102070 0x7fb9301024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 -- 172.21.15.57:0/1519566286 >> 172.21.15.57:0/1519566286 conn(0x7fb9300fc430 msgr2=0x7fb9300fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:45.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.860+0000 7fb936bff700 1 -- 172.21.15.57:0/1519566286 shutdown_connections 2023-10-06T22:13:45.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.861+0000 7fb936bff700 1 -- 172.21.15.57:0/1519566286 wait complete. 2023-10-06T22:13:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.861+0000 7fb936bff700 1 Processor -- start 2023-10-06T22:13:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb936bff700 1 -- start start 2023-10-06T22:13:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb936bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb930100e90 0x7fb930074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb936bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102070 0x7fb9300732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb936bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb930102a30 0x7fb9300737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb936bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb930073f80 con 0x7fb930102070 2023-10-06T22:13:45.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb936bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9300740f0 con 0x7fb930102a30 2023-10-06T22:13:45.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb936bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301a0d60 con 0x7fb930100e90 2023-10-06T22:13:45.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102070 0x7fb9300732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb92ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102070 0x7fb9300732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42896/0 (socket says 172.21.15.57:42896) 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb92ffff700 1 -- 172.21.15.57:0/1231983007 learned_addr learned my addr 172.21.15.57:0/1231983007 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb93519c700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb930102a30 0x7fb9300737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.862+0000 7fb93499b700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb930100e90 0x7fb930074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92ffff700 1 -- 172.21.15.57:0/1231983007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb930100e90 msgr2=0x7fb930074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92ffff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb930100e90 0x7fb930074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92ffff700 1 -- 172.21.15.57:0/1231983007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb930102a30 msgr2=0x7fb9300737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92ffff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb930102a30 0x7fb9300737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92ffff700 1 -- 172.21.15.57:0/1231983007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb928009d00 con 0x7fb930102070 2023-10-06T22:13:45.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92ffff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102070 0x7fb9300732a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d980 tx=0x7fb91c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:45.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92dffb700 1 -- 172.21.15.57:0/1231983007 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c004020 con 0x7fb930102070 2023-10-06T22:13:45.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92dffb700 1 -- 172.21.15.57:0/1231983007 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb91c00f690 con 0x7fb930102070 2023-10-06T22:13:45.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9301a0fe0 con 0x7fb930102070 2023-10-06T22:13:45.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb92dffb700 1 -- 172.21.15.57:0/1231983007 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c003a60 con 0x7fb930102070 2023-10-06T22:13:45.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.863+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9301a1470 con 0x7fb930102070 2023-10-06T22:13:45.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.864+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93010c750 con 0x7fb930102070 2023-10-06T22:13:45.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.866+0000 7fb92dffb700 1 -- 172.21.15.57:0/1231983007 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fb91c00fb00 con 0x7fb930102070 2023-10-06T22:13:45.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.868+0000 7fb92dffb700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb9200a1510 0x7fb9200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:45.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.868+0000 7fb92dffb700 1 -- 172.21.15.57:0/1231983007 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fb91c0cca90 con 0x7fb930102070 2023-10-06T22:13:45.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.868+0000 7fb93499b700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb9200a1510 0x7fb9200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:45.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.869+0000 7fb93499b700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb9200a1510 0x7fb9200a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb930101be0 tx=0x7fb924009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:45.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:45.872+0000 7fb92dffb700 1 -- 172.21.15.57:0/1231983007 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb91c095f60 con 0x7fb930102070 2023-10-06T22:13:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:45 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/4255225770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:45 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2573493378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:13:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:45 smithi057 ceph-mon[98492]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:46.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:45 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/4255225770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:46.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:45 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2573493378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:13:46.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:45 smithi190 ceph-mon[134891]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:46.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.084+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb930107df0 con 0x7fb930102070 2023-10-06T22:13:46.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.084+0000 7fb92dffb700 1 -- 172.21.15.57:0/1231983007 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb91c099580 con 0x7fb930102070 2023-10-06T22:13:46.085 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:13:46.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb9200a1510 msgr2=0x7fb9200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb9200a1510 0x7fb9200a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb930101be0 tx=0x7fb924009260 comp rx=0 tx=0).stop 2023-10-06T22:13:46.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102070 msgr2=0x7fb9300732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102070 0x7fb9300732a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d980 tx=0x7fb91c00dc90 comp rx=0 tx=0).stop 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 shutdown_connections 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fb9200a1510 0x7fb9200a39d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb930102a30 0x7fb9300737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb930100e90 0x7fb930074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 --2- 172.21.15.57:0/1231983007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102070 0x7fb9300732a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 >> 172.21.15.57:0/1231983007 conn(0x7fb9300fc430 msgr2=0x7fb9300fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 shutdown_connections 2023-10-06T22:13:46.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.087+0000 7fb936bff700 1 -- 172.21.15.57:0/1231983007 wait complete. 2023-10-06T22:13:46.107 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:13:46.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:45 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/4255225770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:46.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:45 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2573493378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:13:46.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:45 smithi156 ceph-mon[134358]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:46.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 -- 172.21.15.57:0/3678277493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 msgr2=0x7f67d40732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/3678277493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d40732e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009990 tx=0x7f67d0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 -- 172.21.15.57:0/3678277493 shutdown_connections 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/3678277493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d4108de0 0x7f67d410b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/3678277493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d40732e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/3678277493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67d40738b0 0x7f67d4073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 -- 172.21.15.57:0/3678277493 >> 172.21.15.57:0/3678277493 conn(0x7f67d40fc460 msgr2=0x7f67d40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 -- 172.21.15.57:0/3678277493 shutdown_connections 2023-10-06T22:13:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.311+0000 7f67dc7ff700 1 -- 172.21.15.57:0/3678277493 wait complete. 2023-10-06T22:13:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.312+0000 7f67dc7ff700 1 Processor -- start 2023-10-06T22:13:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.312+0000 7f67dc7ff700 1 -- start start 2023-10-06T22:13:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.312+0000 7f67dc7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67d40738b0 0x7f67d4198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.312+0000 7f67dc7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.312+0000 7f67dc7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d4108de0 0x7f67d41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67dc7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d41a09d0 con 0x7f67d4074e90 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67dc7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d41a0b10 con 0x7f67d4108de0 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67dc7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d41a0ca0 con 0x7f67d40738b0 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d4199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42908/0 (socket says 172.21.15.57:42908) 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 -- 172.21.15.57:0/255189165 learned_addr learned my addr 172.21.15.57:0/255189165 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67da59b700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67d40738b0 0x7f67d4198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67dad9c700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d4108de0 0x7f67d41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 -- 172.21.15.57:0/255189165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67d40738b0 msgr2=0x7f67d4198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67d40738b0 0x7f67d4198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 -- 172.21.15.57:0/255189165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d4108de0 msgr2=0x7f67d41a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d4108de0 0x7f67d41a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 -- 172.21.15.57:0/255189165 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00095f0 con 0x7f67d4074e90 2023-10-06T22:13:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67dad9c700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d4108de0 0x7f67d41a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.313+0000 7f67d9d9a700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d4199390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9a0 tx=0x7f67c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.314+0000 7f67cb7fe700 1 -- 172.21.15.57:0/255189165 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4009790 con 0x7f67d4074e90 2023-10-06T22:13:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.314+0000 7f67cb7fe700 1 -- 172.21.15.57:0/255189165 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c400cbd0 con 0x7f67d4074e90 2023-10-06T22:13:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.314+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d41a0f80 con 0x7f67d4074e90 2023-10-06T22:13:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.314+0000 7f67cb7fe700 1 -- 172.21.15.57:0/255189165 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4018680 con 0x7f67d4074e90 2023-10-06T22:13:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.314+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d41a1450 con 0x7f67d4074e90 2023-10-06T22:13:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.315+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d41074e0 con 0x7f67d4074e90 2023-10-06T22:13:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.317+0000 7f67cb7fe700 1 -- 172.21.15.57:0/255189165 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f67c400cd40 con 0x7f67d4074e90 2023-10-06T22:13:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.318+0000 7f67cb7fe700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f67c00a1590 0x7f67c00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.318+0000 7f67cb7fe700 1 -- 172.21.15.57:0/255189165 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f67c4014070 con 0x7f67d4074e90 2023-10-06T22:13:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.318+0000 7f67da59b700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f67c00a1590 0x7f67c00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.319+0000 7f67da59b700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f67c00a1590 0x7f67c00a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f67d000b2e0 tx=0x7f67d0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.320+0000 7f67cb7fe700 1 -- 172.21.15.57:0/255189165 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f67c4096820 con 0x7f67d4074e90 2023-10-06T22:13:46.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.532+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67d410eae0 con 0x7f67d4074e90 2023-10-06T22:13:46.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.532+0000 7f67cb7fe700 1 -- 172.21.15.57:0/255189165 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f67c40260a0 con 0x7f67d4074e90 2023-10-06T22:13:46.534 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f67c00a1590 msgr2=0x7f67c00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f67c00a1590 0x7f67c00a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f67d000b2e0 tx=0x7f67d0005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 msgr2=0x7f67d4199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d4199390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9a0 tx=0x7f67c400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 shutdown_connections 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f67c00a1590 0x7f67c00a3a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d4108de0 0x7f67d41a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67d40738b0 0x7f67d4198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 --2- 172.21.15.57:0/255189165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67d4074e90 0x7f67d4199390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 >> 172.21.15.57:0/255189165 conn(0x7f67d40fc460 msgr2=0x7f67d40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:46.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 shutdown_connections 2023-10-06T22:13:46.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.535+0000 7f67dc7ff700 1 -- 172.21.15.57:0/255189165 wait complete. 2023-10-06T22:13:46.556 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:13:46.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 -- 172.21.15.57:0/2181890311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 msgr2=0x7fabd01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 --2- 172.21.15.57:0/2181890311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd01012c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009990 tx=0x7fabcc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 -- 172.21.15.57:0/2181890311 shutdown_connections 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 --2- 172.21.15.57:0/2181890311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabd0102a40 0x7fabd0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 --2- 172.21.15.57:0/2181890311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd01012c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 --2- 172.21.15.57:0/2181890311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabd0102080 0x7fabd0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 -- 172.21.15.57:0/2181890311 >> 172.21.15.57:0/2181890311 conn(0x7fabd00fc420 msgr2=0x7fabd00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 -- 172.21.15.57:0/2181890311 shutdown_connections 2023-10-06T22:13:46.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.759+0000 7fabd7fff700 1 -- 172.21.15.57:0/2181890311 wait complete. 2023-10-06T22:13:46.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 Processor -- start 2023-10-06T22:13:46.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 -- start start 2023-10-06T22:13:46.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd0196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabd0102080 0x7fabd0197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabd0102a40 0x7fabd019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019e760 con 0x7fabd0100ea0 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019e8d0 con 0x7fabd0102a40 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.760+0000 7fabd7fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019ea90 con 0x7fabd0102080 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd0196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd0196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42920/0 (socket says 172.21.15.57:42920) 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 -- 172.21.15.57:0/4238678458 learned_addr learned my addr 172.21.15.57:0/4238678458 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd559a700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabd0102080 0x7fabd0197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd659c700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabd0102a40 0x7fabd019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 -- 172.21.15.57:0/4238678458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabd0102080 msgr2=0x7fabd0197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabd0102080 0x7fabd0197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 -- 172.21.15.57:0/4238678458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabd0102a40 msgr2=0x7fabd019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabd0102a40 0x7fabd019e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 -- 172.21.15.57:0/4238678458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc0095f0 con 0x7fabd0100ea0 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.761+0000 7fabd5d9b700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd0196c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00b680 tx=0x7fabcc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.762+0000 7fabc6ffd700 1 -- 172.21.15.57:0/4238678458 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc01d070 con 0x7fabd0100ea0 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.762+0000 7fabc6ffd700 1 -- 172.21.15.57:0/4238678458 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabcc021410 con 0x7fabd0100ea0 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.762+0000 7fabc6ffd700 1 -- 172.21.15.57:0/4238678458 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc017650 con 0x7fabd0100ea0 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.762+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd019ed10 con 0x7fabd0100ea0 2023-10-06T22:13:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.762+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd019f130 con 0x7fabd0100ea0 2023-10-06T22:13:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.763+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd010c760 con 0x7fabd0100ea0 2023-10-06T22:13:46.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.765+0000 7fabc6ffd700 1 -- 172.21.15.57:0/4238678458 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fabcc021580 con 0x7fabd0100ea0 2023-10-06T22:13:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.766+0000 7fabc6ffd700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabbc0a15d0 0x7fabbc0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.766+0000 7fabc6ffd700 1 -- 172.21.15.57:0/4238678458 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fabcc0ce2d0 con 0x7fabd0100ea0 2023-10-06T22:13:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.766+0000 7fabd559a700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabbc0a15d0 0x7fabbc0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.767+0000 7fabd559a700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabbc0a15d0 0x7fabbc0a3a90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fabd0100bc0 tx=0x7fabc000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:46.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:46.770+0000 7fabc6ffd700 1 -- 172.21.15.57:0/4238678458 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fabcc097830 con 0x7fabd0100ea0 2023-10-06T22:13:46.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:46 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1231983007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:13:46.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:46 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/255189165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:13:47.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.047+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabd0107e60 con 0x7fabd0100ea0 2023-10-06T22:13:47.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.048+0000 7fabc6ffd700 1 -- 172.21.15.57:0/4238678458 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fabcc09ae50 con 0x7fabd0100ea0 2023-10-06T22:13:47.049 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:47.049 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:13:47.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.050+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabbc0a15d0 msgr2=0x7fabbc0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.050+0000 7fabd7fff700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabbc0a15d0 0x7fabbc0a3a90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fabd0100bc0 tx=0x7fabc000b410 comp rx=0 tx=0).stop 2023-10-06T22:13:47.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.050+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 msgr2=0x7fabd0196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.050+0000 7fabd7fff700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd0196c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00b680 tx=0x7fabcc005740 comp rx=0 tx=0).stop 2023-10-06T22:13:47.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.050+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 shutdown_connections 2023-10-06T22:13:47.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.051+0000 7fabd7fff700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fabbc0a15d0 0x7fabbc0a3a90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.051+0000 7fabd7fff700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabd0102a40 0x7fabd019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.051+0000 7fabd7fff700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd0100ea0 0x7fabd0196c20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.051+0000 7fabd7fff700 1 --2- 172.21.15.57:0/4238678458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabd0102080 0x7fabd0197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.051+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 >> 172.21.15.57:0/4238678458 conn(0x7fabd00fc420 msgr2=0x7fabd00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:47.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.051+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 shutdown_connections 2023-10-06T22:13:47.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.051+0000 7fabd7fff700 1 -- 172.21.15.57:0/4238678458 wait complete. 2023-10-06T22:13:47.053 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:13:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:46 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1231983007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:13:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:46 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/255189165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:13:47.072 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:13:47.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:46 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1231983007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:13:47.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:46 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/255189165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:13:47.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 -- 172.21.15.57:0/1619277907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 msgr2=0x7fe2f40ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 --2- 172.21.15.57:0/1619277907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f40ffc30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80098f0 tx=0x7fe2e8009c00 comp rx=0 tx=0).stop 2023-10-06T22:13:47.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 -- 172.21.15.57:0/1619277907 shutdown_connections 2023-10-06T22:13:47.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 --2- 172.21.15.57:0/1619277907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2f4100b30 0x7fe2f4100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 --2- 172.21.15.57:0/1619277907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f40ffc30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 --2- 172.21.15.57:0/1619277907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4100170 0x7fe2f41005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 -- 172.21.15.57:0/1619277907 >> 172.21.15.57:0/1619277907 conn(0x7fe2f40fb370 msgr2=0x7fe2f40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:47.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 -- 172.21.15.57:0/1619277907 shutdown_connections 2023-10-06T22:13:47.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.286+0000 7fe2fafff700 1 -- 172.21.15.57:0/1619277907 wait complete. 2023-10-06T22:13:47.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.287+0000 7fe2fafff700 1 Processor -- start 2023-10-06T22:13:47.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.287+0000 7fe2fafff700 1 -- start start 2023-10-06T22:13:47.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2fafff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f4198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2fafff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4100170 0x7fe2f41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2fafff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2f4100b30 0x7fe2f41a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2fafff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f41a0a30 con 0x7fe2f4100b30 2023-10-06T22:13:47.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2fafff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f41a0b70 con 0x7fe2f4100170 2023-10-06T22:13:47.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2fafff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f41a0d00 con 0x7fe2f40ff810 2023-10-06T22:13:47.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f4198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f4198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56764/0 (socket says 172.21.15.57:56764) 2023-10-06T22:13:47.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 -- 172.21.15.57:0/2905109004 learned_addr learned my addr 172.21.15.57:0/2905109004 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:47.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f959c700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2f4100b30 0x7fe2f41a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f3fff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4100170 0x7fe2f41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 -- 172.21.15.57:0/2905109004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4100170 msgr2=0x7fe2f41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4100170 0x7fe2f41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 -- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2f4100b30 msgr2=0x7fe2f41a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2f4100b30 0x7fe2f41a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.288+0000 7fe2f8d9b700 1 -- 172.21.15.57:0/2905109004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00095f0 con 0x7fe2f40ff810 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.289+0000 7fe2f8d9b700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f4198eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009fd0 tx=0x7fe2e800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.289+0000 7fe2f1ffb700 1 -- 172.21.15.57:0/2905109004 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e801c070 con 0x7fe2f40ff810 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.289+0000 7fe2f1ffb700 1 -- 172.21.15.57:0/2905109004 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2e800f7d0 con 0x7fe2f40ff810 2023-10-06T22:13:47.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.289+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e8009520 con 0x7fe2f40ff810 2023-10-06T22:13:47.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.289+0000 7fe2f1ffb700 1 -- 172.21.15.57:0/2905109004 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8004ad0 con 0x7fe2f40ff810 2023-10-06T22:13:47.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.289+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f41a11c0 con 0x7fe2f40ff810 2023-10-06T22:13:47.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.290+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f410a5f0 con 0x7fe2f40ff810 2023-10-06T22:13:47.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.292+0000 7fe2f1ffb700 1 -- 172.21.15.57:0/2905109004 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe2e8020070 con 0x7fe2f40ff810 2023-10-06T22:13:47.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.294+0000 7fe2f1ffb700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2e40a15d0 0x7fe2e40a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.295+0000 7fe2f1ffb700 1 -- 172.21.15.57:0/2905109004 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fe2e80cd9b0 con 0x7fe2f40ff810 2023-10-06T22:13:47.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.295+0000 7fe2f3fff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2e40a15d0 0x7fe2e40a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.295+0000 7fe2f3fff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2e40a15d0 0x7fe2e40a3a90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0005fd0 tx=0x7fe2e0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:47.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.298+0000 7fe2f1ffb700 1 -- 172.21.15.57:0/2905109004 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe2e805d100 con 0x7fe2f40ff810 2023-10-06T22:13:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.522+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2f410ec70 con 0x7fe2f40ff810 2023-10-06T22:13:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.524+0000 7fe2f1ffb700 1 -- 172.21.15.57:0/2905109004 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7fe2e8096f70 con 0x7fe2f40ff810 2023-10-06T22:13:47.536 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:47.536 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:13:47.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2e40a15d0 msgr2=0x7fe2e40a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2e40a15d0 0x7fe2e40a3a90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0005fd0 tx=0x7fe2e0009310 comp rx=0 tx=0).stop 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 msgr2=0x7fe2f4198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f4198eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009fd0 tx=0x7fe2e800b790 comp rx=0 tx=0).stop 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 shutdown_connections 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe2e40a15d0 0x7fe2e40a3a90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2f4100b30 0x7fe2f41a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f40ff810 0x7fe2f4198eb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 --2- 172.21.15.57:0/2905109004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2f4100170 0x7fe2f41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 >> 172.21.15.57:0/2905109004 conn(0x7fe2f40fb370 msgr2=0x7fe2f40fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 shutdown_connections 2023-10-06T22:13:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.526+0000 7fe2fafff700 1 -- 172.21.15.57:0/2905109004 wait complete. 2023-10-06T22:13:47.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:13:47.561 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:13:47.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 -- 172.21.15.57:0/3245714027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 msgr2=0x7f2564073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 --2- 172.21.15.57:0/3245714027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564073d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2558009990 tx=0x7f2558009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 -- 172.21.15.57:0/3245714027 shutdown_connections 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 --2- 172.21.15.57:0/3245714027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2564108db0 0x7f256410b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 --2- 172.21.15.57:0/3245714027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564074ef0 0x7f2564073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 --2- 172.21.15.57:0/3245714027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564073d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 -- 172.21.15.57:0/3245714027 >> 172.21.15.57:0/3245714027 conn(0x7f25640fc460 msgr2=0x7f25640fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 -- 172.21.15.57:0/3245714027 shutdown_connections 2023-10-06T22:13:47.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.791+0000 7f2568fff700 1 -- 172.21.15.57:0/3245714027 wait complete. 2023-10-06T22:13:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.792+0000 7f2568fff700 1 Processor -- start 2023-10-06T22:13:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.792+0000 7f2568fff700 1 -- start start 2023-10-06T22:13:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2568fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2568fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564074ef0 0x7f2564199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2568fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2564108db0 0x7f25641a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2568fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641a09a0 con 0x7f2564108db0 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2568fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641a0ae0 con 0x7f2564074ef0 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2568fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641a0c70 con 0x7f2564073910 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56774/0 (socket says 172.21.15.57:56774) 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 -- 172.21.15.57:0/122427857 learned_addr learned my addr 172.21.15.57:0/122427857 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2561d9b700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564074ef0 0x7f2564199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f2562d9d700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2564108db0 0x7f25641a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 -- 172.21.15.57:0/122427857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564074ef0 msgr2=0x7f2564199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564074ef0 0x7f2564199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 -- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2564108db0 msgr2=0x7f25641a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2564108db0 0x7f25641a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.793+0000 7f256259c700 1 -- 172.21.15.57:0/122427857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2554009520 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.794+0000 7f256259c700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564198e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f255400eb30 tx=0x7f255400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.795+0000 7f25537fe700 1 -- 172.21.15.57:0/122427857 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554009880 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.795+0000 7f25537fe700 1 -- 172.21.15.57:0/122427857 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25540044e0 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.795+0000 7f25537fe700 1 -- 172.21.15.57:0/122427857 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554003d80 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.795+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25580095f0 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.795+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25641a11c0 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.796+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2564107460 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.798+0000 7f25537fe700 1 -- 172.21.15.57:0/122427857 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2554004020 con 0x7f2564073910 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.799+0000 7f25537fe700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f254c0a1590 0x7f254c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:47.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.799+0000 7f25537fe700 1 -- 172.21.15.57:0/122427857 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f2554014070 con 0x7f2564073910 2023-10-06T22:13:47.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.799+0000 7f2561d9b700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f254c0a1590 0x7f254c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:47.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.800+0000 7f2561d9b700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f254c0a1590 0x7f254c0a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f255800b2e0 tx=0x7f2558005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:47.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:47.803+0000 7f25537fe700 1 -- 172.21.15.57:0/122427857 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2554096840 con 0x7f2564073910 2023-10-06T22:13:47.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:47 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:13:47.958 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:47 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/4238678458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:47.958 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:47 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2905109004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:47.958 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:47 smithi057 ceph-mon[98492]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:48.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.020+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f256410ebb0 con 0x7f2564073910 2023-10-06T22:13:48.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.021+0000 7f25537fe700 1 -- 172.21.15.57:0/122427857 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2554099e60 con 0x7f2564073910 2023-10-06T22:13:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:13:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f254c0a1590 msgr2=0x7f254c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f254c0a1590 0x7f254c0a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f255800b2e0 tx=0x7f2558005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 msgr2=0x7f2564198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564198e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f255400eb30 tx=0x7f255400ee40 comp rx=0 tx=0).stop 2023-10-06T22:13:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 shutdown_connections 2023-10-06T22:13:48.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f254c0a1590 0x7f254c0a3a50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2564108db0 0x7f25641a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2564073910 0x7f2564198e20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 --2- 172.21.15.57:0/122427857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2564074ef0 0x7f2564199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 >> 172.21.15.57:0/122427857 conn(0x7f25640fc460 msgr2=0x7f25640fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.023+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 shutdown_connections 2023-10-06T22:13:48.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.024+0000 7f2568fff700 1 -- 172.21.15.57:0/122427857 wait complete. 2023-10-06T22:13:48.046 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:13:48.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:47 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:13:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/4238678458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2905109004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:48.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:47 smithi190 ceph-mon[134891]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:48.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:47 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:13:48.279 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:47 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/4238678458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:48.279 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:47 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2905109004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:48.280 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:47 smithi156 ceph-mon[134358]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:48.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 -- 172.21.15.57:0/4266709482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 msgr2=0x7f342c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 --2- 172.21.15.57:0/4266709482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c073d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3414009990 tx=0x7f3414009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 -- 172.21.15.57:0/4266709482 shutdown_connections 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 --2- 172.21.15.57:0/4266709482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c10d4b0 0x7f342c10f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 --2- 172.21.15.57:0/4266709482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f342c074ef0 0x7f342c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 --2- 172.21.15.57:0/4266709482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c073d90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 -- 172.21.15.57:0/4266709482 >> 172.21.15.57:0/4266709482 conn(0x7f342c0fc430 msgr2=0x7f342c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 -- 172.21.15.57:0/4266709482 shutdown_connections 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.266+0000 7f3430dff700 1 -- 172.21.15.57:0/4266709482 wait complete. 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.267+0000 7f3430dff700 1 Processor -- start 2023-10-06T22:13:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.267+0000 7f3430dff700 1 -- start start 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.267+0000 7f3430dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f3430dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c074ef0 0x7f342c19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f3430dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f342c10d4b0 0x7f342c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f3430dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1a4d90 con 0x7f342c10d4b0 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f3430dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1a4ed0 con 0x7f342c074ef0 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f3430dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1a5060 con 0x7f342c073910 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c074ef0 0x7f342c19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c074ef0 0x7f342c19d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55680/0 (socket says 172.21.15.57:55680) 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56784/0 (socket says 172.21.15.57:56784) 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342a59c700 1 -- 172.21.15.57:0/3419647292 learned_addr learned my addr 172.21.15.57:0/3419647292 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342ad9d700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f342c10d4b0 0x7f342c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342259c700 1 -- 172.21.15.57:0/3419647292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c074ef0 msgr2=0x7f342c19d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342259c700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c074ef0 0x7f342c19d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342259c700 1 -- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f342c10d4b0 msgr2=0x7f342c1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342259c700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f342c10d4b0 0x7f342c1a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342259c700 1 -- 172.21.15.57:0/3419647292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34140095f0 con 0x7f342c073910 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.268+0000 7f342a59c700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c074ef0 0x7f342c19d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.269+0000 7f342259c700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c19d210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f341c00e9f0 tx=0x7f341c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.269+0000 7f3423fff700 1 -- 172.21.15.57:0/3419647292 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00c9c0 con 0x7f342c073910 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.269+0000 7f3423fff700 1 -- 172.21.15.57:0/3419647292 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341c010430 con 0x7f342c073910 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.269+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c1a5340 con 0x7f342c073910 2023-10-06T22:13:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.269+0000 7f3423fff700 1 -- 172.21.15.57:0/3419647292 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c018680 con 0x7f342c073910 2023-10-06T22:13:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.269+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c1a58a0 con 0x7f342c073910 2023-10-06T22:13:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.270+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c103000 con 0x7f342c073910 2023-10-06T22:13:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.272+0000 7f3423fff700 1 -- 172.21.15.57:0/3419647292 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f341c010860 con 0x7f342c073910 2023-10-06T22:13:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.273+0000 7f3423fff700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f34180a1590 0x7f34180a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.273+0000 7f3423fff700 1 -- 172.21.15.57:0/3419647292 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f341c014070 con 0x7f342c073910 2023-10-06T22:13:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.273+0000 7f342a59c700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f34180a1590 0x7f34180a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.274+0000 7f342a59c700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f34180a1590 0x7f34180a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f341400b2e0 tx=0x7f3414005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:48.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.277+0000 7f3423fff700 1 -- 172.21.15.57:0/3419647292 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f341c052100 con 0x7f342c073910 2023-10-06T22:13:48.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.513+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f342c112d20 con 0x7f342c073910 2023-10-06T22:13:48.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.514+0000 7f3423fff700 1 -- 172.21.15.57:0/3419647292 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f342c112d20 con 0x7f342c073910 2023-10-06T22:13:48.515 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f34180a1590 msgr2=0x7f34180a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f34180a1590 0x7f34180a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f341400b2e0 tx=0x7f3414005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 msgr2=0x7f342c19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c19d210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f341c00e9f0 tx=0x7f341c00ed00 comp rx=0 tx=0).stop 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 shutdown_connections 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f34180a1590 0x7f34180a3a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f342c10d4b0 0x7f342c1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c073910 0x7f342c19d210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.516+0000 7f3430dff700 1 --2- 172.21.15.57:0/3419647292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f342c074ef0 0x7f342c19d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.517+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 >> 172.21.15.57:0/3419647292 conn(0x7f342c0fc430 msgr2=0x7f342c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:48.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.517+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 shutdown_connections 2023-10-06T22:13:48.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.517+0000 7f3430dff700 1 -- 172.21.15.57:0/3419647292 wait complete. 2023-10-06T22:13:48.537 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:13:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 -- 172.21.15.57:0/3771649984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 msgr2=0x7f7894103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 --2- 172.21.15.57:0/3771649984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f7894103d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7890009990 tx=0x7f7890009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 -- 172.21.15.57:0/3771649984 shutdown_connections 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 --2- 172.21.15.57:0/3771649984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7894106ba0 0x7f7894109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 --2- 172.21.15.57:0/3771649984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f7894103d30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 --2- 172.21.15.57:0/3771649984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7894104270 0x7f7894106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 -- 172.21.15.57:0/3771649984 >> 172.21.15.57:0/3771649984 conn(0x7f78940fb330 msgr2=0x7f78940fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 -- 172.21.15.57:0/3771649984 shutdown_connections 2023-10-06T22:13:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.752+0000 7f789b7ff700 1 -- 172.21.15.57:0/3771649984 wait complete. 2023-10-06T22:13:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.753+0000 7f789b7ff700 1 Processor -- start 2023-10-06T22:13:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789b7ff700 1 -- start start 2023-10-06T22:13:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f78941012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789b7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7894104270 0x7f78940ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789b7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7894106ba0 0x7f78940ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789b7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941005d0 con 0x7f7894106ba0 2023-10-06T22:13:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894100740 con 0x7f7894101940 2023-10-06T22:13:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789b7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941a0c40 con 0x7f7894104270 2023-10-06T22:13:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f78941012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f78941012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55692/0 (socket says 172.21.15.57:55692) 2023-10-06T22:13:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f789959b700 1 -- 172.21.15.57:0/2887915893 learned_addr learned my addr 172.21.15.57:0/2887915893 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f7899d9c700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7894106ba0 0x7f78940ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.754+0000 7f7898d9a700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7894104270 0x7f78940ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f789959b700 1 -- 172.21.15.57:0/2887915893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7894104270 msgr2=0x7f78940ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f789959b700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7894104270 0x7f78940ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f789959b700 1 -- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7894106ba0 msgr2=0x7f78940ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f789959b700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7894106ba0 0x7f78940ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f789959b700 1 -- 172.21.15.57:0/2887915893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c009c40 con 0x7f7894101940 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f789959b700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f78941012b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f789000b680 tx=0x7f7890005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f788a7fc700 1 -- 172.21.15.57:0/2887915893 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789001d070 con 0x7f7894101940 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78900095f0 con 0x7f7894101940 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f788a7fc700 1 -- 172.21.15.57:0/2887915893 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7890022470 con 0x7f7894101940 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.755+0000 7f788a7fc700 1 -- 172.21.15.57:0/2887915893 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890017650 con 0x7f7894101940 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.756+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78941a1160 con 0x7f7894101940 2023-10-06T22:13:48.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.757+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78940fceb0 con 0x7f7894101940 2023-10-06T22:13:48.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.760+0000 7f788a7fc700 1 -- 172.21.15.57:0/2887915893 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f78900225e0 con 0x7f7894101940 2023-10-06T22:13:48.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.761+0000 7f788a7fc700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f78800a1510 0x7f78800a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:48.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.761+0000 7f788a7fc700 1 -- 172.21.15.57:0/2887915893 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f78900ce390 con 0x7f7894101940 2023-10-06T22:13:48.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.761+0000 7f7898d9a700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f78800a1510 0x7f78800a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:48.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.762+0000 7f7898d9a700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f78800a1510 0x7f78800a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7884009c60 tx=0x7f7884009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:48.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.766+0000 7f788a7fc700 1 -- 172.21.15.57:0/2887915893 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f78900978f0 con 0x7f7894101940 2023-10-06T22:13:48.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.981+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f789410c8a0 con 0x7f7894101940 2023-10-06T22:13:48.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.982+0000 7f788a7fc700 1 -- 172.21.15.57:0/2887915893 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f789009af10 con 0x7f7894101940 2023-10-06T22:13:48.983 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f78800a1510 msgr2=0x7f78800a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f78800a1510 0x7f78800a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7884009c60 tx=0x7f7884009260 comp rx=0 tx=0).stop 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 msgr2=0x7f78941012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f78941012b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f789000b680 tx=0x7f7890005740 comp rx=0 tx=0).stop 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 shutdown_connections 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f78800a1510 0x7f78800a39d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7894106ba0 0x7f78940ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7894101940 0x7f78941012b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 --2- 172.21.15.57:0/2887915893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7894104270 0x7f78940ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 >> 172.21.15.57:0/2887915893 conn(0x7f78940fb330 msgr2=0x7f78941025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 shutdown_connections 2023-10-06T22:13:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:48.984+0000 7f789b7ff700 1 -- 172.21.15.57:0/2887915893 wait complete. 2023-10-06T22:13:49.005 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:13:49.068 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:48 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/122427857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:13:49.068 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:48 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3419647292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:13:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:48 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/122427857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:13:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:48 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3419647292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:13:49.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 -- 172.21.15.57:0/514530117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c104270 msgr2=0x7f229c106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 --2- 172.21.15.57:0/514530117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c104270 0x7f229c106660 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2290009990 tx=0x7f2290009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 -- 172.21.15.57:0/514530117 shutdown_connections 2023-10-06T22:13:49.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 --2- 172.21.15.57:0/514530117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f229c106ba0 0x7f229c109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 --2- 172.21.15.57:0/514530117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f229c101940 0x7f229c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 --2- 172.21.15.57:0/514530117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c104270 0x7f229c106660 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 -- 172.21.15.57:0/514530117 >> 172.21.15.57:0/514530117 conn(0x7f229c0fb310 msgr2=0x7f229c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:49.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.212+0000 7f22a15ff700 1 -- 172.21.15.57:0/514530117 shutdown_connections 2023-10-06T22:13:49.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.213+0000 7f22a15ff700 1 -- 172.21.15.57:0/514530117 wait complete. 2023-10-06T22:13:49.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.214+0000 7f22a15ff700 1 Processor -- start 2023-10-06T22:13:49.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.214+0000 7f22a15ff700 1 -- start start 2023-10-06T22:13:49.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f22a15ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c101940 0x7f229c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f22a15ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f229c104270 0x7f229c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f22a15ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f229c106ba0 0x7f229c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f22a15ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229c19e700 con 0x7f229c101940 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f22a15ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229c06a8d0 con 0x7f229c106ba0 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f22a15ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229c06aa60 con 0x7f229c104270 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f229affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c101940 0x7f229c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f229affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c101940 0x7f229c196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42998/0 (socket says 172.21.15.57:42998) 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f229affd700 1 -- 172.21.15.57:0/1002194054 learned_addr learned my addr 172.21.15.57:0/1002194054 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.215+0000 7f229b7fe700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f229c106ba0 0x7f229c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229a7fc700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f229c104270 0x7f229c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229affd700 1 -- 172.21.15.57:0/1002194054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f229c104270 msgr2=0x7f229c1970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229affd700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f229c104270 0x7f229c1970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229affd700 1 -- 172.21.15.57:0/1002194054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f229c106ba0 msgr2=0x7f229c19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229affd700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f229c106ba0 0x7f229c19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229affd700 1 -- 172.21.15.57:0/1002194054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900095f0 con 0x7f229c101940 2023-10-06T22:13:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229a7fc700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f229c104270 0x7f229c1970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:13:49.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f229affd700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c101940 0x7f229c196b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f228c00e9a0 tx=0x7f228c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:49.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f2283fff700 1 -- 172.21.15.57:0/1002194054 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c009790 con 0x7f229c101940 2023-10-06T22:13:49.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f2283fff700 1 -- 172.21.15.57:0/1002194054 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f228c00cbd0 con 0x7f229c101940 2023-10-06T22:13:49.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f229c06ad40 con 0x7f229c101940 2023-10-06T22:13:49.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.216+0000 7f2283fff700 1 -- 172.21.15.57:0/1002194054 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c018680 con 0x7f229c101940 2023-10-06T22:13:49.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.217+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229c06b2a0 con 0x7f229c101940 2023-10-06T22:13:49.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.218+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229c10c700 con 0x7f229c101940 2023-10-06T22:13:49.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.220+0000 7f2283fff700 1 -- 172.21.15.57:0/1002194054 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f228c00cd40 con 0x7f229c101940 2023-10-06T22:13:49.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.222+0000 7f2283fff700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22840a15d0 0x7f22840a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.223+0000 7f2283fff700 1 -- 172.21.15.57:0/1002194054 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f228c014070 con 0x7f229c101940 2023-10-06T22:13:49.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.223+0000 7f229a7fc700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22840a15d0 0x7f22840a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.223+0000 7f229a7fc700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22840a15d0 0x7f22840a3a90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2290006010 tx=0x7f2290005c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:49.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.225+0000 7f2283fff700 1 -- 172.21.15.57:0/1002194054 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f228c052100 con 0x7f229c101940 2023-10-06T22:13:49.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:48 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/122427857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:13:49.260 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:48 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3419647292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:13:49.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.497+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f229c06b830 con 0x7f229c101940 2023-10-06T22:13:49.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.498+0000 7f2283fff700 1 -- 172.21.15.57:0/1002194054 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f228c094e60 con 0x7f229c101940 2023-10-06T22:13:49.513 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:49.513 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:13:49.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22840a15d0 msgr2=0x7f22840a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22840a15d0 0x7f22840a3a90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2290006010 tx=0x7f2290005c20 comp rx=0 tx=0).stop 2023-10-06T22:13:49.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c101940 msgr2=0x7f229c196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c101940 0x7f229c196b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f228c00e9a0 tx=0x7f228c00ecb0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 shutdown_connections 2023-10-06T22:13:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22840a15d0 0x7f22840a3a90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f229c106ba0 0x7f229c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f229c101940 0x7f229c196b80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 --2- 172.21.15.57:0/1002194054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f229c104270 0x7f229c1970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 >> 172.21.15.57:0/1002194054 conn(0x7f229c0fb310 msgr2=0x7f229c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 shutdown_connections 2023-10-06T22:13:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.501+0000 7f22a15ff700 1 -- 172.21.15.57:0/1002194054 wait complete. 2023-10-06T22:13:49.516 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:13:49.521 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:13:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 -- 172.21.15.57:0/2558311821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc1019a0 msgr2=0x7f69cc103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 --2- 172.21.15.57:0/2558311821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc1019a0 0x7f69cc103d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009990 tx=0x7f69bc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 -- 172.21.15.57:0/2558311821 shutdown_connections 2023-10-06T22:13:49.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 --2- 172.21.15.57:0/2558311821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc106c00 0x7f69cc109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 --2- 172.21.15.57:0/2558311821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc1019a0 0x7f69cc103d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 --2- 172.21.15.57:0/2558311821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69cc1042d0 0x7f69cc1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 -- 172.21.15.57:0/2558311821 >> 172.21.15.57:0/2558311821 conn(0x7f69cc0fb3b0 msgr2=0x7f69cc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:49.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 -- 172.21.15.57:0/2558311821 shutdown_connections 2023-10-06T22:13:49.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.752+0000 7f69d19ff700 1 -- 172.21.15.57:0/2558311821 wait complete. 2023-10-06T22:13:49.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.754+0000 7f69d19ff700 1 Processor -- start 2023-10-06T22:13:49.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.754+0000 7f69d19ff700 1 -- start start 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.754+0000 7f69d19ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc1019a0 0x7f69cc101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.754+0000 7f69d19ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69cc1042d0 0x7f69cc0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69d19ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc106c00 0x7f69cc0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69d19ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc100630 con 0x7f69cc1042d0 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69d19ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1007a0 con 0x7f69cc106c00 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69d19ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1a0b90 con 0x7f69cc1019a0 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc1019a0 0x7f69cc101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc1019a0 0x7f69cc101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56848/0 (socket says 172.21.15.57:56848) 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 -- 172.21.15.57:0/581659976 learned_addr learned my addr 172.21.15.57:0/581659976 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:49.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69ca7fc700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69cc1042d0 0x7f69cc0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69cb7fe700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc106c00 0x7f69cc0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 -- 172.21.15.57:0/581659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc106c00 msgr2=0x7f69cc0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc106c00 0x7f69cc0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 -- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69cc1042d0 msgr2=0x7f69cc0ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69cc1042d0 0x7f69cc0ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 -- 172.21.15.57:0/581659976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc0095f0 con 0x7f69cc1019a0 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.755+0000 7f69caffd700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc1019a0 0x7f69cc101310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b680 tx=0x7f69bc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.756+0000 7f69d09fd700 1 -- 172.21.15.57:0/581659976 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc01d070 con 0x7f69cc1019a0 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.756+0000 7f69d09fd700 1 -- 172.21.15.57:0/581659976 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69bc022470 con 0x7f69cc1019a0 2023-10-06T22:13:49.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.756+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc1a0e10 con 0x7f69cc1019a0 2023-10-06T22:13:49.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.756+0000 7f69d09fd700 1 -- 172.21.15.57:0/581659976 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc017650 con 0x7f69cc1019a0 2023-10-06T22:13:49.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.756+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc1a1180 con 0x7f69cc1019a0 2023-10-06T22:13:49.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.757+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc0fcf30 con 0x7f69cc1019a0 2023-10-06T22:13:49.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.761+0000 7f69d09fd700 1 -- 172.21.15.57:0/581659976 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f69bc0225e0 con 0x7f69cc1019a0 2023-10-06T22:13:49.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.762+0000 7f69d09fd700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f69b40a1550 0x7f69b40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:49.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.762+0000 7f69d09fd700 1 -- 172.21.15.57:0/581659976 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f69bc0ce300 con 0x7f69cc1019a0 2023-10-06T22:13:49.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.762+0000 7f69ca7fc700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f69b40a1550 0x7f69b40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:49.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.763+0000 7f69ca7fc700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f69b40a1550 0x7f69b40a3a10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009c60 tx=0x7f69c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:49.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:49.766+0000 7f69d09fd700 1 -- 172.21.15.57:0/581659976 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f69bc0978c0 con 0x7f69cc1019a0 2023-10-06T22:13:50.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.005+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69cc10c970 con 0x7f69cc1019a0 2023-10-06T22:13:50.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.007+0000 7f69d09fd700 1 -- 172.21.15.57:0/581659976 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7f69bc09aee0 con 0x7f69cc1019a0 2023-10-06T22:13:50.008 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:50.008 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f69b40a1550 msgr2=0x7f69b40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f69b40a1550 0x7f69b40a3a10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009c60 tx=0x7f69c0009260 comp rx=0 tx=0).stop 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc1019a0 msgr2=0x7f69cc101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc1019a0 0x7f69cc101310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b680 tx=0x7f69bc005740 comp rx=0 tx=0).stop 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 shutdown_connections 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f69b40a1550 0x7f69b40a3a10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69cc106c00 0x7f69cc0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69cc1019a0 0x7f69cc101310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.009+0000 7f69d19ff700 1 --2- 172.21.15.57:0/581659976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69cc1042d0 0x7f69cc0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.010+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 >> 172.21.15.57:0/581659976 conn(0x7f69cc0fb3b0 msgr2=0x7f69cc102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.010+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 shutdown_connections 2023-10-06T22:13:50.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.010+0000 7f69d19ff700 1 -- 172.21.15.57:0/581659976 wait complete. 2023-10-06T22:13:50.031 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:13:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:49 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2887915893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:13:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:49 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1002194054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:49 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:49 smithi190 ceph-mon[134891]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:50.097 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:49 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2887915893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:13:50.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:49 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1002194054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:50.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:49 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:49 smithi057 ceph-mon[98492]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.241+0000 7fa0771ff700 1 -- 172.21.15.57:0/1883078821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 msgr2=0x7fa070073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.241+0000 7fa0771ff700 1 --2- 172.21.15.57:0/1883078821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 0x7fa070073340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa064009990 tx=0x7fa064009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.241+0000 7fa0771ff700 1 -- 172.21.15.57:0/1883078821 shutdown_connections 2023-10-06T22:13:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.241+0000 7fa0771ff700 1 --2- 172.21.15.57:0/1883078821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070108db0 0x7fa07010b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.241+0000 7fa0771ff700 1 --2- 172.21.15.57:0/1883078821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 0x7fa070073340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.241+0000 7fa0771ff700 1 --2- 172.21.15.57:0/1883078821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa070073910 0x7fa070073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.241+0000 7fa0771ff700 1 -- 172.21.15.57:0/1883078821 >> 172.21.15.57:0/1883078821 conn(0x7fa0700fc420 msgr2=0x7fa0700fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.242+0000 7fa0771ff700 1 -- 172.21.15.57:0/1883078821 shutdown_connections 2023-10-06T22:13:50.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.242+0000 7fa0771ff700 1 -- 172.21.15.57:0/1883078821 wait complete. 2023-10-06T22:13:50.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.242+0000 7fa0771ff700 1 Processor -- start 2023-10-06T22:13:50.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa0771ff700 1 -- start start 2023-10-06T22:13:50.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa0771ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa070073910 0x7fa070198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa0771ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 0x7fa070199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa0771ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070108db0 0x7fa0701a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa0771ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701a0970 con 0x7fa070074ef0 2023-10-06T22:13:50.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa0771ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701a0ab0 con 0x7fa070073910 2023-10-06T22:13:50.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa0771ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701a0c40 con 0x7fa070108db0 2023-10-06T22:13:50.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa06ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 0x7fa070199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa06ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 0x7fa070199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43034/0 (socket says 172.21.15.57:43034) 2023-10-06T22:13:50.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa07579c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070108db0 0x7fa0701a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa06ffff700 1 -- 172.21.15.57:0/2085445106 learned_addr learned my addr 172.21.15.57:0/2085445106 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:50.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.243+0000 7fa074f9b700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa070073910 0x7fa070198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa07579c700 1 -- 172.21.15.57:0/2085445106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa070073910 msgr2=0x7fa070198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa07579c700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa070073910 0x7fa070198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa07579c700 1 -- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 msgr2=0x7fa070199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa07579c700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 0x7fa070199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa07579c700 1 -- 172.21.15.57:0/2085445106 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0640095f0 con 0x7fa070108db0 2023-10-06T22:13:50.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa07579c700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070108db0 0x7fa0701a03a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa06dffb700 1 -- 172.21.15.57:0/2085445106 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068018070 con 0x7fa070108db0 2023-10-06T22:13:50.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa06dffb700 1 -- 172.21.15.57:0/2085445106 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa06800f040 con 0x7fa070108db0 2023-10-06T22:13:50.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa06dffb700 1 -- 172.21.15.57:0/2085445106 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800cd40 con 0x7fa070108db0 2023-10-06T22:13:50.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0701a0ec0 con 0x7fa070108db0 2023-10-06T22:13:50.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.244+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0701a12f0 con 0x7fa070108db0 2023-10-06T22:13:50.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.246+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa070107460 con 0x7fa070108db0 2023-10-06T22:13:50.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.247+0000 7fa06dffb700 1 -- 172.21.15.57:0/2085445106 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa068003a00 con 0x7fa070108db0 2023-10-06T22:13:50.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.249+0000 7fa06dffb700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa0600a1550 0x7fa0600a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.249+0000 7fa06dffb700 1 -- 172.21.15.57:0/2085445106 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fa0680cca70 con 0x7fa070108db0 2023-10-06T22:13:50.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.249+0000 7fa074f9b700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa0600a1550 0x7fa0600a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.249+0000 7fa074f9b700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa0600a1550 0x7fa0600a3a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b2e0 tx=0x7fa064005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:49 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2887915893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:13:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:49 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1002194054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:49 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:49 smithi156 ceph-mon[134358]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:13:50.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.252+0000 7fa06dffb700 1 -- 172.21.15.57:0/2085445106 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa068095fd0 con 0x7fa070108db0 2023-10-06T22:13:50.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.516+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa07002cf70 con 0x7fa070108db0 2023-10-06T22:13:50.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.517+0000 7fa06dffb700 1 -- 172.21.15.57:0/2085445106 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fa0680995f0 con 0x7fa070108db0 2023-10-06T22:13:50.518 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:13:50.518 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:13:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa0600a1550 msgr2=0x7fa0600a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa0600a1550 0x7fa0600a3a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b2e0 tx=0x7fa064005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070108db0 msgr2=0x7fa0701a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070108db0 0x7fa0701a03a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 shutdown_connections 2023-10-06T22:13:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa0600a1550 0x7fa0600a3a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070108db0 0x7fa0701a03a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa070073910 0x7fa070198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 --2- 172.21.15.57:0/2085445106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070074ef0 0x7fa070199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 >> 172.21.15.57:0/2085445106 conn(0x7fa0700fc420 msgr2=0x7fa0700fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 shutdown_connections 2023-10-06T22:13:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.520+0000 7fa0771ff700 1 -- 172.21.15.57:0/2085445106 wait complete. 2023-10-06T22:13:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:13:50.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-06T22:12:43.212955+0000', 'modified': '2023-10-06T22:12:50.650753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24553, 'mds_1': 24551, 'mds_2': 24529, 'mds_3': 24517, 'mds_4': 24509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14523': {'gid': 14523, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6839/3033019918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6838', 'nonce': 3033019918}, {'type': 'v1', 'addr': '172.21.15.57:6839', 'nonce': 3033019918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14535': {'gid': 14535, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6841/490100321', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6840', 'nonce': 490100321}, {'type': 'v1', 'addr': '172.21.15.57:6841', 'nonce': 490100321}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6833/1142570288', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6832', 'nonce': 1142570288}, {'type': 'v1', 'addr': '172.21.15.156:6833', 'nonce': 1142570288}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/1813464152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 1813464152}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 1813464152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2797676784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2797676784}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2797676784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6837/3773867071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3773867071}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3773867071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.190:6835/871306260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 871306260}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 871306260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.190:6837/3737066182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3737066182}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3737066182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.156:6839/1657784543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1657784543}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1657784543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6839/751138264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 751138264}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 751138264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-06T22:13:50.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-06T22:13:50.542 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-06T22:13:50.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 -- 172.21.15.57:0/1572407045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 msgr2=0x7ff134103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/1572407045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134103df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff130009990 tx=0x7ff130009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 -- 172.21.15.57:0/1572407045 shutdown_connections 2023-10-06T22:13:50.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/1572407045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff134106c60 0x7ff134109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/1572407045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134103df0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/1572407045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff134104330 0x7ff134106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 -- 172.21.15.57:0/1572407045 >> 172.21.15.57:0/1572407045 conn(0x7ff1340fb330 msgr2=0x7ff1340fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.765+0000 7ff13b5ff700 1 -- 172.21.15.57:0/1572407045 shutdown_connections 2023-10-06T22:13:50.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.766+0000 7ff13b5ff700 1 -- 172.21.15.57:0/1572407045 wait complete. 2023-10-06T22:13:50.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.767+0000 7ff13b5ff700 1 Processor -- start 2023-10-06T22:13:50.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.767+0000 7ff13b5ff700 1 -- start start 2023-10-06T22:13:50.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13b5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff134104330 0x7ff1340ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13b5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff134106c60 0x7ff1340ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13b5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff134100690 con 0x7ff134104330 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff134100800 con 0x7ff134101a00 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13b5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1341a0c70 con 0x7ff134106c60 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55758/0 (socket says 172.21.15.57:55758) 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 -- 172.21.15.57:0/2894280679 learned_addr learned my addr 172.21.15.57:0/2894280679 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff138b9a700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff134104330 0x7ff1340ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff139b9c700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff134106c60 0x7ff1340ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 -- 172.21.15.57:0/2894280679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff134106c60 msgr2=0x7ff1340ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff134106c60 0x7ff1340ffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 -- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff134104330 msgr2=0x7ff1340ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff134104330 0x7ff1340ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.768+0000 7ff13939b700 1 -- 172.21.15.57:0/2894280679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300095f0 con 0x7ff134101a00 2023-10-06T22:13:50.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.769+0000 7ff13939b700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134101370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff130004740 tx=0x7ff130009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.769+0000 7ff12a7fc700 1 -- 172.21.15.57:0/2894280679 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13001d070 con 0x7ff134101a00 2023-10-06T22:13:50.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.769+0000 7ff12a7fc700 1 -- 172.21.15.57:0/2894280679 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff130004830 con 0x7ff134101a00 2023-10-06T22:13:50.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.769+0000 7ff12a7fc700 1 -- 172.21.15.57:0/2894280679 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1300053c0 con 0x7ff134101a00 2023-10-06T22:13:50.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.769+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1341a0ef0 con 0x7ff134101a00 2023-10-06T22:13:50.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.769+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1341a1260 con 0x7ff134101a00 2023-10-06T22:13:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.774+0000 7ff12a7fc700 1 -- 172.21.15.57:0/2894280679 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff130005580 con 0x7ff134101a00 2023-10-06T22:13:50.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 Processor -- start 2023-10-06T22:13:50.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 -- start start 2023-10-06T22:13:50.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b04630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04b70 0x55ea34b05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b05e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34a6c410 con 0x55ea34b05a10 2023-10-06T22:13:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34a5dbd0 con 0x55ea34b04210 2023-10-06T22:13:50.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.775+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34a5b4f0 con 0x55ea34b04b70 2023-10-06T22:13:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff139b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b04630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff13939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04b70 0x55ea34b05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff138b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b05e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff139b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b04630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55772/0 (socket says 172.21.15.57:55772) 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff139b9c700 1 -- 172.21.15.57:0/3848348864 learned_addr learned my addr 172.21.15.57:0/3848348864 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff12a7fc700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff1200a1510 0x7ff1200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff12a7fc700 1 -- 172.21.15.57:0/2894280679 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7ff1300ce070 con 0x7ff134101a00 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff138b9a700 1 -- 172.21.15.57:0/3848348864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04b70 msgr2=0x55ea34b05430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff138b9a700 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04b70 0x55ea34b05430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff138b9a700 1 -- 172.21.15.57:0/3848348864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 msgr2=0x55ea34b04630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff138b9a700 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b04630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff138b9a700 1 -- 172.21.15.57:0/3848348864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea34881d90 con 0x55ea34b05a10 2023-10-06T22:13:50.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff139b9c700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff1200a1510 0x7ff1200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff13939b700 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04b70 0x55ea34b05430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.776+0000 7ff138b9a700 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b05e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff124009cb0 tx=0x7ff124011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cae084a78ab78b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.777+0000 7ff139b9c700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff1200a1510 0x7ff1200a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0060b0 tx=0x7ff12c004b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.777+0000 7ff11f7fe700 1 -- 172.21.15.57:0/3848348864 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124011d20 con 0x55ea34b05a10 2023-10-06T22:13:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.777+0000 7ff11f7fe700 1 -- 172.21.15.57:0/3848348864 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff124011e80 con 0x55ea34b05a10 2023-10-06T22:13:50.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.777+0000 7ff11f7fe700 1 -- 172.21.15.57:0/3848348864 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124005280 con 0x55ea34b05a10 2023-10-06T22:13:50.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 -- 172.21.15.57:0/3848348864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 msgr2=0x55ea34b05e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b05e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff124009cb0 tx=0x7ff124011440 comp rx=0 tx=0).stop 2023-10-06T22:13:50.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 -- 172.21.15.57:0/3848348864 shutdown_connections 2023-10-06T22:13:50.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b05e90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b04630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/3848348864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04b70 0x55ea34b05430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 -- 172.21.15.57:0/3848348864 >> 172.21.15.57:0/3848348864 conn(0x55ea34a78210 msgr2=0x55ea34aff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 -- 172.21.15.57:0/3848348864 shutdown_connections 2023-10-06T22:13:50.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 -- 172.21.15.57:0/3848348864 wait complete. 2023-10-06T22:13:50.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.778+0000 7ff14eee3b80 1 -- start start 2023-10-06T22:13:50.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 1 Processor -- start 2023-10-06T22:13:50.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b0dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04b70 0x55ea34b0e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b15530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34a56de0 con 0x55ea34b05a10 2023-10-06T22:13:50.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea349f36c0 con 0x55ea34b04b70 2023-10-06T22:13:50.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea349f3360 con 0x55ea34b04210 2023-10-06T22:13:50.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff14eee3b80 10 client.0 _ll_register_callbacks cb 0x55ea348507b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:50.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff13939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b0dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff13939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b0dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56894/0 (socket says 172.21.15.57:56894) 2023-10-06T22:13:50.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff13939b700 1 -- 172.21.15.57:0/503254727 learned_addr learned my addr 172.21.15.57:0/503254727 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:50.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff139b9c700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b15530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.779+0000 7ff138b9a700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04b70 0x55ea34b0e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff13939b700 1 -- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04b70 msgr2=0x55ea34b0e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff13939b700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04b70 0x55ea34b0e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff13939b700 1 -- 172.21.15.57:0/503254727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 msgr2=0x55ea34b15530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff13939b700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b15530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff13939b700 1 -- 172.21.15.57:0/503254727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1341a1260 con 0x55ea34b04210 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff138b9a700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04b70 0x55ea34b0e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff13939b700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b0dfb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff13000b2e0 tx=0x7ff13009abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2ef26f17a285d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff11cff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff11cff9700 1 -- 172.21.15.57:0/503254727 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130095ea0 con 0x55ea34b04210 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff11cff9700 1 -- 172.21.15.57:0/503254727 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7ff13009b990 con 0x55ea34b04210 2023-10-06T22:13:50.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea34a4fdc0 con 0x55ea34b04210 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff11cff9700 1 -- 172.21.15.57:0/503254727 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff13009b290 con 0x55ea34b04210 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff14eee3b80 10 client.24973 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.780+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff1240098e0 con 0x55ea34b04210 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff11cff9700 1 -- 172.21.15.57:0/503254727 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea34a4fdc0 con 0x55ea34b04210 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff11cff9700 1 -- 172.21.15.57:0/503254727 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff13009a780 con 0x55ea34b04210 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff14eee3b80 10 client.24973 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff14eee3b80 10 client.24973 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff14eee3b80 20 client.24973 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b19800 0x55ea34b18460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff14eee3b80 4 client.24973 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:13:50.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea34a4fae0 con 0x55ea34b19800 2023-10-06T22:13:50.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.781+0000 7ff138b9a700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b19800 0x55ea34b18460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.782+0000 7ff138b9a700 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b19800 0x55ea34b18460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b727bb7a4675968 server_cookie=487d9b89bae7a333 in_seq=0 out_seq=0 2023-10-06T22:13:50.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.782+0000 7ff11cff9700 10 client.24973 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:13:50.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff11cff9700 1 -- 172.21.15.57:0/503254727 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea34a4fae0 con 0x55ea34b19800 2023-10-06T22:13:50.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff11cff9700 10 client.24973 handle_command_reply: tid=0 2023-10-06T22:13:50.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 client.24973 shutdown 2023-10-06T22:13:50.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 msgr2=0x55ea34b0dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b0dfb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff13000b2e0 tx=0x7ff13009abf0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 shutdown_connections 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b19800 0x55ea34b18460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b05a10 0x55ea34b15530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff11cff9700 0 client.24973 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b0dfb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/503254727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04b70 0x55ea34b0e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 >> 172.21.15.57:0/503254727 conn(0x55ea34a78210 msgr2=0x55ea34afef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.783+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 shutdown_connections 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.784+0000 7ff14eee3b80 1 -- 172.21.15.57:0/503254727 wait complete. 2023-10-06T22:13:50.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.784+0000 7ff14eee3b80 20 client.24973 trim_cache size 0 max 16384 2023-10-06T22:13:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 Processor -- start 2023-10-06T22:13:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 -- start start 2023-10-06T22:13:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b0cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b05a10 0x55ea34b08980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea348d78d0 con 0x55ea34b04b70 2023-10-06T22:13:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34a4fdc0 con 0x55ea34b04210 2023-10-06T22:13:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.803+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34a58070 con 0x55ea34b05a10 2023-10-06T22:13:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff138b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff138b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43092/0 (socket says 172.21.15.57:43092) 2023-10-06T22:13:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff13939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b0cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff138b9a700 1 -- 172.21.15.57:0/2878831948 learned_addr learned my addr 172.21.15.57:0/2878831948 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff139b9c700 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b05a10 0x55ea34b08980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff13939b700 1 -- 172.21.15.57:0/2878831948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b05a10 msgr2=0x55ea34b08980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff13939b700 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b05a10 0x55ea34b08980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff13939b700 1 -- 172.21.15.57:0/2878831948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 msgr2=0x55ea34b154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff13939b700 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff13939b700 1 -- 172.21.15.57:0/2878831948 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3486ac10 con 0x55ea34b04210 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff138b9a700 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff139b9c700 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b05a10 0x55ea34b08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.804+0000 7ff13939b700 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b0cc70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff13009ab00 tx=0x7ff1300d16b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b8a290a28f1a108 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff11ffff700 1 -- 172.21.15.57:0/2878831948 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130095ea0 con 0x55ea34b04210 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff11ffff700 1 -- 172.21.15.57:0/2878831948 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1300d0300 con 0x55ea34b04210 2023-10-06T22:13:50.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff11ffff700 1 -- 172.21.15.57:0/2878831948 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13009f3e0 con 0x55ea34b04210 2023-10-06T22:13:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 -- 172.21.15.57:0/2878831948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 msgr2=0x55ea34b0cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b0cc70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff13009ab00 tx=0x7ff1300d16b0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 -- 172.21.15.57:0/2878831948 shutdown_connections 2023-10-06T22:13:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b05a10 0x55ea34b08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b04210 0x55ea34b0cc70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/2878831948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 -- 172.21.15.57:0/2878831948 >> 172.21.15.57:0/2878831948 conn(0x55ea34a78210 msgr2=0x55ea34b22160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 -- 172.21.15.57:0/2878831948 shutdown_connections 2023-10-06T22:13:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.805+0000 7ff14eee3b80 1 -- 172.21.15.57:0/2878831948 wait complete. 2023-10-06T22:13:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.806+0000 7ff14eee3b80 1 -- start start 2023-10-06T22:13:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.806+0000 7ff14eee3b80 1 Processor -- start 2023-10-06T22:13:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b11720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b11d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff14eee3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b05a10 0x55ea34b14a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34b14f80 con 0x55ea34b04b70 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34b150f0 con 0x55ea34b05a10 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff14eee3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34b152b0 con 0x55ea34b04210 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff14eee3b80 10 client.0 _ll_register_callbacks cb 0x55ea348507b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b11720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff139b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b05a10 0x55ea34b14a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b11720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56920/0 (socket says 172.21.15.57:56920) 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 -- 172.21.15.57:0/1222031081 learned_addr learned my addr 172.21.15.57:0/1222031081 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff138b9a700 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b11d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 -- 172.21.15.57:0/1222031081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b05a10 msgr2=0x55ea34b14a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b05a10 0x55ea34b14a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 -- 172.21.15.57:0/1222031081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 msgr2=0x55ea34b11d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b11d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 -- 172.21.15.57:0/1222031081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12401d040 con 0x55ea34b04210 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff13939b700 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b11720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1300995e0 tx=0x7ff1300d1520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933ad6aaaaddaf59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.807+0000 7ff11cff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff11cff9700 1 -- 172.21.15.57:0/1222031081 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130095ea0 con 0x55ea34b04210 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff11cff9700 1 -- 172.21.15.57:0/1222031081 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7ff1300a3590 con 0x55ea34b04210 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea348a2f20 con 0x55ea34b04210 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff11cff9700 1 -- 172.21.15.57:0/1222031081 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff13009b060 con 0x55ea34b04210 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 10 client.24979 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:50.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff12c0090f0 con 0x55ea34b04210 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff11cff9700 1 -- 172.21.15.57:0/1222031081 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea348a2f20 con 0x55ea34b04210 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff11cff9700 1 -- 172.21.15.57:0/1222031081 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff130097080 con 0x55ea34b04210 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 10 client.24979 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 10 client.24979 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 20 client.24979 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b71770 0x55ea34b6dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 4 client.24979 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.808+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ea348d7f10 con 0x55ea34b71770 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.809+0000 7ff138b9a700 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b71770 0x55ea34b6dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.809+0000 7ff11cff9700 10 client.24979 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:13:50.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.809+0000 7ff138b9a700 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b71770 0x55ea34b6dc70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec523743e20f8a33 server_cookie=686527c8f09b0e6e in_seq=0 out_seq=0 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff11cff9700 1 -- 172.21.15.57:0/1222031081 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55ea348d7f10 con 0x55ea34b71770 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff11cff9700 10 client.24979 handle_command_reply: tid=0 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 client.24979 shutdown 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 msgr2=0x55ea34b11720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b11720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1300995e0 tx=0x7ff1300d1520 comp rx=0 tx=0).stop 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 shutdown_connections 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea34b71770 0x55ea34b6dc70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea34b05a10 0x55ea34b14a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea34b04210 0x55ea34b11720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 --2- 172.21.15.57:0/1222031081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea34b04b70 0x55ea34b11d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff11cff9700 0 client.24979 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:13:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 >> 172.21.15.57:0/1222031081 conn(0x55ea34a78210 msgr2=0x55ea34b0be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 shutdown_connections 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 1 -- 172.21.15.57:0/1222031081 wait complete. 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.810+0000 7ff14eee3b80 20 client.24979 trim_cache size 0 max 16384 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 4, 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24509, 2023-10-06T22:13:50.818 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 61.042199601, 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 83.526199465999994 2023-10-06T22:13:50.819 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:13:50.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff1200a1510 msgr2=0x7ff1200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff1200a1510 0x7ff1200a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0060b0 tx=0x7ff12c004b40 comp rx=0 tx=0).stop 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 msgr2=0x7ff134101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134101370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff130004740 tx=0x7ff130009580 comp rx=0 tx=0).stop 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 shutdown_connections 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7ff1200a1510 0x7ff1200a39d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff134106c60 0x7ff1340ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff134101a00 0x7ff134101370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 --2- 172.21.15.57:0/2894280679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff134104330 0x7ff1340ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 >> 172.21.15.57:0/2894280679 conn(0x7ff1340fb330 msgr2=0x7ff134102670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:50.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 shutdown_connections 2023-10-06T22:13:50.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:50.811+0000 7ff13b5ff700 1 -- 172.21.15.57:0/2894280679 wait complete. 2023-10-06T22:13:50.831 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-06T22:13:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.035+0000 7f1fd5fff700 1 -- 172.21.15.57:0/1352546109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0104270 msgr2=0x7f1fd0106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.035+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/1352546109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0104270 0x7f1fd0106660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009990 tx=0x7f1fc4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.035+0000 7f1fd5fff700 1 -- 172.21.15.57:0/1352546109 shutdown_connections 2023-10-06T22:13:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.035+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/1352546109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd0106ba0 0x7f1fd0109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.035+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/1352546109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fd0101940 0x7f1fd0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.035+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/1352546109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0104270 0x7f1fd0106660 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.035+0000 7f1fd5fff700 1 -- 172.21.15.57:0/1352546109 >> 172.21.15.57:0/1352546109 conn(0x7f1fd00fb330 msgr2=0x7f1fd00fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.036+0000 7f1fd5fff700 1 -- 172.21.15.57:0/1352546109 shutdown_connections 2023-10-06T22:13:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.036+0000 7f1fd5fff700 1 -- 172.21.15.57:0/1352546109 wait complete. 2023-10-06T22:13:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.037+0000 7f1fd5fff700 1 Processor -- start 2023-10-06T22:13:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.037+0000 7f1fd5fff700 1 -- start start 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fd5fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0101940 0x7f1fd01012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fd5fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd0104270 0x7f1fd00ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fd5fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fd0106ba0 0x7f1fd00ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fd5fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd01005d0 con 0x7f1fd0101940 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fd5fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd0100740 con 0x7f1fd0104270 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fd5fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd01a0c60 con 0x7f1fd0106ba0 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0101940 0x7f1fd01012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0101940 0x7f1fd01012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43118/0 (socket says 172.21.15.57:43118) 2023-10-06T22:13:51.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/349878757 learned_addr learned my addr 172.21.15.57:0/349878757 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fcffff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fd0106ba0 0x7f1fd00ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.038+0000 7f1fceffd700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd0104270 0x7f1fd00ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/349878757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fd0106ba0 msgr2=0x7f1fd00ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fd0106ba0 0x7f1fd00ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/349878757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd0104270 msgr2=0x7f1fd00ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd0104270 0x7f1fd00ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/349878757 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc0009520 con 0x7f1fd0101940 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fceffd700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd0104270 0x7f1fd00ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:51.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0101940 0x7f1fd01012b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000eb30 tx=0x7f1fc000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fccff9700 1 -- 172.21.15.57:0/349878757 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0009880 con 0x7f1fd0101940 2023-10-06T22:13:51.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.039+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc40095f0 con 0x7f1fd0101940 2023-10-06T22:13:51.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.040+0000 7f1fccff9700 1 -- 172.21.15.57:0/349878757 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc00044e0 con 0x7f1fd0101940 2023-10-06T22:13:51.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.040+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd01a1220 con 0x7f1fd0101940 2023-10-06T22:13:51.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.040+0000 7f1fccff9700 1 -- 172.21.15.57:0/349878757 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0003e90 con 0x7f1fd0101940 2023-10-06T22:13:51.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.044+0000 7f1fccff9700 1 -- 172.21.15.57:0/349878757 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1fc0004020 con 0x7f1fd0101940 2023-10-06T22:13:51.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.045+0000 7f1fe99f8b80 1 Processor -- start 2023-10-06T22:13:51.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fe99f8b80 1 -- start start 2023-10-06T22:13:51.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60733c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a60733d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60733de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6072a4410 con 0x55a60733c210 2023-10-06T22:13:51.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a607295bd0 con 0x55a60733cb70 2023-10-06T22:13:51.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6072934f0 con 0x55a60733da10 2023-10-06T22:13:51.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fccff9700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1fb80a1590 0x7f1fb80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fceffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60733de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fccff9700 1 -- 172.21.15.57:0/349878757 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1fc0014070 con 0x7f1fd0101940 2023-10-06T22:13:51.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fcffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60733c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fcf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a60733d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fceffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60733de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56954/0 (socket says 172.21.15.57:56954) 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.046+0000 7f1fceffd700 1 -- 172.21.15.57:0/3531877543 learned_addr learned my addr 172.21.15.57:0/3531877543 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcffff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1fb80a1590 0x7f1fb80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/3531877543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 msgr2=0x55a60733de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60733de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/3531877543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 msgr2=0x55a60733c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60733c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/3531877543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6070b9d90 con 0x55a60733cb70 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fceffd700 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60733de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a60733d430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00cabf0 tx=0x7f1fc0099e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72c495d5c5b7abb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.047+0000 7f1fcffff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1fb80a1590 0x7f1fb80a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009ec0 tx=0x7f1fc800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fb5ffb700 1 -- 172.21.15.57:0/3531877543 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc009a400 con 0x55a60733cb70 2023-10-06T22:13:51.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fb5ffb700 1 -- 172.21.15.57:0/3531877543 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc009e550 con 0x55a60733cb70 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fb5ffb700 1 -- 172.21.15.57:0/3531877543 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00cae40 con 0x55a60733cb70 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3531877543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 msgr2=0x55a60733d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a60733d430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00cabf0 tx=0x7f1fc0099e30 comp rx=0 tx=0).stop 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3531877543 shutdown_connections 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60733de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60733c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3531877543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a60733d430 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.048+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3531877543 >> 172.21.15.57:0/3531877543 conn(0x55a6072b0210 msgr2=0x55a6073370f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.049+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3531877543 shutdown_connections 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.049+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3531877543 wait complete. 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.049+0000 7f1fe99f8b80 1 -- start start 2023-10-06T22:13:51.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.049+0000 7f1fe99f8b80 1 Processor -- start 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a607346040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733cb70 0x55a607346580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a60734d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60728ede0 con 0x55a60733c210 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60722b6c0 con 0x55a60733da10 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60722b360 con 0x55a60733cb70 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fe99f8b80 10 client.0 _ll_register_callbacks cb 0x55a6070887b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a607346040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a607346040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43130/0 (socket says 172.21.15.57:43130) 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/1593931450 learned_addr learned my addr 172.21.15.57:0/1593931450 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fceffd700 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733cb70 0x55a607346580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcffff700 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a60734d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733cb70 msgr2=0x55a607346580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733cb70 0x55a607346580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/1593931450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 msgr2=0x55a60734d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a60734d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.050+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/1593931450 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd0109c00 con 0x55a60733c210 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a607346040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fc009a100 tx=0x7f1fc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336121dd8e68da87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1faeffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1faeffd700 1 -- 172.21.15.57:0/1593931450 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc009a400 con 0x55a60733c210 2023-10-06T22:13:51.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1faeffd700 1 -- 172.21.15.57:0/1593931450 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1fc00987a0 con 0x55a60733c210 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a60732eec0 con 0x55a60733c210 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1faeffd700 1 -- 172.21.15.57:0/1593931450 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc00cae40 con 0x55a60733c210 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1fe99f8b80 10 client.14961 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f1fd01a1220 con 0x55a60733c210 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.051+0000 7f1faeffd700 1 -- 172.21.15.57:0/1593931450 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc00a6080 con 0x55a60733c210 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1faeffd700 1 -- 172.21.15.57:0/1593931450 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fc00998f0 con 0x55a60733c210 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1fe99f8b80 10 client.14961 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1fe99f8b80 10 client.14961 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1fe99f8b80 20 client.14961 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a607351840 0x55a6073504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1fe99f8b80 4 client.14961 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a607287dc0 con 0x55a607351840 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.052+0000 7f1fceffd700 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a607351840 0x55a6073504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.053+0000 7f1fceffd700 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a607351840 0x55a6073504f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e66b9fedadfa18c server_cookie=e4fe1fe43bcaa0ee in_seq=0 out_seq=0 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.053+0000 7f1faeffd700 10 client.14961 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.053+0000 7f1faeffd700 1 -- 172.21.15.57:0/1593931450 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a607287dc0 con 0x55a607351840 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.053+0000 7f1faeffd700 10 client.14961 handle_command_reply: tid=0 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.053+0000 7f1fe99f8b80 1 client.14961 shutdown 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 msgr2=0x55a607346040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a607346040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fc009a100 tx=0x7f1fc0004750 comp rx=0 tx=0).stop 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 shutdown_connections 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a607351840 0x55a6073504f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a60734d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a607346040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/1593931450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733cb70 0x55a607346580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 >> 172.21.15.57:0/1593931450 conn(0x55a6072b0210 msgr2=0x55a607336f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1faeffd700 0 client.14961 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:13:51.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 shutdown_connections 2023-10-06T22:13:51.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/1593931450 wait complete. 2023-10-06T22:13:51.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.054+0000 7f1fe99f8b80 20 client.14961 trim_cache size 0 max 16384 2023-10-06T22:13:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:50 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/581659976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:50 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:50 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:50 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2085445106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:51.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 Processor -- start 2023-10-06T22:13:51.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 -- start start 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733c210 0x55a607344ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733cb70 0x55a60734d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a607340980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a607345a50 con 0x55a60733cb70 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60710f8d0 con 0x55a60733da10 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a607336290 con 0x55a60733c210 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733cb70 0x55a60734d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733cb70 0x55a60734d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43134/0 (socket says 172.21.15.57:43134) 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fceffd700 1 -- 172.21.15.57:0/3269303238 learned_addr learned my addr 172.21.15.57:0/3269303238 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.073+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733c210 0x55a607344ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fcffff700 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a607340980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fceffd700 1 -- 172.21.15.57:0/3269303238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733c210 msgr2=0x55a607344ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fceffd700 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733c210 0x55a607344ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fceffd700 1 -- 172.21.15.57:0/3269303238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 msgr2=0x55a607340980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fceffd700 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a607340980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fceffd700 1 -- 172.21.15.57:0/3269303238 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6070a2c10 con 0x55a60733cb70 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733c210 0x55a607344ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fceffd700 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733cb70 0x55a60734d550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009990 tx=0x7f1fc400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf49fc0c4760a63b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fb67fc700 1 -- 172.21.15.57:0/3269303238 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4029cf0 con 0x55a60733cb70 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fb67fc700 1 -- 172.21.15.57:0/3269303238 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc401e040 con 0x55a60733cb70 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.074+0000 7f1fb67fc700 1 -- 172.21.15.57:0/3269303238 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4004500 con 0x55a60733cb70 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3269303238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733cb70 msgr2=0x55a60734d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733cb70 0x55a60734d550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009990 tx=0x7f1fc400bd50 comp rx=0 tx=0).stop 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3269303238 shutdown_connections 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733da10 0x55a607340980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733c210 0x55a607344ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/3269303238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733cb70 0x55a60734d550 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3269303238 >> 172.21.15.57:0/3269303238 conn(0x55a6072b0210 msgr2=0x55a60735a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3269303238 shutdown_connections 2023-10-06T22:13:51.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.075+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/3269303238 wait complete. 2023-10-06T22:13:51.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.076+0000 7f1fe99f8b80 1 -- start start 2023-10-06T22:13:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.076+0000 7f1fe99f8b80 1 Processor -- start 2023-10-06T22:13:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60734d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a607349e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fe99f8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60734a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60734a930 con 0x55a60733c210 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60734aaa0 con 0x55a60733cb70 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fe99f8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6073a52b0 con 0x55a60733da10 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fe99f8b80 10 client.0 _ll_register_callbacks cb 0x7f1fc8009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60734d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60734d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43144/0 (socket says 172.21.15.57:43144) 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/2031953705 learned_addr learned my addr 172.21.15.57:0/2031953705 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fceffd700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a607349e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.077+0000 7f1fcffff700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60734a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 msgr2=0x55a60734a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60734a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/2031953705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 msgr2=0x55a607349e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a607349e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fcf7fe700 1 -- 172.21.15.57:0/2031953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd01a1220 con 0x55a60733c210 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fceffd700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a607349e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fcf7fe700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60734d380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009230 tx=0x7f1fc001fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b790560decd3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1faeffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1faeffd700 1 -- 172.21.15.57:0/2031953705 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc009e380 con 0x55a60733c210 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fc009e550 con 0x55a60733c210 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.078+0000 7f1faeffd700 1 -- 172.21.15.57:0/2031953705 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1fc009a400 con 0x55a60733c210 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1faeffd700 1 -- 172.21.15.57:0/2031953705 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc00a2810 con 0x55a60733c210 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 10 client.14967 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f1fc400b990 con 0x55a60733c210 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1faeffd700 1 -- 172.21.15.57:0/2031953705 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc009e550 con 0x55a60733c210 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1faeffd700 1 -- 172.21.15.57:0/2031953705 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fc009a6d0 con 0x55a60733c210 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 10 client.14967 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 10 client.14967 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 20 client.14967 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a6073aa190 0x55a6073a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 4 client.14967 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.079+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a607287ae0 con 0x55a6073aa190 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fceffd700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a6073aa190 0x55a6073a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fceffd700 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a6073aa190 0x55a6073a65d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ea4d6c1d58d13d1 server_cookie=25cb9f681f6ef621 in_seq=0 out_seq=0 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1faeffd700 10 client.14967 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:13:51.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1faeffd700 1 -- 172.21.15.57:0/2031953705 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a607287ae0 con 0x55a6073aa190 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1faeffd700 10 client.14967 handle_command_reply: tid=0 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 client.14967 shutdown 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 msgr2=0x55a60734d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60734d380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009230 tx=0x7f1fc001fde0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 shutdown_connections 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55a6073aa190 0x55a6073a65d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a60733da10 0x55a60734a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a60733c210 0x55a60734d380 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 --2- 172.21.15.57:0/2031953705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a60733cb70 0x55a607349e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.080+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 >> 172.21.15.57:0/2031953705 conn(0x55a6072b0210 msgr2=0x55a607344ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1faeffd700 0 client.14967 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:13:51.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 shutdown_connections 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1fe99f8b80 1 -- 172.21.15.57:0/2031953705 wait complete. 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1fe99f8b80 20 client.14967 trim_cache size 0 max 16384 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 3, 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24517, 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:13:51.089 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:13:51.090 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 62.321514165000004, 2023-10-06T22:13:51.090 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:13:51.090 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:13:51.090 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:13:51.090 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 88.284144369000003 2023-10-06T22:13:51.090 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1fb80a1590 msgr2=0x7f1fb80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1fb80a1590 0x7f1fb80a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009ec0 tx=0x7f1fc800b040 comp rx=0 tx=0).stop 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0101940 msgr2=0x7f1fd01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.081+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0101940 0x7f1fd01012b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000eb30 tx=0x7f1fc000ee40 comp rx=0 tx=0).stop 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 shutdown_connections 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1fb80a1590 0x7f1fb80a3a50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fd0106ba0 0x7f1fd00ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd0101940 0x7f1fd01012b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 --2- 172.21.15.57:0/349878757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd0104270 0x7f1fd00ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 >> 172.21.15.57:0/349878757 conn(0x7f1fd00fb330 msgr2=0x7f1fd01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 shutdown_connections 2023-10-06T22:13:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.082+0000 7f1fd5fff700 1 -- 172.21.15.57:0/349878757 wait complete. 2023-10-06T22:13:51.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:50 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/581659976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:51.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:50 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:51.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:50 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:51.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:50 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2085445106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:51.101 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T22:13:51.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:50 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/581659976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:13:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:50 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:50 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:50 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2085445106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:13:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.330+0000 7fd85f5ff700 1 -- 172.21.15.57:0/3690356486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd858073910 msgr2=0x7fd858073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.330+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/3690356486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd858073910 0x7fd858073d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd848009990 tx=0x7fd848009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.330+0000 7fd85f5ff700 1 -- 172.21.15.57:0/3690356486 shutdown_connections 2023-10-06T22:13:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.330+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/3690356486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd85810d4c0 0x7fd85810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.330+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/3690356486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd858074ef0 0x7fd858073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.330+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/3690356486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd858073910 0x7fd858073d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.330+0000 7fd85f5ff700 1 -- 172.21.15.57:0/3690356486 >> 172.21.15.57:0/3690356486 conn(0x7fd8580fc460 msgr2=0x7fd8580fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.331+0000 7fd85f5ff700 1 -- 172.21.15.57:0/3690356486 shutdown_connections 2023-10-06T22:13:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.331+0000 7fd85f5ff700 1 -- 172.21.15.57:0/3690356486 wait complete. 2023-10-06T22:13:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 Processor -- start 2023-10-06T22:13:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 -- start start 2023-10-06T22:13:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd858073910 0x7fd85819d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858074ef0 0x7fd85819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85810d4c0 0x7fd8581a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581a4da0 con 0x7fd85810d4c0 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581a4ee0 con 0x7fd858074ef0 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85f5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581a5070 con 0x7fd858073910 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85db9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85810d4c0 0x7fd8581a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85db9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85810d4c0 0x7fd8581a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43162/0 (socket says 172.21.15.57:43162) 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85db9c700 1 -- 172.21.15.57:0/707216565 learned_addr learned my addr 172.21.15.57:0/707216565 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.332+0000 7fd85d39b700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd858073910 0x7fd85819d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858074ef0 0x7fd85819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85db9c700 1 -- 172.21.15.57:0/707216565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd858073910 msgr2=0x7fd85819d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85db9c700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd858073910 0x7fd85819d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85db9c700 1 -- 172.21.15.57:0/707216565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858074ef0 msgr2=0x7fd85819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85db9c700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858074ef0 0x7fd85819d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85db9c700 1 -- 172.21.15.57:0/707216565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480095f0 con 0x7fd85810d4c0 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858074ef0 0x7fd85819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85db9c700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85810d4c0 0x7fd8581a47d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd85000ef30 tx=0x7fd85000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd84e7fc700 1 -- 172.21.15.57:0/707216565 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850018070 con 0x7fd85810d4c0 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd84e7fc700 1 -- 172.21.15.57:0/707216565 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd85000f040 con 0x7fd85810d4c0 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd84e7fc700 1 -- 172.21.15.57:0/707216565 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000cd00 con 0x7fd85810d4c0 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.333+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8581a52f0 con 0x7fd85810d4c0 2023-10-06T22:13:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.334+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8581a57b0 con 0x7fd85810d4c0 2023-10-06T22:13:51.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.337+0000 7fd84e7fc700 1 -- 172.21.15.57:0/707216565 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fd850003a00 con 0x7fd85810d4c0 2023-10-06T22:13:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.338+0000 7fd84e7fc700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd8440a1510 0x7fd8440a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd84e7fc700 1 -- 172.21.15.57:0/707216565 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fd8500cca30 con 0x7fd85810d4c0 2023-10-06T22:13:51.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 Processor -- start 2023-10-06T22:13:51.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd85d39b700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd8440a1510 0x7fd8440a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 -- start start 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545313630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313b70 0x560545314430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545314e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56054527b410 con 0x560545313210 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56054526cbd0 con 0x560545314a10 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.339+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56054526a4f0 con 0x560545313b70 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85d39b700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd8440a1510 0x7fd8440a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd854005950 tx=0x7fd85400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313b70 0x560545314430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85d39b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545313630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313b70 0x560545314430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57010/0 (socket says 172.21.15.57:57010) 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3499057226 learned_addr learned my addr 172.21.15.57:0/3499057226 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85db9c700 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545314e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3499057226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 msgr2=0x560545314e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545314e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3499057226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 msgr2=0x560545313630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545313630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3499057226 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560545090d90 con 0x560545313b70 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.340+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313b70 0x560545314430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd858074c10 tx=0x7fd848005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d1cbacf5e0b9c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.341+0000 7fd8437fe700 1 -- 172.21.15.57:0/3499057226 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84801d070 con 0x560545313b70 2023-10-06T22:13:51.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.341+0000 7fd8437fe700 1 -- 172.21.15.57:0/3499057226 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84801e040 con 0x560545313b70 2023-10-06T22:13:51.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.341+0000 7fd8437fe700 1 -- 172.21.15.57:0/3499057226 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848004e90 con 0x560545313b70 2023-10-06T22:13:51.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3499057226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313b70 msgr2=0x560545314430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313b70 0x560545314430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd858074c10 tx=0x7fd848005c70 comp rx=0 tx=0).stop 2023-10-06T22:13:51.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3499057226 shutdown_connections 2023-10-06T22:13:51.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545314e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545313630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3499057226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313b70 0x560545314430 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3499057226 >> 172.21.15.57:0/3499057226 conn(0x560545287210 msgr2=0x56054530e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3499057226 shutdown_connections 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3499057226 wait complete. 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.342+0000 7fd872ea4b80 1 -- start start 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.343+0000 7fd872ea4b80 1 Processor -- start 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x56054531cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x56054531d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545314a10 0x5605453244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560545265de0 con 0x560545314a10 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605452026c0 con 0x560545313b70 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560545202360 con 0x560545313210 2023-10-06T22:13:51.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd872ea4b80 10 client.0 _ll_register_callbacks cb 0x56054505f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd85db9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545314a10 0x5605453244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd85d39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x56054531cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd85cb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x56054531d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd85d39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x56054531cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57022/0 (socket says 172.21.15.57:57022) 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd85d39b700 1 -- 172.21.15.57:0/3357215589 learned_addr learned my addr 172.21.15.57:0/3357215589 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.344+0000 7fd85db9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545314a10 0x5605453244a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43178/0 (socket says 172.21.15.57:43178) 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85d39b700 1 -- 172.21.15.57:0/3357215589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 msgr2=0x56054531d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85d39b700 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x56054531d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85d39b700 1 -- 172.21.15.57:0/3357215589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545314a10 msgr2=0x5605453244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85d39b700 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545314a10 0x5605453244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85d39b700 1 -- 172.21.15.57:0/3357215589 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd858110260 con 0x560545313210 2023-10-06T22:13:51.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x56054531d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85db9c700 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545314a10 0x5605453244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd85d39b700 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x56054531cf20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd8540116c0 tx=0x7fd8540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6aee882efb9b6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd840ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:13:51.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd840ff9700 1 -- 172.21.15.57:0/3357215589 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85401e830 con 0x560545313210 2023-10-06T22:13:51.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560545305ec0 con 0x560545313210 2023-10-06T22:13:51.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.345+0000 7fd840ff9700 1 -- 172.21.15.57:0/3357215589 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fd854017070 con 0x560545313210 2023-10-06T22:13:51.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd840ff9700 1 -- 172.21.15.57:0/3357215589 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd854011e60 con 0x560545313210 2023-10-06T22:13:51.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 10 client.25009 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560545079c10 con 0x560545313210 2023-10-06T22:13:51.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd840ff9700 1 -- 172.21.15.57:0/3357215589 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560545305ec0 con 0x560545313210 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd840ff9700 1 -- 172.21.15.57:0/3357215589 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd854028ae0 con 0x560545313210 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 10 client.25009 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 10 client.25009 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 20 client.25009 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545328bc0 0x560545327650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 4 client.25009 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.346+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56054525edc0 con 0x560545328bc0 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.347+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545328bc0 0x560545327650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.347+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545328bc0 0x560545327650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3110faaedf30b937 server_cookie=5dfb7711d8a00834 in_seq=0 out_seq=0 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.347+0000 7fd840ff9700 10 client.25009 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd840ff9700 1 -- 172.21.15.57:0/3357215589 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56054525edc0 con 0x560545328bc0 2023-10-06T22:13:51.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd840ff9700 10 client.25009 handle_command_reply: tid=0 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 client.25009 shutdown 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 msgr2=0x56054531cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x56054531cf20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd8540116c0 tx=0x7fd8540047c0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 shutdown_connections 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545328bc0 0x560545327650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545314a10 0x5605453244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd840ff9700 0 client.25009 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x56054531cf20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3357215589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x56054531d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 >> 172.21.15.57:0/3357215589 conn(0x560545287210 msgr2=0x56054530df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 shutdown_connections 2023-10-06T22:13:51.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.348+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3357215589 wait complete. 2023-10-06T22:13:51.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.349+0000 7fd872ea4b80 20 client.25009 trim_cache size 0 max 16384 2023-10-06T22:13:51.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 Processor -- start 2023-10-06T22:13:51.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 -- start start 2023-10-06T22:13:51.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545317920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x560545324430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545314a10 0x560545317e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56054531c930 con 0x560545313210 2023-10-06T22:13:51.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605450e68d0 con 0x560545313b70 2023-10-06T22:13:51.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.371+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56054520c4d0 con 0x560545314a10 2023-10-06T22:13:51.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85d39b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545317920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85db9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545314a10 0x560545317e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85db9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545314a10 0x560545317e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57034/0 (socket says 172.21.15.57:57034) 2023-10-06T22:13:51.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85db9c700 1 -- 172.21.15.57:0/1907306746 learned_addr learned my addr 172.21.15.57:0/1907306746 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x560545324430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85d39b700 1 -- 172.21.15.57:0/1907306746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545314a10 msgr2=0x560545317e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85d39b700 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545314a10 0x560545317e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85d39b700 1 -- 172.21.15.57:0/1907306746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 msgr2=0x560545324430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85d39b700 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x560545324430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85d39b700 1 -- 172.21.15.57:0/1907306746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605450b7200 con 0x560545313210 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85db9c700 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545314a10 0x560545317e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x560545324430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.372+0000 7fd85d39b700 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545317920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd854028ec0 tx=0x7fd854005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b131e31fbfa9d77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd843fff700 1 -- 172.21.15.57:0/1907306746 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85400bea0 con 0x560545313210 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd843fff700 1 -- 172.21.15.57:0/1907306746 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd854037070 con 0x560545313210 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd843fff700 1 -- 172.21.15.57:0/1907306746 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854005550 con 0x560545313210 2023-10-06T22:13:51.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd872ea4b80 1 -- 172.21.15.57:0/1907306746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 msgr2=0x560545317920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545317920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd854028ec0 tx=0x7fd854005b40 comp rx=0 tx=0).stop 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd872ea4b80 1 -- 172.21.15.57:0/1907306746 shutdown_connections 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545314a10 0x560545317e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313210 0x560545317920 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/1907306746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545313b70 0x560545324430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.373+0000 7fd872ea4b80 1 -- 172.21.15.57:0/1907306746 >> 172.21.15.57:0/1907306746 conn(0x560545287210 msgr2=0x56054530cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.374+0000 7fd872ea4b80 1 -- 172.21.15.57:0/1907306746 shutdown_connections 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.374+0000 7fd872ea4b80 1 -- 172.21.15.57:0/1907306746 wait complete. 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.374+0000 7fd872ea4b80 1 -- start start 2023-10-06T22:13:51.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.374+0000 7fd872ea4b80 1 Processor -- start 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x5605453213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313b70 0x560545323940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd872ea4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545323e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605453243c0 con 0x560545313b70 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560545324530 con 0x560545314a10 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd872ea4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605453246f0 con 0x560545313210 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd872ea4b80 10 client.0 _ll_register_callbacks cb 0x7fd854010e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313b70 0x560545323940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313b70 0x560545323940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43198/0 (socket says 172.21.15.57:43198) 2023-10-06T22:13:51.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3918899444 learned_addr learned my addr 172.21.15.57:0/3918899444 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85db9c700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545323e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85d39b700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x5605453213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 msgr2=0x5605453213f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x5605453213f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3918899444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 msgr2=0x560545323e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545323e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85cb9a700 1 -- 172.21.15.57:0/3918899444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8581a57b0 con 0x560545313b70 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85db9c700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545323e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.375+0000 7fd85d39b700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x5605453213f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd85cb9a700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313b70 0x560545323940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd848009990 tx=0x7fd848009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59c10fa5bcfdf63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd840ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd840ff9700 1 -- 172.21.15.57:0/3918899444 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848032040 con 0x560545313b70 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd840ff9700 1 -- 172.21.15.57:0/3918899444 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fd848004810 con 0x560545313b70 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd84801e040 con 0x560545313b70 2023-10-06T22:13:51.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd840ff9700 1 -- 172.21.15.57:0/3918899444 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd84801d040 con 0x560545313b70 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd872ea4b80 10 client.14985 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560545079c10 con 0x560545313b70 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.376+0000 7fd840ff9700 1 -- 172.21.15.57:0/3918899444 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84801e040 con 0x560545313b70 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd840ff9700 1 -- 172.21.15.57:0/3918899444 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd848005040 con 0x560545313b70 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd872ea4b80 10 client.14985 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd872ea4b80 10 client.14985 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd872ea4b80 20 client.14985 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545381180 0x56054537d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd872ea4b80 4 client.14985 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56054525eae0 con 0x560545381180 2023-10-06T22:13:51.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.377+0000 7fd85d39b700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545381180 0x56054537d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd85d39b700 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545381180 0x56054537d620 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9f641b764d34532 server_cookie=478d5b0a0475cc2 in_seq=0 out_seq=0 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd840ff9700 10 client.14985 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd840ff9700 1 -- 172.21.15.57:0/3918899444 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56054525eae0 con 0x560545381180 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd840ff9700 10 client.14985 handle_command_reply: tid=0 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 client.14985 shutdown 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313b70 msgr2=0x560545323940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313b70 0x560545323940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd848009990 tx=0x7fd848009290 comp rx=0 tx=0).stop 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 shutdown_connections 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x560545381180 0x56054537d620 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560545314a10 0x560545323e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560545313210 0x5605453213f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 --2- 172.21.15.57:0/3918899444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560545313b70 0x560545323940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd840ff9700 0 client.14985 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.378+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 >> 172.21.15.57:0/3918899444 conn(0x560545287210 msgr2=0x56054531ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 shutdown_connections 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd872ea4b80 1 -- 172.21.15.57:0/3918899444 wait complete. 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd872ea4b80 20 client.14985 trim_cache size 0 max 16384 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:13:51.386 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24529, 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 63.636535145000003, 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:13:51.387 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:13:51.388 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:13:51.388 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 86.298415470999998 2023-10-06T22:13:51.388 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd8440a1510 msgr2=0x7fd8440a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd8440a1510 0x7fd8440a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd854005950 tx=0x7fd85400b410 comp rx=0 tx=0).stop 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85810d4c0 msgr2=0x7fd8581a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85810d4c0 0x7fd8581a47d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd85000ef30 tx=0x7fd85000c3a0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 shutdown_connections 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fd8440a1510 0x7fd8440a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85810d4c0 0x7fd8581a47d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd858073910 0x7fd85819d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 --2- 172.21.15.57:0/707216565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858074ef0 0x7fd85819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.379+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 >> 172.21.15.57:0/707216565 conn(0x7fd8580fc460 msgr2=0x7fd8580fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.380+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 shutdown_connections 2023-10-06T22:13:51.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.380+0000 7fd85f5ff700 1 -- 172.21.15.57:0/707216565 wait complete. 2023-10-06T22:13:51.401 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T22:13:51.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 -- 172.21.15.57:0/2762709024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0074ef0 msgr2=0x7f30c0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 --2- 172.21.15.57:0/2762709024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0074ef0 0x7f30c0073340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009990 tx=0x7f30b0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 -- 172.21.15.57:0/2762709024 shutdown_connections 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 --2- 172.21.15.57:0/2762709024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0108da0 0x7f30c010b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 --2- 172.21.15.57:0/2762709024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0074ef0 0x7f30c0073340 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 --2- 172.21.15.57:0/2762709024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c0073910 0x7f30c0073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 -- 172.21.15.57:0/2762709024 >> 172.21.15.57:0/2762709024 conn(0x7f30c00fc430 msgr2=0x7f30c00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 -- 172.21.15.57:0/2762709024 shutdown_connections 2023-10-06T22:13:51.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.608+0000 7f30c67ff700 1 -- 172.21.15.57:0/2762709024 wait complete. 2023-10-06T22:13:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.609+0000 7f30c67ff700 1 Processor -- start 2023-10-06T22:13:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30c67ff700 1 -- start start 2023-10-06T22:13:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30c67ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0073910 0x7f30c0198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30c67ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0074ef0 0x7f30c0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30c67ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c0108da0 0x7f30c01a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30c67ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01a09a0 con 0x7f30c0073910 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30c67ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01a0ae0 con 0x7f30c0108da0 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30c67ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01a0c70 con 0x7f30c0074ef0 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0073910 0x7f30c0198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0073910 0x7f30c0198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43210/0 (socket says 172.21.15.57:43210) 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30bffff700 1 -- 172.21.15.57:0/1405726838 learned_addr learned my addr 172.21.15.57:0/1405726838 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.610+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0074ef0 0x7f30c0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c0108da0 0x7f30c01a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bffff700 1 -- 172.21.15.57:0/1405726838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0074ef0 msgr2=0x7f30c0199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bffff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0074ef0 0x7f30c0199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bffff700 1 -- 172.21.15.57:0/1405726838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c0108da0 msgr2=0x7f30c01a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bffff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c0108da0 0x7f30c01a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bffff700 1 -- 172.21.15.57:0/1405726838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b00095f0 con 0x7f30c0073910 2023-10-06T22:13:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0074ef0 0x7f30c0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bffff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0073910 0x7f30c0198e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b680 tx=0x7f30b0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bd7fa700 1 -- 172.21.15.57:0/1405726838 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b001d070 con 0x7f30c0073910 2023-10-06T22:13:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c01a0ef0 con 0x7f30c0073910 2023-10-06T22:13:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bd7fa700 1 -- 172.21.15.57:0/1405726838 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30b0022470 con 0x7f30c0073910 2023-10-06T22:13:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.611+0000 7f30bd7fa700 1 -- 172.21.15.57:0/1405726838 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0017650 con 0x7f30c0073910 2023-10-06T22:13:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.612+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c01a12b0 con 0x7f30c0073910 2023-10-06T22:13:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.615+0000 7f30bd7fa700 1 -- 172.21.15.57:0/1405726838 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f30b00225e0 con 0x7f30c0073910 2023-10-06T22:13:51.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 Processor -- start 2023-10-06T22:13:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 -- start start 2023-10-06T22:13:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30bd7fa700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30a80a1510 0x7f30a80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30bd7fa700 1 -- 172.21.15.57:0/1405726838 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f30b00ce360 con 0x7f30c0073910 2023-10-06T22:13:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30a80a1510 0x7f30a80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2db430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad242410 con 0x5651ad2dba10 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad233bd0 con 0x5651ad2dab70 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.617+0000 7f30da162b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad2314f0 con 0x5651ad2da210 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2da630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57080/0 (socket says 172.21.15.57:57080) 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 -- 172.21.15.57:0/2631192692 learned_addr learned my addr 172.21.15.57:0/2631192692 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2db430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 -- 172.21.15.57:0/2631192692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 msgr2=0x5651ad2db430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2db430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 -- 172.21.15.57:0/2631192692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 msgr2=0x5651ad2dbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2dbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 -- 172.21.15.57:0/2631192692 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651ad057d90 con 0x5651ad2da210 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2db430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30a80a1510 0x7f30a80a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30b800f870 tx=0x7f30b800fc30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.618+0000 7f30bffff700 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2da630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30b009b510 tx=0x7f30b009b140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e980a71458602a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.619+0000 7f30a67fc700 1 -- 172.21.15.57:0/2631192692 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b009bc80 con 0x5651ad2da210 2023-10-06T22:13:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.619+0000 7f30a67fc700 1 -- 172.21.15.57:0/2631192692 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30b009b580 con 0x5651ad2da210 2023-10-06T22:13:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.619+0000 7f30a67fc700 1 -- 172.21.15.57:0/2631192692 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b009ba40 con 0x5651ad2da210 2023-10-06T22:13:51.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.619+0000 7f30da162b80 1 -- 172.21.15.57:0/2631192692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 msgr2=0x5651ad2da630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.619+0000 7f30da162b80 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2da630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30b009b510 tx=0x7f30b009b140 comp rx=0 tx=0).stop 2023-10-06T22:13:51.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 -- 172.21.15.57:0/2631192692 shutdown_connections 2023-10-06T22:13:51.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2da630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 --2- 172.21.15.57:0/2631192692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2db430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 -- 172.21.15.57:0/2631192692 >> 172.21.15.57:0/2631192692 conn(0x5651ad24e210 msgr2=0x5651ad2d50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 -- 172.21.15.57:0/2631192692 shutdown_connections 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 -- 172.21.15.57:0/2631192692 wait complete. 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 -- start start 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.620+0000 7f30da162b80 1 Processor -- start 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30da162b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30da162b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dab70 0x5651ad2e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30da162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30da162b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad22cde0 con 0x5651ad2dab70 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30da162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad1c96c0 con 0x5651ad2dba10 2023-10-06T22:13:51.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30da162b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad1c9360 con 0x5651ad2da210 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30da162b80 10 client.0 _ll_register_callbacks cb 0x5651ad0267b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30bf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dab70 0x5651ad2e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30bf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dab70 0x5651ad2e41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43240/0 (socket says 172.21.15.57:43240) 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30bf7fe700 1 -- 172.21.15.57:0/3139185140 learned_addr learned my addr 172.21.15.57:0/3139185140 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30bffff700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.621+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30bf7fe700 1 -- 172.21.15.57:0/3139185140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 msgr2=0x5651ad2e3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30bf7fe700 1 -- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 msgr2=0x5651ad2eb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30bf7fe700 1 -- 172.21.15.57:0/3139185140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c01a12b0 con 0x5651ad2dab70 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30bffff700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dab70 0x5651ad2e41c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30b4012d10 tx=0x7f30b4012d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5218f8d4028dc417 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f309f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f309f7fe700 1 -- 172.21.15.57:0/3139185140 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4017070 con 0x5651ad2dab70 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f309f7fe700 1 -- 172.21.15.57:0/3139185140 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f30b4004050 con 0x5651ad2dab70 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651ad2ccec0 con 0x5651ad2dab70 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f309f7fe700 1 -- 172.21.15.57:0/3139185140 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30b4004210 con 0x5651ad2dab70 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30da162b80 10 client.14997 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.622+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5651ad040c10 con 0x5651ad2dab70 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f309f7fe700 1 -- 172.21.15.57:0/3139185140 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5651ad2ccec0 con 0x5651ad2dab70 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f309f7fe700 1 -- 172.21.15.57:0/3139185140 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f30b40038b0 con 0x5651ad2dab70 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f30da162b80 10 client.14997 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f30da162b80 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f30da162b80 20 client.14997 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f30da162b80 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad2ef890 0x5651ad2ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f30da162b80 4 client.14997 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651ad225dc0 con 0x5651ad2ef890 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.623+0000 7f30bffff700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad2ef890 0x5651ad2ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.624+0000 7f309f7fe700 10 client.14997 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:13:51.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.624+0000 7f30bffff700 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad2ef890 0x5651ad2ee360 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=408d2590e3071614 server_cookie=f7b69b168c29283b in_seq=0 out_seq=0 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f309f7fe700 1 -- 172.21.15.57:0/3139185140 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5651ad225dc0 con 0x5651ad2ef890 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f309f7fe700 10 client.14997 handle_command_reply: tid=0 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 client.14997 shutdown 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dab70 msgr2=0x5651ad2e41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dab70 0x5651ad2e41c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30b4012d10 tx=0x7f30b4012d40 comp rx=0 tx=0).stop 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 shutdown_connections 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad2ef890 0x5651ad2ee360 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 --2- 172.21.15.57:0/3139185140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dab70 0x5651ad2e41c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f309f7fe700 0 client.14997 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:13:51.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 >> 172.21.15.57:0/3139185140 conn(0x5651ad24e210 msgr2=0x5651ad2d4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 shutdown_connections 2023-10-06T22:13:51.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 1 -- 172.21.15.57:0/3139185140 wait complete. 2023-10-06T22:13:51.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.625+0000 7f30da162b80 20 client.14997 trim_cache size 0 max 16384 2023-10-06T22:13:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 Processor -- start 2023-10-06T22:13:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 -- start start 2023-10-06T22:13:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2eba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad0ad8d0 con 0x5651ad2dba10 2023-10-06T22:13:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad228ed0 con 0x5651ad2dab70 2023-10-06T22:13:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30da162b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad1d34d0 con 0x5651ad2da210 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.644+0000 7f30c4d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43250/0 (socket says 172.21.15.57:43250) 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 -- 172.21.15.57:0/1349371025 learned_addr learned my addr 172.21.15.57:0/1349371025 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30bffff700 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 -- 172.21.15.57:0/1349371025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 msgr2=0x5651ad2e1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 -- 172.21.15.57:0/1349371025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 msgr2=0x5651ad2eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 -- 172.21.15.57:0/1349371025 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651ad07e200 con 0x5651ad2dba10 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2eba30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f30b80091f0 tx=0x7f30b801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47604d43e9617178 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.645+0000 7f30a6ffd700 1 -- 172.21.15.57:0/1349371025 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800e040 con 0x5651ad2dba10 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30a6ffd700 1 -- 172.21.15.57:0/1349371025 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30b800f040 con 0x5651ad2dba10 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30a6ffd700 1 -- 172.21.15.57:0/1349371025 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b802c6b0 con 0x5651ad2dba10 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 -- 172.21.15.57:0/1349371025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 msgr2=0x5651ad2eba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2eba30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f30b80091f0 tx=0x7f30b801f000 comp rx=0 tx=0).stop 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 -- 172.21.15.57:0/1349371025 shutdown_connections 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2dba10 0x5651ad2eba30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2da210 0x5651ad2e1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 --2- 172.21.15.57:0/1349371025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dab70 0x5651ad2eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 -- 172.21.15.57:0/1349371025 >> 172.21.15.57:0/1349371025 conn(0x5651ad24e210 msgr2=0x5651ad2e29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 -- 172.21.15.57:0/1349371025 shutdown_connections 2023-10-06T22:13:51.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.646+0000 7f30da162b80 1 -- 172.21.15.57:0/1349371025 wait complete. 2023-10-06T22:13:51.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.647+0000 7f30da162b80 1 -- start start 2023-10-06T22:13:51.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.647+0000 7f30da162b80 1 Processor -- start 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30da162b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2da210 0x5651ad2e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30da162b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2dab70 0x5651ad2ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30da162b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30da162b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad2e46f0 con 0x5651ad2da210 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30da162b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad2eb1f0 con 0x5651ad2dba10 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30da162b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ad2eb3b0 con 0x5651ad2dab70 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30da162b80 10 client.0 _ll_register_callbacks cb 0x7f30b40127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2da210 0x5651ad2e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2da210 0x5651ad2e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43254/0 (socket says 172.21.15.57:43254) 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 -- 172.21.15.57:0/1180501868 learned_addr learned my addr 172.21.15.57:0/1180501868 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2dab70 0x5651ad2ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 -- 172.21.15.57:0/1180501868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2dab70 msgr2=0x5651ad2ea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2dab70 0x5651ad2ea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 -- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 msgr2=0x5651ad2eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30bffff700 1 -- 172.21.15.57:0/1180501868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651ad040c10 con 0x5651ad2da210 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.648+0000 7f30c4d9c700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30bffff700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2da210 0x5651ad2e81f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f30b00cbc20 tx=0x7f30b00972b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbfb3e01e763e83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f309f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f309f7fe700 1 -- 172.21.15.57:0/1180501868 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b009f6d0 con 0x5651ad2da210 2023-10-06T22:13:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30b009b580 con 0x5651ad2da210 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f309f7fe700 1 -- 172.21.15.57:0/1180501868 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f30b0097040 con 0x5651ad2da210 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f309f7fe700 1 -- 172.21.15.57:0/1180501868 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30b00a3880 con 0x5651ad2da210 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 10 client.15003 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5651ad056260 con 0x5651ad2da210 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f309f7fe700 1 -- 172.21.15.57:0/1180501868 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30b009b580 con 0x5651ad2da210 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f309f7fe700 1 -- 172.21.15.57:0/1180501868 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f30b00a33f0 con 0x5651ad2da210 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 10 client.15003 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 10 client.15003 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 20 client.15003 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad347ef0 0x5651ad344390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 4 client.15003 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:13:51.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.649+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5651ad225ae0 con 0x5651ad347ef0 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.650+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad347ef0 0x5651ad344390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.650+0000 7f30bf7fe700 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad347ef0 0x5651ad344390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=211f2ae985f88945 server_cookie=8630b4a13684eeba in_seq=0 out_seq=0 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.650+0000 7f309f7fe700 10 client.15003 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f309f7fe700 1 -- 172.21.15.57:0/1180501868 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5651ad225ae0 con 0x5651ad347ef0 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f309f7fe700 10 client.15003 handle_command_reply: tid=0 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 client.15003 shutdown 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2da210 msgr2=0x5651ad2e81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2da210 0x5651ad2e81f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f30b00cbc20 tx=0x7f30b00972b0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 shutdown_connections 2023-10-06T22:13:51.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5651ad347ef0 0x5651ad344390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651ad2dba10 0x5651ad2eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f309f7fe700 0 client.15003 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651ad2da210 0x5651ad2e81f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 --2- 172.21.15.57:0/1180501868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5651ad2dab70 0x5651ad2ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 >> 172.21.15.57:0/1180501868 conn(0x5651ad24e210 msgr2=0x5651ad2e29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 shutdown_connections 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 1 -- 172.21.15.57:0/1180501868 wait complete. 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.651+0000 7f30da162b80 20 client.15003 trim_cache size 0 max 16384 2023-10-06T22:13:51.658 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24551, 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 64.91905285, 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:13:51.659 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:13:51.660 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 73.298270493999993 2023-10-06T22:13:51.660 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:13:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30a80a1510 msgr2=0x7f30a80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30a80a1510 0x7f30a80a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30b800f870 tx=0x7f30b800fc30 comp rx=0 tx=0).stop 2023-10-06T22:13:51.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0073910 msgr2=0x7f30c0198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0073910 0x7f30c0198e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b680 tx=0x7f30b0005740 comp rx=0 tx=0).stop 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 shutdown_connections 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f30a80a1510 0x7f30a80a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c0108da0 0x7f30c01a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30c0073910 0x7f30c0198e20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 --2- 172.21.15.57:0/1405726838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0074ef0 0x7f30c0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 >> 172.21.15.57:0/1405726838 conn(0x7f30c00fc430 msgr2=0x7f30c00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 shutdown_connections 2023-10-06T22:13:51.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.652+0000 7f30c67ff700 1 -- 172.21.15.57:0/1405726838 wait complete. 2023-10-06T22:13:51.671 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-06T22:13:51.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 -- 172.21.15.57:0/684100037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4100ea0 msgr2=0x7f99c41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/684100037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4100ea0 0x7f99c41012c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009990 tx=0x7f99b4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 -- 172.21.15.57:0/684100037 shutdown_connections 2023-10-06T22:13:51.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/684100037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99c4102a40 0x7f99c4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/684100037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4100ea0 0x7f99c41012c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/684100037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4102080 0x7f99c4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 -- 172.21.15.57:0/684100037 >> 172.21.15.57:0/684100037 conn(0x7f99c40fc460 msgr2=0x7f99c40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 -- 172.21.15.57:0/684100037 shutdown_connections 2023-10-06T22:13:51.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.876+0000 7f99ca5ff700 1 -- 172.21.15.57:0/684100037 wait complete. 2023-10-06T22:13:51.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 Processor -- start 2023-10-06T22:13:51.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 -- start start 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4100ea0 0x7f99c4196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4102080 0x7f99c41971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99c4102a40 0x7f99c419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c419e800 con 0x7f99c4102080 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c419e970 con 0x7f99c4102a40 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.877+0000 7f99ca5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c419eb30 con 0x7f99c4100ea0 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4102080 0x7f99c41971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4100ea0 0x7f99c4196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4102080 0x7f99c41971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49636/0 (socket says 172.21.15.57:49636) 2023-10-06T22:13:51.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4100ea0 0x7f99c4196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57260/0 (socket says 172.21.15.57:57260) 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c37fe700 1 -- 172.21.15.57:0/2775718494 learned_addr learned my addr 172.21.15.57:0/2775718494 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99c4102a40 0x7f99c419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 -- 172.21.15.57:0/2775718494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99c4102a40 msgr2=0x7f99c419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99c4102a40 0x7f99c419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 -- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4102080 msgr2=0x7f99c41971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4102080 0x7f99c41971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 -- 172.21.15.57:0/2775718494 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40095f0 con 0x7f99c4100ea0 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c37fe700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4102080 0x7f99c41971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.878+0000 7f99c3fff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4100ea0 0x7f99c4196c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99b400b680 tx=0x7f99b4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.879+0000 7f99c17fa700 1 -- 172.21.15.57:0/2775718494 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b401d070 con 0x7f99c4100ea0 2023-10-06T22:13:51.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.879+0000 7f99c17fa700 1 -- 172.21.15.57:0/2775718494 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b4021410 con 0x7f99c4100ea0 2023-10-06T22:13:51.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.879+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c419edb0 con 0x7f99c4100ea0 2023-10-06T22:13:51.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.879+0000 7f99c17fa700 1 -- 172.21.15.57:0/2775718494 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4021ea0 con 0x7f99c4100ea0 2023-10-06T22:13:51.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.879+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c419f180 con 0x7f99c4100ea0 2023-10-06T22:13:51.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.882+0000 7f99c17fa700 1 -- 172.21.15.57:0/2775718494 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f99b40217f0 con 0x7f99c4100ea0 2023-10-06T22:13:51.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.883+0000 7f99ddea5b80 1 Processor -- start 2023-10-06T22:13:51.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.883+0000 7f99c17fa700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f99ac0a1590 0x7f99ac0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.883+0000 7f99ddea5b80 1 -- start start 2023-10-06T22:13:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c17fa700 1 -- 172.21.15.57:0/2775718494 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f99b40cdfc0 con 0x7f99c4100ea0 2023-10-06T22:13:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f99ac0a1590 0x7f99ac0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ab1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1a18410 con 0x55e2e1ab0b70 2023-10-06T22:13:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1a09bd0 con 0x55e2e1ab1a10 2023-10-06T22:13:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1a074f0 con 0x55e2e1ab0210 2023-10-06T22:13:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ab1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ab1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49646/0 (socket says 172.21.15.57:49646) 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c37fe700 1 -- 172.21.15.57:0/378824261 learned_addr learned my addr 172.21.15.57:0/378824261 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c3fff700 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f99ac0a1590 0x7f99ac0a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99bc003ee0 tx=0x7f99bc009100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.884+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c3fff700 1 -- 172.21.15.57:0/378824261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 msgr2=0x55e2e1ab1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c3fff700 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c3fff700 1 -- 172.21.15.57:0/378824261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 msgr2=0x55e2e1ab1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c3fff700 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ab1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c3fff700 1 -- 172.21.15.57:0/378824261 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e182dd90 con 0x55e2e1ab0210 2023-10-06T22:13:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c37fe700 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ab1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:51.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:13:51.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99c3fff700 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab0630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f99b40cb940 tx=0x7f99b409b0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e6800bde2a4048 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99aa7fc700 1 -- 172.21.15.57:0/378824261 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b409f120 con 0x55e2e1ab0210 2023-10-06T22:13:51.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99aa7fc700 1 -- 172.21.15.57:0/378824261 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b409b1e0 con 0x55e2e1ab0210 2023-10-06T22:13:51.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.885+0000 7f99aa7fc700 1 -- 172.21.15.57:0/378824261 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b409a6d0 con 0x55e2e1ab0210 2023-10-06T22:13:51.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 -- 172.21.15.57:0/378824261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 msgr2=0x55e2e1ab0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab0630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f99b40cb940 tx=0x7f99b409b0c0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 -- 172.21.15.57:0/378824261 shutdown_connections 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab0630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/378824261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ab1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 -- 172.21.15.57:0/378824261 >> 172.21.15.57:0/378824261 conn(0x55e2e1a24210 msgr2=0x55e2e1aab0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 -- 172.21.15.57:0/378824261 shutdown_connections 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 -- 172.21.15.57:0/378824261 wait complete. 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.886+0000 7f99ddea5b80 1 -- start start 2023-10-06T22:13:51.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 1 Processor -- start 2023-10-06T22:13:51.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1abe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1abe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1a02de0 con 0x55e2e1ab1a10 2023-10-06T22:13:51.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e199f6c0 con 0x55e2e1ab0b70 2023-10-06T22:13:51.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e199f360 con 0x55e2e1ab0210 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99ddea5b80 10 client.0 _ll_register_callbacks cb 0x55e2e17fc7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99c8b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99c8b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49660/0 (socket says 172.21.15.57:49660) 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99c8b9c700 1 -- 172.21.15.57:0/3586306678 learned_addr learned my addr 172.21.15.57:0/3586306678 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99c37fe700 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1abe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.887+0000 7f99c3fff700 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1abe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99c8b9c700 1 -- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 msgr2=0x55e2e1abe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1abe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99c8b9c700 1 -- 172.21.15.57:0/3586306678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 msgr2=0x55e2e1abe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1abe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99c8b9c700 1 -- 172.21.15.57:0/3586306678 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b8009e00 con 0x55e2e1ab1a10 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f99bc008860 tx=0x7f99bc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa84e5616bcf05ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99a37fe700 1 -- 172.21.15.57:0/3586306678 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc015850 con 0x55e2e1ab1a10 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2e1aa2ec0 con 0x55e2e1ab1a10 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.888+0000 7f99a37fe700 1 -- 172.21.15.57:0/3586306678 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f99bc0159b0 con 0x55e2e1ab1a10 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99a37fe700 1 -- 172.21.15.57:0/3586306678 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99bc015b70 con 0x55e2e1ab1a10 2023-10-06T22:13:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 10 client.15015 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f99c419f180 con 0x55e2e1ab1a10 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99a37fe700 1 -- 172.21.15.57:0/3586306678 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2e1aa2ec0 con 0x55e2e1ab1a10 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99a37fe700 1 -- 172.21.15.57:0/3586306678 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99bc02d400 con 0x55e2e1ab1a10 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 10 client.15015 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 10 client.15015 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 20 client.15015 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1ac9c60 0x55e2e1ac86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 4 client.15015 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:13:51.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.889+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2e19fbdc0 con 0x55e2e1ac9c60 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.890+0000 7f99c3fff700 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1ac9c60 0x55e2e1ac86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.890+0000 7f99c3fff700 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1ac9c60 0x55e2e1ac86e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a2113f79adda51 server_cookie=e35f21bbe7c66e0f in_seq=0 out_seq=0 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.890+0000 7f99a37fe700 10 client.15015 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99a37fe700 1 -- 172.21.15.57:0/3586306678 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2e19fbdc0 con 0x55e2e1ac9c60 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99a37fe700 10 client.15015 handle_command_reply: tid=0 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 client.15015 shutdown 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 msgr2=0x55e2e1ac5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5580 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f99bc008860 tx=0x7f99bc004b40 comp rx=0 tx=0).stop 2023-10-06T22:13:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 shutdown_connections 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1ac9c60 0x55e2e1ac86e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5580 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1abe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99a37fe700 0 client.15015 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/3586306678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1abe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 >> 172.21.15.57:0/3586306678 conn(0x55e2e1a24210 msgr2=0x55e2e1aaaf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 shutdown_connections 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.891+0000 7f99ddea5b80 1 -- 172.21.15.57:0/3586306678 wait complete. 2023-10-06T22:13:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.892+0000 7f99ddea5b80 20 client.15015 trim_cache size 0 max 16384 2023-10-06T22:13:51.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 Processor -- start 2023-10-06T22:13:51.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 -- start start 2023-10-06T22:13:51.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e18838d0 con 0x55e2e1ab1a10 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1aa52b0 con 0x55e2e1ab0b70 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1aa79d0 con 0x55e2e1ab0210 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99c8b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99c8b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49668/0 (socket says 172.21.15.57:49668) 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.910+0000 7f99c8b9c700 1 -- 172.21.15.57:0/1928371585 learned_addr learned my addr 172.21.15.57:0/1928371585 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c3fff700 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c37fe700 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c8b9c700 1 -- 172.21.15.57:0/1928371585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 msgr2=0x55e2e1ab1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c8b9c700 1 -- 172.21.15.57:0/1928371585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 msgr2=0x55e2e1ac54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c8b9c700 1 -- 172.21.15.57:0/1928371585 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e1816c10 con 0x55e2e1ab1a10 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5cf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f99bc008f50 tx=0x7f99bc0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e716b119693ec8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99aaffd700 1 -- 172.21.15.57:0/1928371585 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc02d600 con 0x55e2e1ab1a10 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99aaffd700 1 -- 172.21.15.57:0/1928371585 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99bc00f040 con 0x55e2e1ab1a10 2023-10-06T22:13:51.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.911+0000 7f99aaffd700 1 -- 172.21.15.57:0/1928371585 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc036480 con 0x55e2e1ab1a10 2023-10-06T22:13:51.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 -- 172.21.15.57:0/1928371585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 msgr2=0x55e2e1ac5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5cf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f99bc008f50 tx=0x7f99bc0049f0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 -- 172.21.15.57:0/1928371585 shutdown_connections 2023-10-06T22:13:51.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ac5cf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/1928371585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 -- 172.21.15.57:0/1928371585 >> 172.21.15.57:0/1928371585 conn(0x55e2e1a24210 msgr2=0x55e2e1aa9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 -- 172.21.15.57:0/1928371585 shutdown_connections 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 -- 172.21.15.57:0/1928371585 wait complete. 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.912+0000 7f99ddea5b80 1 -- start start 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 1 Processor -- start 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1ab7300 con 0x55e2e1ab0b70 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1ab7470 con 0x55e2e1ab1a10 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1ab7630 con 0x55e2e1ab0210 2023-10-06T22:13:51.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99ddea5b80 10 client.0 _ll_register_callbacks cb 0x7f99bc004730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49670/0 (socket says 172.21.15.57:49670) 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99c37fe700 1 -- 172.21.15.57:0/2710210208 learned_addr learned my addr 172.21.15.57:0/2710210208 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99c8b9c700 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.913+0000 7f99c3fff700 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99c37fe700 1 -- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 msgr2=0x55e2e1ab6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99c37fe700 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99c37fe700 1 -- 172.21.15.57:0/2710210208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 msgr2=0x55e2e1ab6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99c37fe700 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99c37fe700 1 -- 172.21.15.57:0/2710210208 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c419f180 con 0x55e2e1ab0b70 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99c37fe700 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005950 tx=0x7f99b8010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e015fbf1064adea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99a37fe700 1 -- 172.21.15.57:0/2710210208 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8017040 con 0x55e2e1ab0b70 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99b409b1e0 con 0x55e2e1ab0b70 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99a37fe700 1 -- 172.21.15.57:0/2710210208 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f99b8004050 con 0x55e2e1ab0b70 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99a37fe700 1 -- 172.21.15.57:0/2710210208 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b8004210 con 0x55e2e1ab0b70 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99ddea5b80 10 client.15021 fetch_fsmap learned FSMap version 26 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.914+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e2e1854200 con 0x55e2e1ab0b70 2023-10-06T22:13:51.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99a37fe700 1 -- 172.21.15.57:0/2710210208 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b409b1e0 con 0x55e2e1ab0b70 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99a37fe700 1 -- 172.21.15.57:0/2710210208 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99b801c3d0 con 0x55e2e1ab0b70 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99ddea5b80 10 client.15021 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99ddea5b80 10 client.15021 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99ddea5b80 20 client.15021 populate_metadata read hostname 'smithi057' 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1b226b0 0x55e2e1b1eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99ddea5b80 4 client.15021 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e2e19fbae0 con 0x55e2e1b226b0 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.915+0000 7f99c3fff700 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1b226b0 0x55e2e1b1eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99c3fff700 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1b226b0 0x55e2e1b1eb10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92cfd4c6c9f7b8a9 server_cookie=2a83f55450af9530 in_seq=0 out_seq=0 2023-10-06T22:13:51.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99a37fe700 10 client.15021 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99a37fe700 1 -- 172.21.15.57:0/2710210208 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e2e19fbae0 con 0x55e2e1b226b0 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99a37fe700 10 client.15021 handle_command_reply: tid=0 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 client.15021 shutdown 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 msgr2=0x55e2e1ac54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005950 tx=0x7f99b8010710 comp rx=0 tx=0).stop 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 shutdown_connections 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e2e1b226b0 0x55e2e1b1eb10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2e1ab1a10 0x55e2e1ab6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2e1ab0210 0x55e2e1ab6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 --2- 172.21.15.57:0/2710210208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2e1ab0b70 0x55e2e1ac54a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99a37fe700 0 client.15021 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:13:51.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 >> 172.21.15.57:0/2710210208 conn(0x55e2e1a24210 msgr2=0x55e2e1ac5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 shutdown_connections 2023-10-06T22:13:51.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.916+0000 7f99ddea5b80 1 -- 172.21.15.57:0/2710210208 wait complete. 2023-10-06T22:13:51.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ddea5b80 20 client.15021 trim_cache size 0 max 16384 2023-10-06T22:13:51.925 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:13:51.925 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24553, 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 68.201796639999998, 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:13:51.926 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:13:51.927 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 82.271390412000002 2023-10-06T22:13:51.927 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:13:51.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f99ac0a1590 msgr2=0x7f99ac0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f99ac0a1590 0x7f99ac0a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f99bc003ee0 tx=0x7f99bc009100 comp rx=0 tx=0).stop 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4100ea0 msgr2=0x7f99c4196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4100ea0 0x7f99c4196c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99b400b680 tx=0x7f99b4005740 comp rx=0 tx=0).stop 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 shutdown_connections 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f99ac0a1590 0x7f99ac0a3a50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99c4102a40 0x7f99c419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c4100ea0 0x7f99c4196c90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 --2- 172.21.15.57:0/2775718494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c4102080 0x7f99c41971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 >> 172.21.15.57:0/2775718494 conn(0x7f99c40fc460 msgr2=0x7f99c4105640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:51.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 shutdown_connections 2023-10-06T22:13:51.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:51.917+0000 7f99ca5ff700 1 -- 172.21.15.57:0/2775718494 wait complete. 2023-10-06T22:13:51.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:51 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:13:51.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:51 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:13:51.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:51 smithi057 ceph-mon[98492]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:51.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:51 smithi057 ceph-mon[98492]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:13:51.937 INFO:tasks.cephadm:Setup complete, yielding 2023-10-06T22:13:51.937 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-06T22:13:51.947 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:13:51.947 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-06T22:13:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:51 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:13:52.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:51 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:13:52.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:51 smithi190 ceph-mon[134891]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:52.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:51 smithi190 ceph-mon[134891]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:13:52.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:51 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:13:52.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:51 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:13:52.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:51 smithi156 ceph-mon[134358]: from='mgr.14150 172.21.15.57:0/143916195' entity='mgr.x' 2023-10-06T22:13:52.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:51 smithi156 ceph-mon[134358]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:13:52.681 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 -- 172.21.15.57:0/3027277026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0100f70 msgr2=0x7efdb0101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 --2- 172.21.15.57:0/3027277026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0100f70 0x7efdb0101390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efda00099c0 tx=0x7efda0009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 -- 172.21.15.57:0/3027277026 shutdown_connections 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 --2- 172.21.15.57:0/3027277026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb0102b10 0x7efdb01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 --2- 172.21.15.57:0/3027277026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0100f70 0x7efdb0101390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 --2- 172.21.15.57:0/3027277026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdb0102170 0x7efdb01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 -- 172.21.15.57:0/3027277026 >> 172.21.15.57:0/3027277026 conn(0x7efdb00fc4d0 msgr2=0x7efdb00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 -- 172.21.15.57:0/3027277026 shutdown_connections 2023-10-06T22:13:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.291+0000 7efdb645a700 1 -- 172.21.15.57:0/3027277026 wait complete. 2023-10-06T22:13:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.292+0000 7efdb645a700 1 Processor -- start 2023-10-06T22:13:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.292+0000 7efdb645a700 1 -- start start 2023-10-06T22:13:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb645a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb0100f70 0x7efdb0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb645a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdb0102170 0x7efdb00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb645a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0102b10 0x7efdb00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb645a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0073f60 con 0x7efdb0102b10 2023-10-06T22:13:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb645a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00740d0 con 0x7efdb0102170 2023-10-06T22:13:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb645a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb01a0cf0 con 0x7efdb0100f70 2023-10-06T22:13:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb49f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0102b10 0x7efdb00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb49f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0102b10 0x7efdb00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49692/0 (socket says 172.21.15.57:49692) 2023-10-06T22:13:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdb49f7700 1 -- 172.21.15.57:0/580405030 learned_addr learned my addr 172.21.15.57:0/580405030 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdaffff700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb0100f70 0x7efdb0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.293+0000 7efdaf7fe700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdb0102170 0x7efdb00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb49f7700 1 -- 172.21.15.57:0/580405030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb0100f70 msgr2=0x7efdb0074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb49f7700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb0100f70 0x7efdb0074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb49f7700 1 -- 172.21.15.57:0/580405030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdb0102170 msgr2=0x7efdb00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb49f7700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdb0102170 0x7efdb00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb49f7700 1 -- 172.21.15.57:0/580405030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00095f0 con 0x7efdb0102b10 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdaffff700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb0100f70 0x7efdb0074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb49f7700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0102b10 0x7efdb00737e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efda8007ee0 tx=0x7efda800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdad7fa700 1 -- 172.21.15.57:0/580405030 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8017070 con 0x7efdb0102b10 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdad7fa700 1 -- 172.21.15.57:0/580405030 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efda800f040 con 0x7efdb0102b10 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb01a0fd0 con 0x7efdb0102b10 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdad7fa700 1 -- 172.21.15.57:0/580405030 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8005580 con 0x7efdb0102b10 2023-10-06T22:13:54.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.294+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb01a13f0 con 0x7efdb0102b10 2023-10-06T22:13:54.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.295+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb0108560 con 0x7efdb0102b10 2023-10-06T22:13:54.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.297+0000 7efdad7fa700 1 -- 172.21.15.57:0/580405030 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7efda800db20 con 0x7efdb0102b10 2023-10-06T22:13:54.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.299+0000 7efdad7fa700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efd980a1510 0x7efd980a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:54.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.299+0000 7efdad7fa700 1 -- 172.21.15.57:0/580405030 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7efda80cc620 con 0x7efdb0102b10 2023-10-06T22:13:54.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.299+0000 7efdaffff700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efd980a1510 0x7efd980a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:54.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.300+0000 7efdaffff700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efd980a1510 0x7efd980a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efda0009990 tx=0x7efda0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:54.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.303+0000 7efdad7fa700 1 -- 172.21.15.57:0/580405030 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efda8095af0 con 0x7efdb0102b10 2023-10-06T22:13:54.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.531+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7efdb002d200 con 0x7efd980a1510 2023-10-06T22:13:54.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.532+0000 7efdad7fa700 1 -- 172.21.15.57:0/580405030 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7efdb002d200 con 0x7efd980a1510 2023-10-06T22:13:54.533 INFO:teuthology.orchestra.run.smithi057.stdout:Backend: cephadm 2023-10-06T22:13:54.533 INFO:teuthology.orchestra.run.smithi057.stdout:Available: Yes 2023-10-06T22:13:54.533 INFO:teuthology.orchestra.run.smithi057.stdout:Paused: No 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efd980a1510 msgr2=0x7efd980a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efd980a1510 0x7efd980a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efda0009990 tx=0x7efda0005fb0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0102b10 msgr2=0x7efdb00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0102b10 0x7efdb00737e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efda8007ee0 tx=0x7efda800d3b0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 shutdown_connections 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efd980a1510 0x7efd980a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdb0102b10 0x7efdb00737e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb0100f70 0x7efdb0074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 --2- 172.21.15.57:0/580405030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdb0102170 0x7efdb00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:54.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 >> 172.21.15.57:0/580405030 conn(0x7efdb00fc4d0 msgr2=0x7efdb01056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:54.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.534+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 shutdown_connections 2023-10-06T22:13:54.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:54.535+0000 7efdb645a700 1 -- 172.21.15.57:0/580405030 wait complete. 2023-10-06T22:13:54.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:54 smithi057 ceph-mon[98492]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:13:55.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:54 smithi156 ceph-mon[134358]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:13:55.062 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-06T22:13:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:54 smithi190 ceph-mon[134891]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:13:55.800 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:56.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:55 smithi156 ceph-mon[134358]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:13:56.064 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:55 smithi057 ceph-mon[98492]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:13:56.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:55 smithi190 ceph-mon[134891]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:13:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:56 smithi156 ceph-mon[134358]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:56 smithi057 ceph-mon[98492]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:56 smithi190 ceph-mon[134891]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:13:57.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 -- 172.21.15.57:0/900998059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102170 msgr2=0x7f13f41025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:57.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/900998059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102170 0x7f13f41025d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13e40099c0 tx=0x7f13e4009cd0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 -- 172.21.15.57:0/900998059 shutdown_connections 2023-10-06T22:13:57.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/900998059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f4102b10 0x7f13f41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/900998059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f4100f70 0x7f13f4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/900998059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102170 0x7f13f41025d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 -- 172.21.15.57:0/900998059 >> 172.21.15.57:0/900998059 conn(0x7f13f40fc4d0 msgr2=0x7f13f40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:57.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.288+0000 7f13fbb0f700 1 -- 172.21.15.57:0/900998059 shutdown_connections 2023-10-06T22:13:57.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.289+0000 7f13fbb0f700 1 -- 172.21.15.57:0/900998059 wait complete. 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.289+0000 7f13fbb0f700 1 Processor -- start 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fbb0f700 1 -- start start 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fbb0f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f4100f70 0x7f13f4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fbb0f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f4102170 0x7f13f4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fbb0f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102b10 0x7f13f41a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fbb0f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f41a0990 con 0x7f13f4102b10 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fbb0f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f41a0ad0 con 0x7f13f4100f70 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fbb0f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f41a0c60 con 0x7f13f4102170 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fa0ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102b10 0x7f13f41a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:57.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fa0ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102b10 0x7f13f41a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49712/0 (socket says 172.21.15.57:49712) 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13fa0ac700 1 -- 172.21.15.57:0/764796266 learned_addr learned my addr 172.21.15.57:0/764796266 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13f90aa700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f4102170 0x7f13f4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.290+0000 7f13f98ab700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f4100f70 0x7f13f4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13fa0ac700 1 -- 172.21.15.57:0/764796266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f4102170 msgr2=0x7f13f4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13fa0ac700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f4102170 0x7f13f4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13fa0ac700 1 -- 172.21.15.57:0/764796266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f4100f70 msgr2=0x7f13f4198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13fa0ac700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f4100f70 0x7f13f4198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13fa0ac700 1 -- 172.21.15.57:0/764796266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec009c40 con 0x7f13f4102b10 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13fa0ac700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102b10 0x7f13f41a03c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00c3a0 tx=0x7f13ec00c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13eaffd700 1 -- 172.21.15.57:0/764796266 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec017070 con 0x7f13f4102b10 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13eaffd700 1 -- 172.21.15.57:0/764796266 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ec00f040 con 0x7f13f4102b10 2023-10-06T22:13:57.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13eaffd700 1 -- 172.21.15.57:0/764796266 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00e050 con 0x7f13f4102b10 2023-10-06T22:13:57.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.291+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e40095f0 con 0x7f13f4102b10 2023-10-06T22:13:57.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.292+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f41a11a0 con 0x7f13f4102b10 2023-10-06T22:13:57.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.293+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f41935b0 con 0x7f13f4102b10 2023-10-06T22:13:57.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.294+0000 7f13eaffd700 1 -- 172.21.15.57:0/764796266 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f13ec0044c0 con 0x7f13f4102b10 2023-10-06T22:13:57.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.296+0000 7f13eaffd700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f13e00a1590 0x7f13e00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:13:57.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.296+0000 7f13eaffd700 1 -- 172.21.15.57:0/764796266 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f13ec0cc770 con 0x7f13f4102b10 2023-10-06T22:13:57.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.296+0000 7f13f98ab700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f13e00a1590 0x7f13e00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:13:57.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.297+0000 7f13f98ab700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f13e00a1590 0x7f13e00a3a50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f13f4101fd0 tx=0x7f13f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:13:57.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.300+0000 7f13eaffd700 1 -- 172.21.15.57:0/764796266 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f13ec095c40 con 0x7f13f4102b10 2023-10-06T22:13:57.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.520+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f13f402d2b0 con 0x7f13e00a1590 2023-10-06T22:13:57.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.534+0000 7f13eaffd700 1 -- 172.21.15.57:0/764796266 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f13f402d2b0 con 0x7f13e00a1590 2023-10-06T22:13:57.536 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-06T22:13:57.536 INFO:teuthology.orchestra.run.smithi057.stdout:mds.a smithi057 running (81s) 7s ago 81s 16.8M - 18.0.0-6616-g14d8d032 c754feb42fd5 86c3e142d113 2023-10-06T22:13:57.536 INFO:teuthology.orchestra.run.smithi057.stdout:mds.b smithi156 running (79s) 8s ago 79s 18.4M - 18.0.0-6616-g14d8d032 c754feb42fd5 836c3f2609a5 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.c smithi190 running (92s) 8s ago 92s 19.2M - 18.0.0-6616-g14d8d032 c754feb42fd5 c20a81c897dc 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.d smithi057 running (99s) 7s ago 99s 13.8M - 18.0.0-6616-g14d8d032 c754feb42fd5 f55de95e1dca 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.e smithi156 running (97s) 8s ago 97s 22.8M - 18.0.0-6616-g14d8d032 c754feb42fd5 83419adac93b 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.f smithi190 running (95s) 8s ago 94s 19.2M - 18.0.0-6616-g14d8d032 c754feb42fd5 92969380c533 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.g smithi057 running (85s) 7s ago 84s 16.9M - 18.0.0-6616-g14d8d032 c754feb42fd5 a33466e8eed0 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.h smithi156 running (90s) 8s ago 90s 21.4M - 18.0.0-6616-g14d8d032 c754feb42fd5 794c647a2c7a 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.i smithi190 running (88s) 8s ago 87s 30.2M - 18.0.0-6616-g14d8d032 c754feb42fd5 602cb53cc294 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.j smithi057 running (102s) 7s ago 102s 13.8M - 18.0.0-6616-g14d8d032 c754feb42fd5 0a6d1f5a750d 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.k smithi156 running (105s) 8s ago 105s 18.2M - 18.0.0-6616-g14d8d032 c754feb42fd5 0c8ab06a4d19 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mds.l smithi190 running (108s) 8s ago 107s 17.1M - 18.0.0-6616-g14d8d032 c754feb42fd5 7eb4897d6420 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi057 *:9283,8765 running (8m) 7s ago 8m 526M - 18.0.0-6616-g14d8d032 c754feb42fd5 b055af868061 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi156 *:8443,8765 running (6m) 8s ago 6m 452M - 18.0.0-6616-g14d8d032 c754feb42fd5 5b13adeaa1a3 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.z smithi190 *:8443,8765 running (6m) 8s ago 6m 452M - 18.0.0-6616-g14d8d032 c754feb42fd5 f94a1257c0c3 2023-10-06T22:13:57.537 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (8m) 7s ago 8m 92.9M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 3c01fa3d03fa 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi156 running (7m) 8s ago 7m 64.0M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 f12a53d717ac 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi190 running (7m) 8s ago 7m 65.4M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 f75ab992de01 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (6m) 7s ago 6m 63.2M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 48102e6cfbde 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi156 running (6m) 8s ago 6m 70.1M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 e8cf77b74a82 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi190 running (5m) 8s ago 5m 67.5M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 1977be4eab0e 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (5m) 7s ago 5m 60.5M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 8cbe97e571fa 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi156 running (4m) 8s ago 4m 57.8M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 43afa8526060 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi190 running (4m) 8s ago 4m 62.3M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 79d08b3eb2a7 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi057 running (4m) 7s ago 4m 59.8M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 b2a0e49ffc5c 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi156 running (3m) 8s ago 3m 64.4M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 ce418663c9d1 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 smithi190 running (3m) 8s ago 3m 60.3M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 a7b6116c5bf4 2023-10-06T22:13:57.538 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 smithi057 running (2m) 7s ago 2m 56.1M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 99d2059df032 2023-10-06T22:13:57.539 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 smithi156 running (2m) 8s ago 2m 60.5M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 a935b29c7b3a 2023-10-06T22:13:57.539 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 smithi190 running (2m) 8s ago 2m 56.6M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 b9726885a713 2023-10-06T22:13:57.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.536+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f13e00a1590 msgr2=0x7f13e00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:57.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.536+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f13e00a1590 0x7f13e00a3a50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f13f4101fd0 tx=0x7f13f0009260 comp rx=0 tx=0).stop 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.536+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102b10 msgr2=0x7f13f41a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.536+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102b10 0x7f13f41a03c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00c3a0 tx=0x7f13ec00c6b0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 shutdown_connections 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f13e00a1590 0x7f13e00a3a50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4102b10 0x7f13f41a03c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13f4100f70 0x7f13f4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 --2- 172.21.15.57:0/764796266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f4102170 0x7f13f4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 >> 172.21.15.57:0/764796266 conn(0x7f13f40fc4d0 msgr2=0x7f13f4107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 shutdown_connections 2023-10-06T22:13:57.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:13:57.537+0000 7f13fbb0f700 1 -- 172.21.15.57:0/764796266 wait complete. 2023-10-06T22:13:57.985 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-06T22:13:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:13:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:58 smithi156 ceph-mon[134358]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:13:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:13:58 smithi156 ceph-mon[134358]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:59.004 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:58 smithi057 ceph-mon[98492]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:13:59.005 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:13:58 smithi057 ceph-mon[98492]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:13:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:58 smithi190 ceph-mon[134891]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:13:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:13:58 smithi190 ceph-mon[134891]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:14:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 -- 172.21.15.57:0/337944044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 msgr2=0x7f25801025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 --2- 172.21.15.57:0/337944044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 0x7f25801025d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f25740099c0 tx=0x7f2574009cd0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 -- 172.21.15.57:0/337944044 shutdown_connections 2023-10-06T22:14:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 --2- 172.21.15.57:0/337944044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580102b10 0x7f25801093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 --2- 172.21.15.57:0/337944044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 0x7f2580101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 --2- 172.21.15.57:0/337944044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 0x7f25801025d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 -- 172.21.15.57:0/337944044 >> 172.21.15.57:0/337944044 conn(0x7f25800fc4d0 msgr2=0x7f25800fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 -- 172.21.15.57:0/337944044 shutdown_connections 2023-10-06T22:14:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.254+0000 7f25854e6700 1 -- 172.21.15.57:0/337944044 wait complete. 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 Processor -- start 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 -- start start 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 0x7f2580074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 0x7f25800732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580102b10 0x7f25800737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2580073f60 con 0x7f2580102170 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800740d0 con 0x7f2580102b10 2023-10-06T22:14:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.255+0000 7f25854e6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801a0cd0 con 0x7f2580100f70 2023-10-06T22:14:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 0x7f2580074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 0x7f2580074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57370/0 (socket says 172.21.15.57:57370) 2023-10-06T22:14:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 -- 172.21.15.57:0/2046423597 learned_addr learned my addr 172.21.15.57:0/2046423597 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257e7fc700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 0x7f25800732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257f7fe700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580102b10 0x7f25800737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 -- 172.21.15.57:0/2046423597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580102b10 msgr2=0x7f25800737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580102b10 0x7f25800737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 -- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 msgr2=0x7f25800732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 0x7f25800732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 -- 172.21.15.57:0/2046423597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740095f0 con 0x7f2580100f70 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.256+0000 7f257effd700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 0x7f2580074c50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f257000d980 tx=0x7f257000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.257+0000 7f2567fff700 1 -- 172.21.15.57:0/2046423597 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570004020 con 0x7f2580100f70 2023-10-06T22:14:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.257+0000 7f2567fff700 1 -- 172.21.15.57:0/2046423597 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257000f690 con 0x7f2580100f70 2023-10-06T22:14:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.257+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25801a0fb0 con 0x7f2580100f70 2023-10-06T22:14:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.257+0000 7f2567fff700 1 -- 172.21.15.57:0/2046423597 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570003a60 con 0x7f2580100f70 2023-10-06T22:14:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.257+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25801a13d0 con 0x7f2580100f70 2023-10-06T22:14:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.258+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258010c7c0 con 0x7f2580100f70 2023-10-06T22:14:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.260+0000 7f2567fff700 1 -- 172.21.15.57:0/2046423597 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f257000fb00 con 0x7f2580100f70 2023-10-06T22:14:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.261+0000 7f2567fff700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f25680a1590 0x7f25680a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.261+0000 7f2567fff700 1 -- 172.21.15.57:0/2046423597 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f25700cc9c0 con 0x7f2580100f70 2023-10-06T22:14:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.261+0000 7f257e7fc700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f25680a1590 0x7f25680a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.262+0000 7f257e7fc700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f25680a1590 0x7f25680a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f257400b580 tx=0x7f2574005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:00.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.263+0000 7f2567fff700 1 -- 172.21.15.57:0/2046423597 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2570095f10 con 0x7f2580100f70 2023-10-06T22:14:00.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.480+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f25800612f0 con 0x7f25680a1590 2023-10-06T22:14:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.486+0000 7f2567fff700 1 -- 172.21.15.57:0/2046423597 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f25800612f0 con 0x7f25680a1590 2023-10-06T22:14:00.488 INFO:teuthology.orchestra.run.smithi057.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-06T22:14:00.489 INFO:teuthology.orchestra.run.smithi057.stdout:mds.all 12/12 10s ago 113s smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-10-06T22:14:00.489 INFO:teuthology.orchestra.run.smithi057.stdout:mgr 3/3 10s ago 6m smithi057=x;smithi156=y;smithi190=z;count:3 2023-10-06T22:14:00.489 INFO:teuthology.orchestra.run.smithi057.stdout:mon 3/3 10s ago 7m smithi057:172.21.15.57=a;smithi156:172.21.15.156=b;smithi190:172.21.15.190=c;count:3 2023-10-06T22:14:00.489 INFO:teuthology.orchestra.run.smithi057.stdout:osd 12 10s ago - 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f25680a1590 msgr2=0x7f25680a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f25680a1590 0x7f25680a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f257400b580 tx=0x7f2574005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 msgr2=0x7f2580074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 0x7f2580074c50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f257000d980 tx=0x7f257000dc90 comp rx=0 tx=0).stop 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 shutdown_connections 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f25680a1590 0x7f25680a3a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2580102b10 0x7f25800737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2580100f70 0x7f2580074c50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.488+0000 7f25854e6700 1 --2- 172.21.15.57:0/2046423597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2580102170 0x7f25800732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.489+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 >> 172.21.15.57:0/2046423597 conn(0x7f25800fc4d0 msgr2=0x7f25800fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.489+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 shutdown_connections 2023-10-06T22:14:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:00.489+0000 7f25854e6700 1 -- 172.21.15.57:0/2046423597 wait complete. 2023-10-06T22:14:00.913 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:00 smithi057 ceph-mon[98492]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:14:01.002 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-06T22:14:01.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:00 smithi156 ceph-mon[134358]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:14:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:00 smithi190 ceph-mon[134891]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-10-06T22:14:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:14:02.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:01 smithi156 ceph-mon[134358]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:02.021 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:01 smithi057 ceph-mon[98492]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:01 smithi190 ceph-mon[134891]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:03.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:02 smithi156 ceph-mon[134358]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:14:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:02 smithi190 ceph-mon[134891]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:14:03.110 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:02 smithi057 ceph-mon[98492]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:14:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 -- 172.21.15.57:0/3807204645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 msgr2=0x7f3540101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 --2- 172.21.15.57:0/3807204645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540101390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f35300099c0 tx=0x7f3530009cd0 comp rx=0 tx=0).stop 2023-10-06T22:14:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 -- 172.21.15.57:0/3807204645 shutdown_connections 2023-10-06T22:14:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 --2- 172.21.15.57:0/3807204645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3540102b10 0x7f35401093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 --2- 172.21.15.57:0/3807204645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540101390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 --2- 172.21.15.57:0/3807204645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3540102170 0x7f35401025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 -- 172.21.15.57:0/3807204645 >> 172.21.15.57:0/3807204645 conn(0x7f35400fc4d0 msgr2=0x7f35400fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 -- 172.21.15.57:0/3807204645 shutdown_connections 2023-10-06T22:14:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.364+0000 7f354539d700 1 -- 172.21.15.57:0/3807204645 wait complete. 2023-10-06T22:14:03.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.365+0000 7f354539d700 1 Processor -- start 2023-10-06T22:14:03.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.365+0000 7f354539d700 1 -- start start 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f354539d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f354539d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3540102170 0x7f35400732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f354539d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3540102b10 0x7f35400737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f354539d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3540073f60 con 0x7f3540100f70 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f354539d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35400740d0 con 0x7f3540102b10 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f354539d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35401a0cf0 con 0x7f3540102170 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50838/0 (socket says 172.21.15.57:50838) 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 -- 172.21.15.57:0/243894589 learned_addr learned my addr 172.21.15.57:0/243894589 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353e7fc700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3540102170 0x7f35400732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353f7fe700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3540102b10 0x7f35400737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:03.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 -- 172.21.15.57:0/243894589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3540102170 msgr2=0x7f35400732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:03.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3540102170 0x7f35400732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:03.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 -- 172.21.15.57:0/243894589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3540102b10 msgr2=0x7f35400737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:03.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3540102b10 0x7f35400737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:03.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353effd700 1 -- 172.21.15.57:0/243894589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35300095f0 con 0x7f3540100f70 2023-10-06T22:14:03.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.366+0000 7f353e7fc700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3540102170 0x7f35400732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.367+0000 7f353effd700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540074c50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3530009990 tx=0x7f353000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.367+0000 7f3527fff700 1 -- 172.21.15.57:0/243894589 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353001d070 con 0x7f3540100f70 2023-10-06T22:14:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.367+0000 7f3527fff700 1 -- 172.21.15.57:0/243894589 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3530005030 con 0x7f3540100f70 2023-10-06T22:14:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.367+0000 7f3527fff700 1 -- 172.21.15.57:0/243894589 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530004a70 con 0x7f3540100f70 2023-10-06T22:14:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.367+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35401a0f70 con 0x7f3540100f70 2023-10-06T22:14:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.367+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35401a12d0 con 0x7f3540100f70 2023-10-06T22:14:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.368+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3540108560 con 0x7f3540100f70 2023-10-06T22:14:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.370+0000 7f3527fff700 1 -- 172.21.15.57:0/243894589 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3530005410 con 0x7f3540100f70 2023-10-06T22:14:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.372+0000 7f3527fff700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35280a1590 0x7f35280a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.373+0000 7f3527fff700 1 -- 172.21.15.57:0/243894589 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f35300cdb70 con 0x7f3540100f70 2023-10-06T22:14:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.373+0000 7f353e7fc700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35280a1590 0x7f35280a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.373+0000 7f353e7fc700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35280a1590 0x7f35280a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3534011490 tx=0x7f3534009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.377+0000 7f3527fff700 1 -- 172.21.15.57:0/243894589 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f35300970d0 con 0x7f3540100f70 2023-10-06T22:14:03.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.594+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f354010c7c0 con 0x7f35280a1590 2023-10-06T22:14:04.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.596+0000 7f3527fff700 1 -- 172.21.15.57:0/243894589 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f354010c7c0 con 0x7f35280a1590 2023-10-06T22:14:04.865 INFO:teuthology.orchestra.run.smithi057.stdout:HOST ADDR LABELS STATUS 2023-10-06T22:14:04.865 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 172.21.15.57 2023-10-06T22:14:04.865 INFO:teuthology.orchestra.run.smithi057.stdout:smithi156 172.21.15.156 2023-10-06T22:14:04.865 INFO:teuthology.orchestra.run.smithi057.stdout:smithi190 172.21.15.190 2023-10-06T22:14:04.865 INFO:teuthology.orchestra.run.smithi057.stdout:3 hosts in cluster 2023-10-06T22:14:04.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35280a1590 msgr2=0x7f35280a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:04.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35280a1590 0x7f35280a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3534011490 tx=0x7f3534009260 comp rx=0 tx=0).stop 2023-10-06T22:14:04.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 msgr2=0x7f3540074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540074c50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3530009990 tx=0x7f353000f690 comp rx=0 tx=0).stop 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 shutdown_connections 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35280a1590 0x7f35280a3a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3540102b10 0x7f35400737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3540100f70 0x7f3540074c50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 --2- 172.21.15.57:0/243894589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3540102170 0x7f35400732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 >> 172.21.15.57:0/243894589 conn(0x7f35400fc4d0 msgr2=0x7f35401056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 shutdown_connections 2023-10-06T22:14:04.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:03.598+0000 7f354539d700 1 -- 172.21.15.57:0/243894589 wait complete. 2023-10-06T22:14:04.868 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-06T22:14:05.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:04 smithi156 ceph-mon[134358]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:05.019 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:04 smithi156 ceph-mon[134358]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:14:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:04 smithi190 ceph-mon[134891]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:04 smithi190 ceph-mon[134891]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:14:05.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:04 smithi057 ceph-mon[98492]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:05.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:04 smithi057 ceph-mon[98492]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-06T22:14:05.614 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:14:06.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:06 smithi057 ceph-mon[98492]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:14:06.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:06 smithi057 ceph-mon[98492]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-06T22:14:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:06 smithi156 ceph-mon[134358]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:14:07.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:06 smithi156 ceph-mon[134358]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-06T22:14:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:06 smithi190 ceph-mon[134891]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:14:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:06 smithi190 ceph-mon[134891]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-06T22:14:07.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.171+0000 7f05c8d64700 1 -- 172.21.15.57:0/3902000017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4102b10 msgr2=0x7f05c41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:07.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.171+0000 7f05c8d64700 1 --2- 172.21.15.57:0/3902000017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4102b10 0x7f05c41093c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b0d0 tx=0x7f05bc00b3e0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.172+0000 7f05c8d64700 1 -- 172.21.15.57:0/3902000017 shutdown_connections 2023-10-06T22:14:07.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.172+0000 7f05c8d64700 1 --2- 172.21.15.57:0/3902000017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4102b10 0x7f05c41093c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.172+0000 7f05c8d64700 1 --2- 172.21.15.57:0/3902000017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c4100f70 0x7f05c4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.172+0000 7f05c8d64700 1 --2- 172.21.15.57:0/3902000017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05c4102170 0x7f05c41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.172+0000 7f05c8d64700 1 -- 172.21.15.57:0/3902000017 >> 172.21.15.57:0/3902000017 conn(0x7f05c40fc4d0 msgr2=0x7f05c40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:07.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.172+0000 7f05c8d64700 1 -- 172.21.15.57:0/3902000017 shutdown_connections 2023-10-06T22:14:07.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.172+0000 7f05c8d64700 1 -- 172.21.15.57:0/3902000017 wait complete. 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.173+0000 7f05c8d64700 1 Processor -- start 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.173+0000 7f05c8d64700 1 -- start start 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.173+0000 7f05c8d64700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4100f70 0x7f05c4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.173+0000 7f05c8d64700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c4102170 0x7f05c40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.173+0000 7f05c8d64700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05c4102b10 0x7f05c40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c8d64700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4073f60 con 0x7f05c4100f70 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c8d64700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40740d0 con 0x7f05c4102b10 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c8d64700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41a0dc0 con 0x7f05c4102170 2023-10-06T22:14:07.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4100f70 0x7f05c4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:07.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4100f70 0x7f05c4074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50864/0 (socket says 172.21.15.57:50864) 2023-10-06T22:14:07.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 -- 172.21.15.57:0/2207374492 learned_addr learned my addr 172.21.15.57:0/2207374492 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:07.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c1d9b700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c4102170 0x7f05c40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:07.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c2d9d700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05c4102b10 0x7f05c40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:07.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 -- 172.21.15.57:0/2207374492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c4102170 msgr2=0x7f05c40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:07.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c4102170 0x7f05c40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 -- 172.21.15.57:0/2207374492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05c4102b10 msgr2=0x7f05c40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:07.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05c4102b10 0x7f05c40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 -- 172.21.15.57:0/2207374492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009d00 con 0x7f05c4100f70 2023-10-06T22:14:07.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.174+0000 7f05c259c700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4100f70 0x7f05c4074c50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f05b400d980 tx=0x7f05b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:07.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.175+0000 7f05b37fe700 1 -- 172.21.15.57:0/2207374492 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400f890 con 0x7f05c4100f70 2023-10-06T22:14:07.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.175+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c41a10a0 con 0x7f05c4100f70 2023-10-06T22:14:07.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.175+0000 7f05b37fe700 1 -- 172.21.15.57:0/2207374492 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b4010460 con 0x7f05c4100f70 2023-10-06T22:14:07.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.175+0000 7f05b37fe700 1 -- 172.21.15.57:0/2207374492 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b40195c0 con 0x7f05c4100f70 2023-10-06T22:14:07.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.175+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c41a15f0 con 0x7f05c4100f70 2023-10-06T22:14:07.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.176+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c41084b0 con 0x7f05c4100f70 2023-10-06T22:14:07.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.178+0000 7f05b37fe700 1 -- 172.21.15.57:0/2207374492 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f05b40105d0 con 0x7f05c4100f70 2023-10-06T22:14:07.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.179+0000 7f05b37fe700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05ac0a1480 0x7f05ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:07.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.180+0000 7f05b37fe700 1 -- 172.21.15.57:0/2207374492 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f05b40cca70 con 0x7f05c4100f70 2023-10-06T22:14:07.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.180+0000 7f05c1d9b700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05ac0a1480 0x7f05ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:07.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.180+0000 7f05c1d9b700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05ac0a1480 0x7f05ac0a3940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f05b8005950 tx=0x7f05b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:07.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.183+0000 7f05b37fe700 1 -- 172.21.15.57:0/2207374492 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f05b4096100 con 0x7f05c4100f70 2023-10-06T22:14:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.400+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f05c402d160 con 0x7f05ac0a1480 2023-10-06T22:14:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.402+0000 7f05b37fe700 1 -- 172.21.15.57:0/2207374492 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f05c402d160 con 0x7f05ac0a1480 2023-10-06T22:14:07.405 INFO:teuthology.orchestra.run.smithi057.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-06T22:14:07.405 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE7361049H4P0KGN 3726G No 2m ago Has a FileSystem, LVM detected 2023-10-06T22:14:07.405 INFO:teuthology.orchestra.run.smithi057.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-10-06T22:14:07.405 INFO:teuthology.orchestra.run.smithi057.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05ac0a1480 msgr2=0x7f05ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05ac0a1480 0x7f05ac0a3940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f05b8005950 tx=0x7f05b800b410 comp rx=0 tx=0).stop 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4100f70 msgr2=0x7f05c4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4100f70 0x7f05c4074c50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f05b400d980 tx=0x7f05b400dc90 comp rx=0 tx=0).stop 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 shutdown_connections 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05ac0a1480 0x7f05ac0a3940 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05c4102b10 0x7f05c40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05c4100f70 0x7f05c4074c50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 --2- 172.21.15.57:0/2207374492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c4102170 0x7f05c40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:07.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.404+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 >> 172.21.15.57:0/2207374492 conn(0x7f05c40fc4d0 msgr2=0x7f05c41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:07.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.405+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 shutdown_connections 2023-10-06T22:14:07.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:07.405+0000 7f05c8d64700 1 -- 172.21.15.57:0/2207374492 wait complete. 2023-10-06T22:14:07.923 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-06T22:14:07.932 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:14:07.933 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-06T22:14:08.672 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:14:09.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:08 smithi156 ceph-mon[134358]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:09.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:08 smithi156 ceph-mon[134358]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:14:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:08 smithi190 ceph-mon[134891]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:08 smithi190 ceph-mon[134891]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:14:09.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:08 smithi057 ceph-mon[98492]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:09.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:08 smithi057 ceph-mon[98492]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:14:10.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 -- 172.21.15.57:0/2526275477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 msgr2=0x7fe51c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 --2- 172.21.15.57:0/2526275477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1025d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0099c0 tx=0x7fe50c009cd0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 -- 172.21.15.57:0/2526275477 shutdown_connections 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 --2- 172.21.15.57:0/2526275477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe51c102b10 0x7fe51c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 --2- 172.21.15.57:0/2526275477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe51c100f70 0x7fe51c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 --2- 172.21.15.57:0/2526275477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1025d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 -- 172.21.15.57:0/2526275477 >> 172.21.15.57:0/2526275477 conn(0x7fe51c0fc4d0 msgr2=0x7fe51c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.203+0000 7fe523a7b700 1 -- 172.21.15.57:0/2526275477 shutdown_connections 2023-10-06T22:14:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.204+0000 7fe523a7b700 1 -- 172.21.15.57:0/2526275477 wait complete. 2023-10-06T22:14:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 Processor -- start 2023-10-06T22:14:10.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 -- start start 2023-10-06T22:14:10.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe51c100f70 0x7fe51c198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe51c102b10 0x7fe51c1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c1a09f0 con 0x7fe51c102170 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c1a0b30 con 0x7fe51c102b10 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe523a7b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c1a0cc0 con 0x7fe51c100f70 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe521016700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe521016700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50882/0 (socket says 172.21.15.57:50882) 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.205+0000 7fe521817700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe51c100f70 0x7fe51c198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe521016700 1 -- 172.21.15.57:0/3617935265 learned_addr learned my addr 172.21.15.57:0/3617935265 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe522018700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe51c102b10 0x7fe51c1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe521016700 1 -- 172.21.15.57:0/3617935265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe51c100f70 msgr2=0x7fe51c198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe521016700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe51c100f70 0x7fe51c198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe521016700 1 -- 172.21.15.57:0/3617935265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe51c102b10 msgr2=0x7fe51c1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe521016700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe51c102b10 0x7fe51c1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe521016700 1 -- 172.21.15.57:0/3617935265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0095f0 con 0x7fe51c102170 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe521016700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1993a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0098e0 tx=0x7fe50c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe512ffd700 1 -- 172.21.15.57:0/3617935265 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c01d070 con 0x7fe51c102170 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.206+0000 7fe512ffd700 1 -- 172.21.15.57:0/3617935265 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50c005030 con 0x7fe51c102170 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.207+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c1a0f40 con 0x7fe51c102170 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.207+0000 7fe512ffd700 1 -- 172.21.15.57:0/3617935265 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004a70 con 0x7fe51c102170 2023-10-06T22:14:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.207+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c1a1350 con 0x7fe51c102170 2023-10-06T22:14:10.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.208+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c10c7c0 con 0x7fe51c102170 2023-10-06T22:14:10.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.210+0000 7fe512ffd700 1 -- 172.21.15.57:0/3617935265 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe50c0051a0 con 0x7fe51c102170 2023-10-06T22:14:10.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.211+0000 7fe512ffd700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1500 0x7fe5080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:10.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.211+0000 7fe512ffd700 1 -- 172.21.15.57:0/3617935265 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fe50c0cdaa0 con 0x7fe51c102170 2023-10-06T22:14:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.211+0000 7fe521817700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1500 0x7fe5080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.212+0000 7fe521817700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1500 0x7fe5080a39c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe51c101fd0 tx=0x7fe518009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:10.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.215+0000 7fe512ffd700 1 -- 172.21.15.57:0/3617935265 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe50c097160 con 0x7fe51c102170 2023-10-06T22:14:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.468+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe51c1a1600 con 0x7fe51c102170 2023-10-06T22:14:10.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.468+0000 7fe512ffd700 1 -- 172.21.15.57:0/3617935265 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2971 (secure 0 0 0) 0x7fe50c09a780 con 0x7fe51c102170 2023-10-06T22:14:10.471 INFO:teuthology.orchestra.run.smithi057.stdout:e26 2023-10-06T22:14:10.471 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T22:14:10.471 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T22:14:10.471 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2023-10-06T22:14:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:10.471 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 26 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:created 2023-10-06T22:12:43.212955+0000 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2023-10-06T22:12:50.650753+0000 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 300 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2023-10-06T22:14:10.472 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 0 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 5 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1,2,3,4 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=24553,1=24551,2=24529,3=24517,4=24509} 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3] 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2023-10-06T22:14:10.473 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 0 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.i{0:24553} state up:active seq 5 addr [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.e{0:24482} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/2797676784,v1:172.21.15.156:6835/2797676784] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b{1:24551} state up:active seq 4 addr [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.l{1:24463} state up:standby-replay seq 1 addr [v2:172.21.15.190:6832/1813464152,v1:172.21.15.190:6833/1813464152] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c{2:24529} state up:active seq 7 addr [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.k{2:24446} state up:standby-replay seq 1 addr [v2:172.21.15.156:6832/1142570288,v1:172.21.15.156:6833/1142570288] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.f{3:24517} state up:active seq 8 addr [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a{3:14535} state up:standby-replay seq 1 addr [v2:172.21.15.57:6840/490100321,v1:172.21.15.57:6841/490100321] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.474 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.h{4:24509} state up:active seq 7 addr [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.g{4:14523} state up:standby-replay seq 1 addr [v2:172.21.15.57:6838/3033019918,v1:172.21.15.57:6839/3033019918] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:10.475 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2023-10-06T22:14:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:10.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.j{-1:14463} state up:standby seq 1 addr [v2:172.21.15.57:6834/1029690870,v1:172.21.15.57:6835/1029690870] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.475 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d{-1:14475} state up:standby seq 1 addr [v2:172.21.15.57:6836/2389529664,v1:172.21.15.57:6837/2389529664] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T22:14:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1500 msgr2=0x7fe5080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1500 0x7fe5080a39c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe51c101fd0 tx=0x7fe518009260 comp rx=0 tx=0).stop 2023-10-06T22:14:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 msgr2=0x7fe51c1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1993a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0098e0 tx=0x7fe50c00f690 comp rx=0 tx=0).stop 2023-10-06T22:14:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 shutdown_connections 2023-10-06T22:14:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe5080a1500 0x7fe5080a39c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe51c102b10 0x7fe51c1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe51c100f70 0x7fe51c198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 --2- 172.21.15.57:0/3617935265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe51c102170 0x7fe51c1993a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:10.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 >> 172.21.15.57:0/3617935265 conn(0x7fe51c0fc4d0 msgr2=0x7fe51c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:10.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 shutdown_connections 2023-10-06T22:14:10.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:10.471+0000 7fe523a7b700 1 -- 172.21.15.57:0/3617935265 wait complete. 2023-10-06T22:14:10.477 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:14:10.952 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-06T22:14:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:10 smithi190 ceph-mon[134891]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2023-10-06T22:14:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:10 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3617935265' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T22:14:11.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:10 smithi057 ceph-mon[98492]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2023-10-06T22:14:11.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:10 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3617935265' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T22:14:11.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:10 smithi156 ceph-mon[134358]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2023-10-06T22:14:11.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:10 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3617935265' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T22:14:11.772 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:14:12.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:12 smithi057 ceph-mon[98492]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-06T22:14:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:12 smithi190 ceph-mon[134891]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-06T22:14:13.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:12 smithi156 ceph-mon[134358]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 -- 172.21.15.57:0/2576284340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608100f70 msgr2=0x7f0608101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 --2- 172.21.15.57:0/2576284340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608100f70 0x7f0608101390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f05f80099c0 tx=0x7f05f8009cd0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 -- 172.21.15.57:0/2576284340 shutdown_connections 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 --2- 172.21.15.57:0/2576284340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0608102b10 0x7f06081093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 --2- 172.21.15.57:0/2576284340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608100f70 0x7f0608101390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 --2- 172.21.15.57:0/2576284340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608102170 0x7f06081025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 -- 172.21.15.57:0/2576284340 >> 172.21.15.57:0/2576284340 conn(0x7f06080fc4d0 msgr2=0x7f06080fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:13.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 -- 172.21.15.57:0/2576284340 shutdown_connections 2023-10-06T22:14:13.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.269+0000 7f060df46700 1 -- 172.21.15.57:0/2576284340 wait complete. 2023-10-06T22:14:13.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.270+0000 7f060df46700 1 Processor -- start 2023-10-06T22:14:13.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f060df46700 1 -- start start 2023-10-06T22:14:13.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f060df46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608100f70 0x7f0608074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:13.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f060df46700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608102170 0x7f06080732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:13.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f060df46700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0608102b10 0x7f06080737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:13.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f060df46700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819aab0 con 0x7f0608102170 2023-10-06T22:14:13.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f060df46700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0608073d20 con 0x7f0608102b10 2023-10-06T22:14:13.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f060df46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0608073ee0 con 0x7f0608100f70 2023-10-06T22:14:13.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f06077fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608100f70 0x7f0608074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:13.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f0606ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608102170 0x7f06080732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:13.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f06077fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608100f70 0x7f0608074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45212/0 (socket says 172.21.15.57:45212) 2023-10-06T22:14:13.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f06077fe700 1 -- 172.21.15.57:0/1079070324 learned_addr learned my addr 172.21.15.57:0/1079070324 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:13.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.271+0000 7f0607fff700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0608102b10 0x7f06080737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:13.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f06077fe700 1 -- 172.21.15.57:0/1079070324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0608102b10 msgr2=0x7f06080737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f06077fe700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0608102b10 0x7f06080737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f06077fe700 1 -- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608102170 msgr2=0x7f06080732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f06077fe700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608102170 0x7f06080732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f06077fe700 1 -- 172.21.15.57:0/1079070324 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80095f0 con 0x7f0608100f70 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f06077fe700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608100f70 0x7f0608074c50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009fd0 tx=0x7f05f8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f0604ff9700 1 -- 172.21.15.57:0/1079070324 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f801d070 con 0x7f0608100f70 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.272+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06081a5480 con 0x7f0608100f70 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.273+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06081a57e0 con 0x7f0608100f70 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.273+0000 7f0604ff9700 1 -- 172.21.15.57:0/1079070324 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05f8005030 con 0x7f0608100f70 2023-10-06T22:14:13.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.273+0000 7f0604ff9700 1 -- 172.21.15.57:0/1079070324 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004a90 con 0x7f0608100f70 2023-10-06T22:14:13.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.274+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0608108550 con 0x7f0608100f70 2023-10-06T22:14:13.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.276+0000 7f0604ff9700 1 -- 172.21.15.57:0/1079070324 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f05f80051a0 con 0x7f0608100f70 2023-10-06T22:14:13.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.277+0000 7f0604ff9700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05f00a1480 0x7f05f00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:13.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.277+0000 7f0604ff9700 1 -- 172.21.15.57:0/1079070324 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f05f80cdb60 con 0x7f0608100f70 2023-10-06T22:14:13.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.277+0000 7f0606ffd700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05f00a1480 0x7f05f00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:13.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.278+0000 7f0606ffd700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05f00a1480 0x7f05f00a3940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009c60 tx=0x7f05fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:13.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.279+0000 7f0604ff9700 1 -- 172.21.15.57:0/1079070324 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f05f8097220 con 0x7f0608100f70 2023-10-06T22:14:13.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.489+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f060810c970 con 0x7f0608100f70 2023-10-06T22:14:13.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.491+0000 7f0604ff9700 1 -- 172.21.15.57:0/1079070324 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5259 (secure 0 0 0) 0x7f05f809a840 con 0x7f0608100f70 2023-10-06T22:14:13.493 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 85 2023-10-06T22:14:13.493 INFO:teuthology.orchestra.run.smithi057.stdout:fsid 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:14:13.493 INFO:teuthology.orchestra.run.smithi057.stdout:created 2023-10-06T22:05:04.289350+0000 2023-10-06T22:14:13.493 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2023-10-06T22:13:06.715910+0000 2023-10-06T22:14:13.493 INFO:teuthology.orchestra.run.smithi057.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-06T22:14:13.493 INFO:teuthology.orchestra.run.smithi057.stdout:crush_version 26 2023-10-06T22:14:13.493 INFO:teuthology.orchestra.run.smithi057.stdout:full_ratio 0.95 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:backfillfull_ratio 0.9 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:nearfull_ratio 0.85 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:require_min_compat_client luminous 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:min_compat_client luminous 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:require_osd_release reef 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:stretch_mode_enabled false 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:max_osd 12 2023-10-06T22:14:13.494 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6802/2194428703,v1:172.21.15.57:6803/2194428703] [v2:172.21.15.57:6804/2194428703,v1:172.21.15.57:6805/2194428703] exists,up 25492bd6-3367-475f-b073-59bd613eab8a 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/2038383629,v1:172.21.15.156:6801/2038383629] [v2:172.21.15.156:6802/2038383629,v1:172.21.15.156:6803/2038383629] exists,up b7f3e993-648c-4873-9ecc-fee568a3a076 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/445057088,v1:172.21.15.190:6801/445057088] [v2:172.21.15.190:6802/445057088,v1:172.21.15.190:6803/445057088] exists,up 4ce3125d-3f65-4413-9fcb-486f5b565ad5 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6810/140569851,v1:172.21.15.57:6811/140569851] [v2:172.21.15.57:6812/140569851,v1:172.21.15.57:6813/140569851] exists,up 8df05b17-2d03-43d9-a57d-5abed0d51e65 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/3180512737,v1:172.21.15.156:6809/3180512737] [v2:172.21.15.156:6810/3180512737,v1:172.21.15.156:6811/3180512737] exists,up 48270831-4102-4536-b87e-6f3373e1125c 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/3750215386,v1:172.21.15.190:6809/3750215386] [v2:172.21.15.190:6810/3750215386,v1:172.21.15.190:6811/3750215386] exists,up 9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 up in weight 1 up_from 46 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6818/2530618700,v1:172.21.15.57:6819/2530618700] [v2:172.21.15.57:6820/2530618700,v1:172.21.15.57:6821/2530618700] exists,up a987a0ba-b9f4-43e5-be83-274279f3b00c 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 up in weight 1 up_from 51 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/2039885607,v1:172.21.15.156:6817/2039885607] [v2:172.21.15.156:6818/2039885607,v1:172.21.15.156:6819/2039885607] exists,up e4b99ae3-8f83-4e52-a9c0-9536579dc262 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 up in weight 1 up_from 58 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/1850516582,v1:172.21.15.190:6817/1850516582] [v2:172.21.15.190:6818/1850516582,v1:172.21.15.190:6819/1850516582] exists,up ae70781d-2670-4606-a280-604141f62861 2023-10-06T22:14:13.495 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6826/377680664,v1:172.21.15.57:6827/377680664] [v2:172.21.15.57:6828/377680664,v1:172.21.15.57:6829/377680664] exists,up ea6cc1cb-92e6-4cf1-a891-ca82e3e20079 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 up in weight 1 up_from 71 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/1227399112,v1:172.21.15.156:6825/1227399112] [v2:172.21.15.156:6826/1227399112,v1:172.21.15.156:6827/1227399112] exists,up d591b412-f98a-438b-bd2c-fd491fe9cf11 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/826697758,v1:172.21.15.190:6825/826697758] [v2:172.21.15.190:6826/826697758,v1:172.21.15.190:6827/826697758] exists,up 88d41d0c-a9d4-45ea-9a40-6612ee98597e 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.4 [2,11] 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.1d [2,4,8,11] 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.1f [8,1] 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.e [4,8] 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.25 [5,9] 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.34 [4,9] 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.38 [3,9] 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/3599227077 expires 2023-10-07T22:06:05.527819+0000 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/4144294466 expires 2023-10-07T22:06:05.527819+0000 2023-10-06T22:14:13.496 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/4144294466 expires 2023-10-07T22:06:05.527819+0000 2023-10-06T22:14:13.497 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1942116136 expires 2023-10-07T22:05:38.356695+0000 2023-10-06T22:14:13.497 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/3830563323 expires 2023-10-07T22:06:05.527819+0000 2023-10-06T22:14:13.497 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/1962312800 expires 2023-10-07T22:05:38.356695+0000 2023-10-06T22:14:13.497 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/1962312800 expires 2023-10-07T22:05:38.356695+0000 2023-10-06T22:14:13.497 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/3188291259 expires 2023-10-07T22:05:38.356695+0000 2023-10-06T22:14:13.497 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/3827189862 expires 2023-10-07T22:05:38.356695+0000 2023-10-06T22:14:13.497 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/2091122692 expires 2023-10-07T22:06:05.527819+0000 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05f00a1480 msgr2=0x7f05f00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05f00a1480 0x7f05f00a3940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009c60 tx=0x7f05fc009260 comp rx=0 tx=0).stop 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608100f70 msgr2=0x7f0608074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608100f70 0x7f0608074c50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009fd0 tx=0x7f05f8009580 comp rx=0 tx=0).stop 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 shutdown_connections 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f05f00a1480 0x7f05f00a3940 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0608102b10 0x7f06080737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0608100f70 0x7f0608074c50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 --2- 172.21.15.57:0/1079070324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0608102170 0x7f06080732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 >> 172.21.15.57:0/1079070324 conn(0x7f06080fc4d0 msgr2=0x7f0608105690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 shutdown_connections 2023-10-06T22:14:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:13.493+0000 7f060df46700 1 -- 172.21.15.57:0/1079070324 wait complete. 2023-10-06T22:14:13.962 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-06T22:14:13.973 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:14:14.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:13 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1079070324' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T22:14:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:13 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1079070324' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 -- 172.21.15.57:0/3510333735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038106ba0 msgr2=0x7f4038109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 --2- 172.21.15.57:0/3510333735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038106ba0 0x7f4038109010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f403000b0a0 tx=0x7f403000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 -- 172.21.15.57:0/3510333735 shutdown_connections 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 --2- 172.21.15.57:0/3510333735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038106ba0 0x7f4038109010 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 --2- 172.21.15.57:0/3510333735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4038101940 0x7f4038103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 --2- 172.21.15.57:0/3510333735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4038104270 0x7f4038106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 -- 172.21.15.57:0/3510333735 >> 172.21.15.57:0/3510333735 conn(0x7f40380fb310 msgr2=0x7f40380fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 -- 172.21.15.57:0/3510333735 shutdown_connections 2023-10-06T22:14:14.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.179+0000 7f40407ff700 1 -- 172.21.15.57:0/3510333735 wait complete. 2023-10-06T22:14:14.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.180+0000 7f40407ff700 1 Processor -- start 2023-10-06T22:14:14.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.180+0000 7f40407ff700 1 -- start start 2023-10-06T22:14:14.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f40407ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038101940 0x7f4038196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f40407ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4038104270 0x7f40381971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f40407ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4038106ba0 0x7f403819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f40407ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403819e800 con 0x7f4038101940 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f40407ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403819e970 con 0x7f4038104270 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f40407ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403819eb30 con 0x7f4038106ba0 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038101940 0x7f4038196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038101940 0x7f4038196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42206/0 (socket says 172.21.15.57:42206) 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 -- 172.21.15.57:0/3392924140 learned_addr learned my addr 172.21.15.57:0/3392924140 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403ed9c700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4038106ba0 0x7f403819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403dd9a700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4038104270 0x7f40381971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 -- 172.21.15.57:0/3392924140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4038106ba0 msgr2=0x7f403819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4038106ba0 0x7f403819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 -- 172.21.15.57:0/3392924140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4038104270 msgr2=0x7f40381971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4038104270 0x7f40381971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403e59b700 1 -- 172.21.15.57:0/3392924140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030009d00 con 0x7f4038101940 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.181+0000 7f403dd9a700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4038104270 0x7f40381971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.182+0000 7f403e59b700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038101940 0x7f4038196c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f403400e9a0 tx=0x7f403400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.182+0000 7f402f7fe700 1 -- 172.21.15.57:0/3392924140 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034009790 con 0x7f4038101940 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.182+0000 7f402f7fe700 1 -- 172.21.15.57:0/3392924140 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f403400cbd0 con 0x7f4038101940 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.182+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403819ee10 con 0x7f4038101940 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.182+0000 7f402f7fe700 1 -- 172.21.15.57:0/3392924140 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034018680 con 0x7f4038101940 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.182+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403819f230 con 0x7f4038101940 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.183+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40380fce90 con 0x7f4038101940 2023-10-06T22:14:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.185+0000 7f402f7fe700 1 -- 172.21.15.57:0/3392924140 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f403400cd40 con 0x7f4038101940 2023-10-06T22:14:14.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.186+0000 7f402f7fe700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f40240a14d0 0x7f40240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.186+0000 7f402f7fe700 1 -- 172.21.15.57:0/3392924140 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f4034014070 con 0x7f4038101940 2023-10-06T22:14:14.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.187+0000 7f403dd9a700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f40240a14d0 0x7f40240a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.187+0000 7f403dd9a700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f40240a14d0 0x7f40240a3990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4038198190 tx=0x7f4028009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:14.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.190+0000 7f402f7fe700 1 -- 172.21.15.57:0/3392924140 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4034052100 con 0x7f4038101940 2023-10-06T22:14:14.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:13 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1079070324' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T22:14:14.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.404+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f403810c910 con 0x7f4038101940 2023-10-06T22:14:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.405+0000 7f402f7fe700 1 -- 172.21.15.57:0/3392924140 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7f40340d0030 con 0x7f4038101940 2023-10-06T22:14:14.408 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:14.408 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:14:14.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f40240a14d0 msgr2=0x7f40240a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f40240a14d0 0x7f40240a3990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4038198190 tx=0x7f4028009260 comp rx=0 tx=0).stop 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038101940 msgr2=0x7f4038196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038101940 0x7f4038196c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f403400e9a0 tx=0x7f403400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 shutdown_connections 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f40240a14d0 0x7f40240a3990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4038106ba0 0x7f403819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4038101940 0x7f4038196c90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 --2- 172.21.15.57:0/3392924140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4038104270 0x7f40381971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 >> 172.21.15.57:0/3392924140 conn(0x7f40380fb310 msgr2=0x7f40381025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 shutdown_connections 2023-10-06T22:14:14.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.408+0000 7f40407ff700 1 -- 172.21.15.57:0/3392924140 wait complete. 2023-10-06T22:14:14.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:14:14.429 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 -- 172.21.15.57:0/1064197836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 msgr2=0x7f4dc41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/1064197836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 0x7f4dc41012b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009990 tx=0x7f4dc0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 -- 172.21.15.57:0/1064197836 shutdown_connections 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/1064197836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102a30 0x7f4dc4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/1064197836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 0x7f4dc41012b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/1064197836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc4102070 0x7f4dc41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 -- 172.21.15.57:0/1064197836 >> 172.21.15.57:0/1064197836 conn(0x7f4dc40fc430 msgr2=0x7f4dc40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 -- 172.21.15.57:0/1064197836 shutdown_connections 2023-10-06T22:14:14.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.634+0000 7f4dcbbff700 1 -- 172.21.15.57:0/1064197836 wait complete. 2023-10-06T22:14:14.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.635+0000 7f4dcbbff700 1 Processor -- start 2023-10-06T22:14:14.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dcbbff700 1 -- start start 2023-10-06T22:14:14.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dcbbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 0x7f4dc4198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dcbbff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102070 0x7f4dc4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dcbbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc4102a30 0x7f4dc41a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dcbbff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc41a0960 con 0x7f4dc4102070 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dcbbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc41a0aa0 con 0x7f4dc4100e90 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dcbbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc41a0c30 con 0x7f4dc4102a30 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dc919a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102070 0x7f4dc4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dc919a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102070 0x7f4dc4199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42224/0 (socket says 172.21.15.57:42224) 2023-10-06T22:14:14.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dc919a700 1 -- 172.21.15.57:0/3463808433 learned_addr learned my addr 172.21.15.57:0/3463808433 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dc999b700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 0x7f4dc4198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.636+0000 7f4dca19c700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc4102a30 0x7f4dc41a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dc919a700 1 -- 172.21.15.57:0/3463808433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc4102a30 msgr2=0x7f4dc41a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dc919a700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc4102a30 0x7f4dc41a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dc919a700 1 -- 172.21.15.57:0/3463808433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 msgr2=0x7f4dc4198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dc919a700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 0x7f4dc4198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dc919a700 1 -- 172.21.15.57:0/3463808433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00095f0 con 0x7f4dc4102070 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dc919a700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102070 0x7f4dc4199320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e970 tx=0x7f4db400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dbaffd700 1 -- 172.21.15.57:0/3463808433 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db4009790 con 0x7f4dc4102070 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dbaffd700 1 -- 172.21.15.57:0/3463808433 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4db400c7e0 con 0x7f4dc4102070 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dbaffd700 1 -- 172.21.15.57:0/3463808433 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db4003c70 con 0x7f4dc4102070 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc41a0f10 con 0x7f4dc4102070 2023-10-06T22:14:14.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.637+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc41a1390 con 0x7f4dc4102070 2023-10-06T22:14:14.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.638+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc41934f0 con 0x7f4dc4102070 2023-10-06T22:14:14.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.640+0000 7f4dbaffd700 1 -- 172.21.15.57:0/3463808433 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4db400cc50 con 0x7f4dc4102070 2023-10-06T22:14:14.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.642+0000 7f4dbaffd700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4db00a1500 0x7f4db00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:14.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.642+0000 7f4dbaffd700 1 -- 172.21.15.57:0/3463808433 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f4db4014070 con 0x7f4dc4102070 2023-10-06T22:14:14.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.642+0000 7f4dc999b700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4db00a1500 0x7f4db00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:14.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.643+0000 7f4dc999b700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4db00a1500 0x7f4db00a39c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000b2e0 tx=0x7f4dc0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:14.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.645+0000 7f4dbaffd700 1 -- 172.21.15.57:0/3463808433 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4db4096480 con 0x7f4dc4102070 2023-10-06T22:14:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.857+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4dc402d270 con 0x7f4dc4102070 2023-10-06T22:14:14.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.857+0000 7f4dbaffd700 1 -- 172.21.15.57:0/3463808433 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4db4003dd0 con 0x7f4dc4102070 2023-10-06T22:14:14.860 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:14:14.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4db00a1500 msgr2=0x7f4db00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4db00a1500 0x7f4db00a39c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000b2e0 tx=0x7f4dc0005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102070 msgr2=0x7f4dc4199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:14.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102070 0x7f4dc4199320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e970 tx=0x7f4db400ec80 comp rx=0 tx=0).stop 2023-10-06T22:14:14.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 shutdown_connections 2023-10-06T22:14:14.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4db00a1500 0x7f4db00a39c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dc4102a30 0x7f4dc41a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc4100e90 0x7f4dc4198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 --2- 172.21.15.57:0/3463808433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dc4102070 0x7f4dc4199320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:14.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 >> 172.21.15.57:0/3463808433 conn(0x7f4dc40fc430 msgr2=0x7f4dc4107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:14.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 shutdown_connections 2023-10-06T22:14:14.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:14.860+0000 7f4dcbbff700 1 -- 172.21.15.57:0/3463808433 wait complete. 2023-10-06T22:14:14.882 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:14:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:14 smithi190 ceph-mon[134891]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-06T22:14:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:14 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3392924140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:15.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 -- 172.21.15.57:0/1855214145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394074ef0 msgr2=0x7fa394073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 --2- 172.21.15.57:0/1855214145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394074ef0 0x7fa394073340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa3840098f0 tx=0x7fa384009c00 comp rx=0 tx=0).stop 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 -- 172.21.15.57:0/1855214145 shutdown_connections 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 --2- 172.21.15.57:0/1855214145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394108e00 0x7fa39410b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 --2- 172.21.15.57:0/1855214145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394074ef0 0x7fa394073340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 --2- 172.21.15.57:0/1855214145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394073910 0x7fa394073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 -- 172.21.15.57:0/1855214145 >> 172.21.15.57:0/1855214145 conn(0x7fa3940fc4d0 msgr2=0x7fa3940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 -- 172.21.15.57:0/1855214145 shutdown_connections 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.089+0000 7fa399bff700 1 -- 172.21.15.57:0/1855214145 wait complete. 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 Processor -- start 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 -- start start 2023-10-06T22:14:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394073910 0x7fa394198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394074ef0 0x7fa3941993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394108e00 0x7fa3941a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941a0a10 con 0x7fa394074ef0 2023-10-06T22:14:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941a0b50 con 0x7fa394108e00 2023-10-06T22:14:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.090+0000 7fa399bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941a0ce0 con 0x7fa394073910 2023-10-06T22:14:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394074ef0 0x7fa3941993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394074ef0 0x7fa3941993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42248/0 (socket says 172.21.15.57:42248) 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 -- 172.21.15.57:0/2710763138 learned_addr learned my addr 172.21.15.57:0/2710763138 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa3937fe700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394073910 0x7fa394198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa393fff700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394108e00 0x7fa3941a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 -- 172.21.15.57:0/2710763138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394073910 msgr2=0x7fa394198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394073910 0x7fa394198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 -- 172.21.15.57:0/2710763138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394108e00 msgr2=0x7fa3941a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394108e00 0x7fa3941a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 -- 172.21.15.57:0/2710763138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa384009520 con 0x7fa394074ef0 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa392ffd700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394074ef0 0x7fa3941993d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e970 tx=0x7fa38800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa390ff9700 1 -- 172.21.15.57:0/2710763138 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388009790 con 0x7fa394074ef0 2023-10-06T22:14:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.091+0000 7fa390ff9700 1 -- 172.21.15.57:0/2710763138 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa38800c7e0 con 0x7fa394074ef0 2023-10-06T22:14:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.092+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3941a0fc0 con 0x7fa394074ef0 2023-10-06T22:14:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.092+0000 7fa390ff9700 1 -- 172.21.15.57:0/2710763138 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388003c70 con 0x7fa394074ef0 2023-10-06T22:14:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.092+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3941a1440 con 0x7fa394074ef0 2023-10-06T22:14:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.093+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3941074b0 con 0x7fa394074ef0 2023-10-06T22:14:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.095+0000 7fa390ff9700 1 -- 172.21.15.57:0/2710763138 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa38800cc50 con 0x7fa394074ef0 2023-10-06T22:14:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.097+0000 7fa390ff9700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa37c0a1480 0x7fa37c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.097+0000 7fa390ff9700 1 -- 172.21.15.57:0/2710763138 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fa388014070 con 0x7fa394074ef0 2023-10-06T22:14:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.097+0000 7fa3937fe700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa37c0a1480 0x7fa37c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.098+0000 7fa3937fe700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa37c0a1480 0x7fa37c0a3940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa384005e90 tx=0x7fa384019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.099+0000 7fa390ff9700 1 -- 172.21.15.57:0/2710763138 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa388096640 con 0x7fa394074ef0 2023-10-06T22:14:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:14 smithi057 ceph-mon[98492]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-06T22:14:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:14 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3392924140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:15.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:14 smithi156 ceph-mon[134358]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-06T22:14:15.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:14 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3392924140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:15.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.309+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa39410eaa0 con 0x7fa394074ef0 2023-10-06T22:14:15.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.310+0000 7fa390ff9700 1 -- 172.21.15.57:0/2710763138 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa388003dd0 con 0x7fa394074ef0 2023-10-06T22:14:15.312 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa37c0a1480 msgr2=0x7fa37c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa37c0a1480 0x7fa37c0a3940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa384005e90 tx=0x7fa384019040 comp rx=0 tx=0).stop 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394074ef0 msgr2=0x7fa3941993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394074ef0 0x7fa3941993d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e970 tx=0x7fa38800ec80 comp rx=0 tx=0).stop 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 shutdown_connections 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa37c0a1480 0x7fa37c0a3940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa394108e00 0x7fa3941a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394073910 0x7fa394198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 --2- 172.21.15.57:0/2710763138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa394074ef0 0x7fa3941993d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 >> 172.21.15.57:0/2710763138 conn(0x7fa3940fc4d0 msgr2=0x7fa3940fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:15.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 shutdown_connections 2023-10-06T22:14:15.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.312+0000 7fa399bff700 1 -- 172.21.15.57:0/2710763138 wait complete. 2023-10-06T22:14:15.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:14:15.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 -- 172.21.15.57:0/2183174785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 msgr2=0x7f8c8c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 --2- 172.21.15.57:0/2183174785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c109350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400b0a0 tx=0x7f8c8400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 -- 172.21.15.57:0/2183174785 shutdown_connections 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 --2- 172.21.15.57:0/2183174785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c109350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 --2- 172.21.15.57:0/2183174785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c100e90 0x7f8c8c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 --2- 172.21.15.57:0/2183174785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c8c102070 0x7f8c8c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 -- 172.21.15.57:0/2183174785 >> 172.21.15.57:0/2183174785 conn(0x7f8c8c0fc430 msgr2=0x7f8c8c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 -- 172.21.15.57:0/2183174785 shutdown_connections 2023-10-06T22:14:15.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.536+0000 7f8c90bff700 1 -- 172.21.15.57:0/2183174785 wait complete. 2023-10-06T22:14:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.537+0000 7f8c90bff700 1 Processor -- start 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.537+0000 7f8c90bff700 1 -- start start 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c90bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c8c100e90 0x7f8c8c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c90bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c102070 0x7f8c8c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c90bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c90bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c1a0a90 con 0x7f8c8c102a30 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c90bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c1a0bd0 con 0x7f8c8c100e90 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c90bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c1a0d60 con 0x7f8c8c102070 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c1a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42270/0 (socket says 172.21.15.57:42270) 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 -- 172.21.15.57:0/1205127951 learned_addr learned my addr 172.21.15.57:0/1205127951 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:15.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c89d9b700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c102070 0x7f8c8c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8a59c700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c8c100e90 0x7f8c8c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 -- 172.21.15.57:0/1205127951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c102070 msgr2=0x7f8c8c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c102070 0x7f8c8c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 -- 172.21.15.57:0/1205127951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c8c100e90 msgr2=0x7f8c8c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c8c100e90 0x7f8c8c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.538+0000 7f8c8ad9d700 1 -- 172.21.15.57:0/1205127951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c84009d00 con 0x7f8c8c102a30 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.539+0000 7f8c8ad9d700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c1a04c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009fd0 tx=0x7f8c8400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.539+0000 7f8c7b7fe700 1 -- 172.21.15.57:0/1205127951 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400e040 con 0x7f8c8c102a30 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.539+0000 7f8c7b7fe700 1 -- 172.21.15.57:0/1205127951 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c840094a0 con 0x7f8c8c102a30 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.539+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c1a0fe0 con 0x7f8c8c102a30 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.539+0000 7f8c7b7fe700 1 -- 172.21.15.57:0/1205127951 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8401b6a0 con 0x7f8c8c102a30 2023-10-06T22:14:15.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.539+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c1a13a0 con 0x7f8c8c102a30 2023-10-06T22:14:15.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.540+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c10c750 con 0x7f8c8c102a30 2023-10-06T22:14:15.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.542+0000 7f8c7b7fe700 1 -- 172.21.15.57:0/1205127951 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8c8401d470 con 0x7f8c8c102a30 2023-10-06T22:14:15.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.544+0000 7f8c7b7fe700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8c740a1480 0x7f8c740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.544+0000 7f8c7b7fe700 1 -- 172.21.15.57:0/1205127951 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f8c840cddf0 con 0x7f8c8c102a30 2023-10-06T22:14:15.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.544+0000 7f8c8a59c700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8c740a1480 0x7f8c740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.544+0000 7f8c8a59c700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8c740a1480 0x7f8c740a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c101be0 tx=0x7f8c7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:15.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.547+0000 7f8c7b7fe700 1 -- 172.21.15.57:0/1205127951 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8c84097510 con 0x7f8c8c102a30 2023-10-06T22:14:15.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.761+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c8c1a1590 con 0x7f8c8c102a30 2023-10-06T22:14:15.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.761+0000 7f8c7b7fe700 1 -- 172.21.15.57:0/1205127951 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c8409ab30 con 0x7f8c8c102a30 2023-10-06T22:14:15.763 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:14:15.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8c740a1480 msgr2=0x7f8c740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8c740a1480 0x7f8c740a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c101be0 tx=0x7f8c7c009260 comp rx=0 tx=0).stop 2023-10-06T22:14:15.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 msgr2=0x7f8c8c1a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c1a04c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009fd0 tx=0x7f8c8400bce0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 shutdown_connections 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8c740a1480 0x7f8c740a3940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c8c102a30 0x7f8c8c1a04c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c8c100e90 0x7f8c8c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 --2- 172.21.15.57:0/1205127951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c102070 0x7f8c8c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 >> 172.21.15.57:0/1205127951 conn(0x7f8c8c0fc430 msgr2=0x7f8c8c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 shutdown_connections 2023-10-06T22:14:15.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.764+0000 7f8c90bff700 1 -- 172.21.15.57:0/1205127951 wait complete. 2023-10-06T22:14:15.788 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 -- 172.21.15.57:0/473995630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820100ea0 msgr2=0x7f88201012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 --2- 172.21.15.57:0/473995630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820100ea0 0x7f88201012c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8810009990 tx=0x7f8810009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 -- 172.21.15.57:0/473995630 shutdown_connections 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 --2- 172.21.15.57:0/473995630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8820102a40 0x7f8820109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 --2- 172.21.15.57:0/473995630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820100ea0 0x7f88201012c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 --2- 172.21.15.57:0/473995630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8820102080 0x7f8820102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 -- 172.21.15.57:0/473995630 >> 172.21.15.57:0/473995630 conn(0x7f88200fc420 msgr2=0x7f88200fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.989+0000 7f8824dff700 1 -- 172.21.15.57:0/473995630 shutdown_connections 2023-10-06T22:14:15.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.990+0000 7f8824dff700 1 -- 172.21.15.57:0/473995630 wait complete. 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.990+0000 7f8824dff700 1 Processor -- start 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f8824dff700 1 -- start start 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f8824dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8820100ea0 0x7f8820198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f8824dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8820102080 0x7f8820199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f8824dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820102a40 0x7f88201a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f8824dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201a0980 con 0x7f8820102a40 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f8824dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201a0ac0 con 0x7f8820102080 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f8824dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201a0c50 con 0x7f8820100ea0 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f881ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820102a40 0x7f88201a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f881ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820102a40 0x7f88201a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42286/0 (socket says 172.21.15.57:42286) 2023-10-06T22:14:15.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f881ed9d700 1 -- 172.21.15.57:0/2558846662 learned_addr learned my addr 172.21.15.57:0/2558846662 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f881e59c700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8820102080 0x7f8820199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.991+0000 7f881659c700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8820100ea0 0x7f8820198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f881ed9d700 1 -- 172.21.15.57:0/2558846662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8820100ea0 msgr2=0x7f8820198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f881ed9d700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8820100ea0 0x7f8820198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f881ed9d700 1 -- 172.21.15.57:0/2558846662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8820102080 msgr2=0x7f8820199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f881ed9d700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8820102080 0x7f8820199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f881ed9d700 1 -- 172.21.15.57:0/2558846662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100095f0 con 0x7f8820102a40 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f881e59c700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8820102080 0x7f8820199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f881ed9d700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820102a40 0x7f88201a03b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f881800ef30 tx=0x7f881800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f8817fff700 1 -- 172.21.15.57:0/2558846662 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818018070 con 0x7f8820102a40 2023-10-06T22:14:15.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f8817fff700 1 -- 172.21.15.57:0/2558846662 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f881800f040 con 0x7f8820102a40 2023-10-06T22:14:15.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f8817fff700 1 -- 172.21.15.57:0/2558846662 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800cd00 con 0x7f8820102a40 2023-10-06T22:14:15.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.992+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88201a0ed0 con 0x7f8820102a40 2023-10-06T22:14:15.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.993+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88201a1300 con 0x7f8820102a40 2023-10-06T22:14:15.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.994+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8820193550 con 0x7f8820102a40 2023-10-06T22:14:15.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.996+0000 7f8817fff700 1 -- 172.21.15.57:0/2558846662 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8818003a00 con 0x7f8820102a40 2023-10-06T22:14:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.997+0000 7f8817fff700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f880c0a1550 0x7f880c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.997+0000 7f8817fff700 1 -- 172.21.15.57:0/2558846662 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f88180cd140 con 0x7f8820102a40 2023-10-06T22:14:15.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.997+0000 7f881659c700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f880c0a1550 0x7f880c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:15.998+0000 7f881659c700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f880c0a1550 0x7f880c0a3a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f881000b2e0 tx=0x7f8810005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:16.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.000+0000 7f8817fff700 1 -- 172.21.15.57:0/2558846662 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8818096800 con 0x7f8820102a40 2023-10-06T22:14:16.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:15 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3463808433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:15 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2710763138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:15 smithi190 ceph-mon[134891]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-06T22:14:16.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:15 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1205127951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:15 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3463808433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:15 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2710763138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:15 smithi057 ceph-mon[98492]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-06T22:14:16.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:15 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1205127951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:15 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3463808433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:15 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2710763138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:15 smithi156 ceph-mon[134358]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-06T22:14:16.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:15 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1205127951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:16.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.263+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f882010ca00 con 0x7f8820102a40 2023-10-06T22:14:16.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.264+0000 7f8817fff700 1 -- 172.21.15.57:0/2558846662 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f8818099e20 con 0x7f8820102a40 2023-10-06T22:14:16.266 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:16.266 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:14:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f880c0a1550 msgr2=0x7f880c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f880c0a1550 0x7f880c0a3a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f881000b2e0 tx=0x7f8810005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820102a40 msgr2=0x7f88201a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820102a40 0x7f88201a03b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f881800ef30 tx=0x7f881800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 shutdown_connections 2023-10-06T22:14:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f880c0a1550 0x7f880c0a3a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8820102a40 0x7f88201a03b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8820100ea0 0x7f8820198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 --2- 172.21.15.57:0/2558846662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8820102080 0x7f8820199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.266+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 >> 172.21.15.57:0/2558846662 conn(0x7f88200fc420 msgr2=0x7f8820107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.267+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 shutdown_connections 2023-10-06T22:14:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.267+0000 7f8824dff700 1 -- 172.21.15.57:0/2558846662 wait complete. 2023-10-06T22:14:16.272 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:14:16.288 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:14:16.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 -- 172.21.15.57:0/1936999582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901019a0 msgr2=0x7f6890103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 --2- 172.21.15.57:0/1936999582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901019a0 0x7f6890103d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f68840098f0 tx=0x7f6884009c00 comp rx=0 tx=0).stop 2023-10-06T22:14:16.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 -- 172.21.15.57:0/1936999582 shutdown_connections 2023-10-06T22:14:16.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 --2- 172.21.15.57:0/1936999582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6890106c00 0x7f6890109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 --2- 172.21.15.57:0/1936999582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901019a0 0x7f6890103d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 --2- 172.21.15.57:0/1936999582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68901042d0 0x7f68901066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 -- 172.21.15.57:0/1936999582 >> 172.21.15.57:0/1936999582 conn(0x7f68900fb310 msgr2=0x7f68900fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:16.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 -- 172.21.15.57:0/1936999582 shutdown_connections 2023-10-06T22:14:16.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.489+0000 7f6896bff700 1 -- 172.21.15.57:0/1936999582 wait complete. 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 Processor -- start 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 -- start start 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68901019a0 0x7f6890196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901042d0 0x7f68901970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6890106c00 0x7f689019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689019e6f0 con 0x7f6890106c00 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689019e830 con 0x7f68901042d0 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f6896bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689019e9c0 con 0x7f68901019a0 2023-10-06T22:14:16.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f689519c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6890106c00 0x7f689019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f688ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901042d0 0x7f68901970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.490+0000 7f688ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901042d0 0x7f68901970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59238/0 (socket says 172.21.15.57:59238) 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688ffff700 1 -- 172.21.15.57:0/3970097728 learned_addr learned my addr 172.21.15.57:0/3970097728 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f689499b700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68901019a0 0x7f6890196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688ffff700 1 -- 172.21.15.57:0/3970097728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68901019a0 msgr2=0x7f6890196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688ffff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68901019a0 0x7f6890196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688ffff700 1 -- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6890106c00 msgr2=0x7f689019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688ffff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6890106c00 0x7f689019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688ffff700 1 -- 172.21.15.57:0/3970097728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6884009520 con 0x7f68901042d0 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688ffff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901042d0 0x7f68901970b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f687c00e9a0 tx=0x7f687c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.491+0000 7f688dffb700 1 -- 172.21.15.57:0/3970097728 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c009790 con 0x7f68901042d0 2023-10-06T22:14:16.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.492+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689019eca0 con 0x7f68901042d0 2023-10-06T22:14:16.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.492+0000 7f688dffb700 1 -- 172.21.15.57:0/3970097728 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f687c010430 con 0x7f68901042d0 2023-10-06T22:14:16.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.492+0000 7f688dffb700 1 -- 172.21.15.57:0/3970097728 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c018680 con 0x7f68901042d0 2023-10-06T22:14:16.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.492+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689019f1b0 con 0x7f68901042d0 2023-10-06T22:14:16.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.493+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689010c6a0 con 0x7f68901042d0 2023-10-06T22:14:16.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.495+0000 7f688dffb700 1 -- 172.21.15.57:0/3970097728 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f687c010810 con 0x7f68901042d0 2023-10-06T22:14:16.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.496+0000 7f688dffb700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f68800a1550 0x7f68800a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.496+0000 7f688dffb700 1 -- 172.21.15.57:0/3970097728 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f687c014070 con 0x7f68901042d0 2023-10-06T22:14:16.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.496+0000 7f689499b700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f68800a1550 0x7f68800a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.497+0000 7f689499b700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f68800a1550 0x7f68800a3a10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f688400fa40 tx=0x7f6884019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.498+0000 7f688dffb700 1 -- 172.21.15.57:0/3970097728 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f687c052100 con 0x7f68901042d0 2023-10-06T22:14:16.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.711+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f689019f3a0 con 0x7f68901042d0 2023-10-06T22:14:16.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.713+0000 7f688dffb700 1 -- 172.21.15.57:0/3970097728 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7f687c09a030 con 0x7f68901042d0 2023-10-06T22:14:16.744 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:16.744 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.715+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f68800a1550 msgr2=0x7f68800a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.715+0000 7f6896bff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f68800a1550 0x7f68800a3a10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f688400fa40 tx=0x7f6884019040 comp rx=0 tx=0).stop 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.715+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901042d0 msgr2=0x7f68901970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.715+0000 7f6896bff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901042d0 0x7f68901970b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f687c00e9a0 tx=0x7f687c00ecb0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 shutdown_connections 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f68800a1550 0x7f68800a3a10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6890106c00 0x7f689019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68901019a0 0x7f6890196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 --2- 172.21.15.57:0/3970097728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68901042d0 0x7f68901970b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 >> 172.21.15.57:0/3970097728 conn(0x7f68900fb310 msgr2=0x7f6890102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 shutdown_connections 2023-10-06T22:14:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.716+0000 7f6896bff700 1 -- 172.21.15.57:0/3970097728 wait complete. 2023-10-06T22:14:16.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:14:16.748 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:14:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 -- 172.21.15.57:0/1206873221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 msgr2=0x7f1f58109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/1206873221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 0x7f1f58109370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b0a0 tx=0x7f1f5000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 -- 172.21.15.57:0/1206873221 shutdown_connections 2023-10-06T22:14:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/1206873221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 0x7f1f58109370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/1206873221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58100f40 0x7f1f58101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/1206873221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58102090 0x7f1f58102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 -- 172.21.15.57:0/1206873221 >> 172.21.15.57:0/1206873221 conn(0x7f1f580fc4d0 msgr2=0x7f1f580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 -- 172.21.15.57:0/1206873221 shutdown_connections 2023-10-06T22:14:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.954+0000 7f1f5cdff700 1 -- 172.21.15.57:0/1206873221 wait complete. 2023-10-06T22:14:16.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 Processor -- start 2023-10-06T22:14:16.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 -- start start 2023-10-06T22:14:16.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58100f40 0x7f1f58198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58102090 0x7f1f58199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 0x7f1f581a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f581a0a50 con 0x7f1f58102a50 2023-10-06T22:14:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f581a0b90 con 0x7f1f58102090 2023-10-06T22:14:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.955+0000 7f1f5cdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f581a0d20 con 0x7f1f58100f40 2023-10-06T22:14:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f55d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58102090 0x7f1f58199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f56d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 0x7f1f581a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f55d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58102090 0x7f1f58199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59254/0 (socket says 172.21.15.57:59254) 2023-10-06T22:14:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f55d9b700 1 -- 172.21.15.57:0/910077875 learned_addr learned my addr 172.21.15.57:0/910077875 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f5659c700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58100f40 0x7f1f58198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f56d9d700 1 -- 172.21.15.57:0/910077875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58100f40 msgr2=0x7f1f58198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f56d9d700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58100f40 0x7f1f58198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f56d9d700 1 -- 172.21.15.57:0/910077875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58102090 msgr2=0x7f1f58199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:16.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f56d9d700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58102090 0x7f1f58199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:16.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f56d9d700 1 -- 172.21.15.57:0/910077875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f50009d00 con 0x7f1f58102a50 2023-10-06T22:14:16.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f55d9b700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58102090 0x7f1f58199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:16.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f56d9d700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 0x7f1f581a0480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b070 tx=0x7f1f5000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:16.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.956+0000 7f1f477fe700 1 -- 172.21.15.57:0/910077875 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000e040 con 0x7f1f58102a50 2023-10-06T22:14:16.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.957+0000 7f1f477fe700 1 -- 172.21.15.57:0/910077875 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f50009480 con 0x7f1f58102a50 2023-10-06T22:14:16.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.957+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f581a0fa0 con 0x7f1f58102a50 2023-10-06T22:14:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.957+0000 7f1f477fe700 1 -- 172.21.15.57:0/910077875 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500052a0 con 0x7f1f58102a50 2023-10-06T22:14:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.957+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f581a1360 con 0x7f1f58102a50 2023-10-06T22:14:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.958+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5810c720 con 0x7f1f58102a50 2023-10-06T22:14:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.960+0000 7f1f477fe700 1 -- 172.21.15.57:0/910077875 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1f5001d480 con 0x7f1f58102a50 2023-10-06T22:14:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.961+0000 7f1f477fe700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1f400a1480 0x7f1f400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:16.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.961+0000 7f1f477fe700 1 -- 172.21.15.57:0/910077875 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1f500cdce0 con 0x7f1f58102a50 2023-10-06T22:14:16.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.961+0000 7f1f5659c700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1f400a1480 0x7f1f400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:16.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.962+0000 7f1f5659c700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1f400a1480 0x7f1f400a3940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009c60 tx=0x7f1f48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:16.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:16.965+0000 7f1f477fe700 1 -- 172.21.15.57:0/910077875 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1f500973a0 con 0x7f1f58102a50 2023-10-06T22:14:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:16 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2558846662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:16 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3970097728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:16 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2558846662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:16 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3970097728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:17.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.177+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f581a1550 con 0x7f1f58102a50 2023-10-06T22:14:17.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.178+0000 7f1f477fe700 1 -- 172.21.15.57:0/910077875 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f5009a9c0 con 0x7f1f58102a50 2023-10-06T22:14:17.180 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:14:17.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1f400a1480 msgr2=0x7f1f400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1f400a1480 0x7f1f400a3940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009c60 tx=0x7f1f48009260 comp rx=0 tx=0).stop 2023-10-06T22:14:17.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 msgr2=0x7f1f581a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 0x7f1f581a0480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b070 tx=0x7f1f5000bc80 comp rx=0 tx=0).stop 2023-10-06T22:14:17.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 shutdown_connections 2023-10-06T22:14:17.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1f400a1480 0x7f1f400a3940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f58102a50 0x7f1f581a0480 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58100f40 0x7f1f58198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 --2- 172.21.15.57:0/910077875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f58102090 0x7f1f58199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 >> 172.21.15.57:0/910077875 conn(0x7f1f580fc4d0 msgr2=0x7f1f581079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:17.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 shutdown_connections 2023-10-06T22:14:17.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.181+0000 7f1f5cdff700 1 -- 172.21.15.57:0/910077875 wait complete. 2023-10-06T22:14:17.201 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:14:17.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:16 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2558846662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:17.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:16 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3970097728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:17.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.405+0000 7feab17ff700 1 -- 172.21.15.57:0/448164832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac102070 msgr2=0x7feaac1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.405+0000 7feab17ff700 1 --2- 172.21.15.57:0/448164832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac102070 0x7feaac1024f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009990 tx=0x7feaa0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.406+0000 7feab17ff700 1 -- 172.21.15.57:0/448164832 shutdown_connections 2023-10-06T22:14:17.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.406+0000 7feab17ff700 1 --2- 172.21.15.57:0/448164832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102a30 0x7feaac109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.406+0000 7feab17ff700 1 --2- 172.21.15.57:0/448164832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaac100e90 0x7feaac1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.406+0000 7feab17ff700 1 --2- 172.21.15.57:0/448164832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac102070 0x7feaac1024f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.406+0000 7feab17ff700 1 -- 172.21.15.57:0/448164832 >> 172.21.15.57:0/448164832 conn(0x7feaac0fc430 msgr2=0x7feaac0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:17.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.406+0000 7feab17ff700 1 -- 172.21.15.57:0/448164832 shutdown_connections 2023-10-06T22:14:17.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.406+0000 7feab17ff700 1 -- 172.21.15.57:0/448164832 wait complete. 2023-10-06T22:14:17.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 Processor -- start 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 -- start start 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac100e90 0x7feaac198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102070 0x7feaac199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaac102a30 0x7feaac1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1a09c0 con 0x7feaac100e90 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1a0b00 con 0x7feaac102a30 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feab17ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1a0c90 con 0x7feaac102070 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feaaa7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102070 0x7feaac199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feaaaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac100e90 0x7feaac198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feaaa7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102070 0x7feaac199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45372/0 (socket says 172.21.15.57:45372) 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.407+0000 7feaaa7fc700 1 -- 172.21.15.57:0/3554136542 learned_addr learned my addr 172.21.15.57:0/3554136542 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:17.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaab7fe700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaac102a30 0x7feaac1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaaa7fc700 1 -- 172.21.15.57:0/3554136542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaac102a30 msgr2=0x7feaac1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaaa7fc700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaac102a30 0x7feaac1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaaa7fc700 1 -- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac100e90 msgr2=0x7feaac198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaaa7fc700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac100e90 0x7feaac198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaaa7fc700 1 -- 172.21.15.57:0/3554136542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00095f0 con 0x7feaac102070 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaaaffd700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac100e90 0x7feaac198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feaaa7fc700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102070 0x7feaac199380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feaa00098e0 tx=0x7feaa0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7fea93fff700 1 -- 172.21.15.57:0/3554136542 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa001d070 con 0x7feaac102070 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac1a0f10 con 0x7feaac102070 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7fea93fff700 1 -- 172.21.15.57:0/3554136542 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa00054f0 con 0x7feaac102070 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.408+0000 7fea93fff700 1 -- 172.21.15.57:0/3554136542 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0004be0 con 0x7feaac102070 2023-10-06T22:14:17.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.409+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac1a1280 con 0x7feaac102070 2023-10-06T22:14:17.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.409+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac10c7c0 con 0x7feaac102070 2023-10-06T22:14:17.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.412+0000 7fea93fff700 1 -- 172.21.15.57:0/3554136542 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feaa000fbc0 con 0x7feaac102070 2023-10-06T22:14:17.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.413+0000 7fea93fff700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fea940a1480 0x7fea940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.413+0000 7fea93fff700 1 -- 172.21.15.57:0/3554136542 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7feaa00ce3f0 con 0x7feaac102070 2023-10-06T22:14:17.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.413+0000 7feaaaffd700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fea940a1480 0x7fea940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.414+0000 7feaaaffd700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fea940a1480 0x7fea940a3940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feaac101be0 tx=0x7fea9c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:17.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.415+0000 7fea93fff700 1 -- 172.21.15.57:0/3554136542 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feaa0097ab0 con 0x7feaac102070 2023-10-06T22:14:17.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.626+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feaac1a1470 con 0x7feaac102070 2023-10-06T22:14:17.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.626+0000 7fea93fff700 1 -- 172.21.15.57:0/3554136542 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7feaa009b0d0 con 0x7feaac102070 2023-10-06T22:14:17.629 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:14:17.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fea940a1480 msgr2=0x7fea940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fea940a1480 0x7fea940a3940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feaac101be0 tx=0x7fea9c007400 comp rx=0 tx=0).stop 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102070 msgr2=0x7feaac199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102070 0x7feaac199380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feaa00098e0 tx=0x7feaa0004730 comp rx=0 tx=0).stop 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 shutdown_connections 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fea940a1480 0x7fea940a3940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feaac102a30 0x7feaac1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feaac100e90 0x7feaac198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 --2- 172.21.15.57:0/3554136542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaac102070 0x7feaac199380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 >> 172.21.15.57:0/3554136542 conn(0x7feaac0fc430 msgr2=0x7feaac0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 shutdown_connections 2023-10-06T22:14:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.629+0000 7feab17ff700 1 -- 172.21.15.57:0/3554136542 wait complete. 2023-10-06T22:14:17.650 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:14:17.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 -- 172.21.15.57:0/3581045036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8100e90 msgr2=0x7efcb81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 --2- 172.21.15.57:0/3581045036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8100e90 0x7efcb81012b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efca8009990 tx=0x7efca8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 -- 172.21.15.57:0/3581045036 shutdown_connections 2023-10-06T22:14:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 --2- 172.21.15.57:0/3581045036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8102a30 0x7efcb8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 --2- 172.21.15.57:0/3581045036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8100e90 0x7efcb81012b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 --2- 172.21.15.57:0/3581045036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb8102070 0x7efcb81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 -- 172.21.15.57:0/3581045036 >> 172.21.15.57:0/3581045036 conn(0x7efcb80fc430 msgr2=0x7efcb80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 -- 172.21.15.57:0/3581045036 shutdown_connections 2023-10-06T22:14:17.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.856+0000 7efcbcfff700 1 -- 172.21.15.57:0/3581045036 wait complete. 2023-10-06T22:14:17.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 Processor -- start 2023-10-06T22:14:17.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 -- start start 2023-10-06T22:14:17.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb8100e90 0x7efcb8198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8102070 0x7efcb81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8102a30 0x7efcb81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb81a09f0 con 0x7efcb8102070 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb81a0b30 con 0x7efcb8102a30 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.857+0000 7efcbcfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb81a0cc0 con 0x7efcb8100e90 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8102070 0x7efcb81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8102070 0x7efcb81993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42368/0 (socket says 172.21.15.57:42368) 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 -- 172.21.15.57:0/379807796 learned_addr learned my addr 172.21.15.57:0/379807796 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcb659c700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb8100e90 0x7efcb8198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcb6d9d700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8102a30 0x7efcb81a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 -- 172.21.15.57:0/379807796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb8100e90 msgr2=0x7efcb8198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb8100e90 0x7efcb8198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 -- 172.21.15.57:0/379807796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8102a30 msgr2=0x7efcb81a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8102a30 0x7efcb81a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 -- 172.21.15.57:0/379807796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80095f0 con 0x7efcb8102070 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.858+0000 7efcaffff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8102070 0x7efcb81993b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efca000e970 tx=0x7efca000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.859+0000 7efcaf7fe700 1 -- 172.21.15.57:0/379807796 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0009790 con 0x7efcb8102070 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.859+0000 7efcaf7fe700 1 -- 172.21.15.57:0/379807796 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efca000c7e0 con 0x7efcb8102070 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.859+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb81a0fa0 con 0x7efcb8102070 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.859+0000 7efcaf7fe700 1 -- 172.21.15.57:0/379807796 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0003c70 con 0x7efcb8102070 2023-10-06T22:14:17.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.859+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb81a1420 con 0x7efcb8102070 2023-10-06T22:14:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.860+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb810c750 con 0x7efcb8102070 2023-10-06T22:14:17.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.862+0000 7efcaf7fe700 1 -- 172.21.15.57:0/379807796 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efca000cc50 con 0x7efcb8102070 2023-10-06T22:14:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.863+0000 7efcaf7fe700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efca40a1480 0x7efca40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.863+0000 7efcaf7fe700 1 -- 172.21.15.57:0/379807796 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7efca0014070 con 0x7efcb8102070 2023-10-06T22:14:17.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.863+0000 7efcb659c700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efca40a1480 0x7efca40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:17.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.864+0000 7efcb659c700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efca40a1480 0x7efca40a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efca800b2e0 tx=0x7efca8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:17.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:17.867+0000 7efcaf7fe700 1 -- 172.21.15.57:0/379807796 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efca00964f0 con 0x7efcb8102070 2023-10-06T22:14:17.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:17 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/910077875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:17.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:17 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3554136542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:17.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:17 smithi057 ceph-mon[98492]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-06T22:14:18.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:17 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/910077875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:18.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:17 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3554136542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:18.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:17 smithi190 ceph-mon[134891]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-06T22:14:18.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.081+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcb81a1670 con 0x7efcb8102070 2023-10-06T22:14:18.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.082+0000 7efcaf7fe700 1 -- 172.21.15.57:0/379807796 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7efca0003dd0 con 0x7efcb8102070 2023-10-06T22:14:18.084 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:14:18.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.084+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efca40a1480 msgr2=0x7efca40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.084+0000 7efcbcfff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efca40a1480 0x7efca40a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efca800b2e0 tx=0x7efca8005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.084+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8102070 msgr2=0x7efcb81993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.084+0000 7efcbcfff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8102070 0x7efcb81993b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efca000e970 tx=0x7efca000ec80 comp rx=0 tx=0).stop 2023-10-06T22:14:18.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 shutdown_connections 2023-10-06T22:14:18.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efca40a1480 0x7efca40a3940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb8102a30 0x7efcb81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb8100e90 0x7efcb8198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 --2- 172.21.15.57:0/379807796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcb8102070 0x7efcb81993b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 >> 172.21.15.57:0/379807796 conn(0x7efcb80fc430 msgr2=0x7efcb80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 shutdown_connections 2023-10-06T22:14:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.085+0000 7efcbcfff700 1 -- 172.21.15.57:0/379807796 wait complete. 2023-10-06T22:14:18.106 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:14:18.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:17 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/910077875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:18.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:17 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3554136542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:18.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:17 smithi156 ceph-mon[134358]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-06T22:14:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/2920907211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102080 msgr2=0x7fe7d4102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/2920907211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102080 0x7fe7d4102500 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009990 tx=0x7fe7c4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/2920907211 shutdown_connections 2023-10-06T22:14:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/2920907211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4102a40 0x7fe7d4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/2920907211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7d4100ea0 0x7fe7d41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/2920907211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102080 0x7fe7d4102500 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/2920907211 >> 172.21.15.57:0/2920907211 conn(0x7fe7d40fc420 msgr2=0x7fe7d40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/2920907211 shutdown_connections 2023-10-06T22:14:18.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.307+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/2920907211 wait complete. 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 Processor -- start 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 -- start start 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4100ea0 0x7fe7d4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7d4102080 0x7fe7d40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102a40 0x7fe7d40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4073f80 con 0x7fe7d4102a40 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d40740f0 con 0x7fe7d4102080 2023-10-06T22:14:18.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.309+0000 7fe7dc1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d41a0c50 con 0x7fe7d4100ea0 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4100ea0 0x7fe7d4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4100ea0 0x7fe7d4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45400/0 (socket says 172.21.15.57:45400) 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 -- 172.21.15.57:0/3775792207 learned_addr learned my addr 172.21.15.57:0/3775792207 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7da79c700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102a40 0x7fe7d40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d979a700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7d4102080 0x7fe7d40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 -- 172.21.15.57:0/3775792207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7d4102080 msgr2=0x7fe7d40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7d4102080 0x7fe7d40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 -- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102a40 msgr2=0x7fe7d40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102a40 0x7fe7d40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 -- 172.21.15.57:0/3775792207 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40095f0 con 0x7fe7d4100ea0 2023-10-06T22:14:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7d9f9b700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4100ea0 0x7fe7d4074c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d980 tx=0x7fe7d000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.310+0000 7fe7caffd700 1 -- 172.21.15.57:0/3775792207 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000f890 con 0x7fe7d4100ea0 2023-10-06T22:14:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.311+0000 7fe7caffd700 1 -- 172.21.15.57:0/3775792207 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7d0010460 con 0x7fe7d4100ea0 2023-10-06T22:14:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.311+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d41a0f30 con 0x7fe7d4100ea0 2023-10-06T22:14:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.311+0000 7fe7caffd700 1 -- 172.21.15.57:0/3775792207 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00195c0 con 0x7fe7d4100ea0 2023-10-06T22:14:18.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.311+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d41a1350 con 0x7fe7d4100ea0 2023-10-06T22:14:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.312+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d410c7d0 con 0x7fe7d4100ea0 2023-10-06T22:14:18.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.314+0000 7fe7caffd700 1 -- 172.21.15.57:0/3775792207 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe7d00105d0 con 0x7fe7d4100ea0 2023-10-06T22:14:18.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.315+0000 7fe7caffd700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7c00a14d0 0x7fe7c00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.315+0000 7fe7caffd700 1 -- 172.21.15.57:0/3775792207 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fe7d00ccaf0 con 0x7fe7d4100ea0 2023-10-06T22:14:18.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.315+0000 7fe7d979a700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7c00a14d0 0x7fe7c00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.316+0000 7fe7d979a700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7c00a14d0 0x7fe7c00a3990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400b2e0 tx=0x7fe7c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:18.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.320+0000 7fe7caffd700 1 -- 172.21.15.57:0/3775792207 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe7d0096120 con 0x7fe7d4100ea0 2023-10-06T22:14:18.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.578+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7d410c9e0 con 0x7fe7d4100ea0 2023-10-06T22:14:18.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.579+0000 7fe7caffd700 1 -- 172.21.15.57:0/3775792207 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fe7d0016090 con 0x7fe7d4100ea0 2023-10-06T22:14:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:18.581 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:14:18.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.581+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7c00a14d0 msgr2=0x7fe7c00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.581+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7c00a14d0 0x7fe7c00a3990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400b2e0 tx=0x7fe7c4005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.581+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4100ea0 msgr2=0x7fe7d4074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.581+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4100ea0 0x7fe7d4074c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d980 tx=0x7fe7d000dc90 comp rx=0 tx=0).stop 2023-10-06T22:14:18.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 shutdown_connections 2023-10-06T22:14:18.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe7c00a14d0 0x7fe7c00a3990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7d4102a40 0x7fe7d40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4100ea0 0x7fe7d4074c60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 --2- 172.21.15.57:0/3775792207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7d4102080 0x7fe7d40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 >> 172.21.15.57:0/3775792207 conn(0x7fe7d40fc420 msgr2=0x7fe7d40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:18.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 shutdown_connections 2023-10-06T22:14:18.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.582+0000 7fe7dc1ff700 1 -- 172.21.15.57:0/3775792207 wait complete. 2023-10-06T22:14:18.585 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:14:18.603 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:14:18.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 -- 172.21.15.57:0/4271701237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 msgr2=0x7f28201012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 --2- 172.21.15.57:0/4271701237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 0x7f28201012c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f281c009990 tx=0x7f281c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 -- 172.21.15.57:0/4271701237 shutdown_connections 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 --2- 172.21.15.57:0/4271701237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2820102a40 0x7f2820109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 --2- 172.21.15.57:0/4271701237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 0x7f28201012c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 --2- 172.21.15.57:0/4271701237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2820102080 0x7f2820102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 -- 172.21.15.57:0/4271701237 >> 172.21.15.57:0/4271701237 conn(0x7f28200fc420 msgr2=0x7f28200fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 -- 172.21.15.57:0/4271701237 shutdown_connections 2023-10-06T22:14:18.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.804+0000 7f28283ff700 1 -- 172.21.15.57:0/4271701237 wait complete. 2023-10-06T22:14:18.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.805+0000 7f28283ff700 1 Processor -- start 2023-10-06T22:14:18.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.805+0000 7f28283ff700 1 -- start start 2023-10-06T22:14:18.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f28283ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 0x7f2820074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f28283ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2820102080 0x7f28200732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f28283ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2820102a40 0x7f28200737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f28283ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820073f80 con 0x7f2820102a40 2023-10-06T22:14:18.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f28283ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28200740f0 con 0x7f2820100ea0 2023-10-06T22:14:18.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f28283ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201a0c40 con 0x7f2820102080 2023-10-06T22:14:18.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f282699c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2820102a40 0x7f28200737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f282599a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2820102080 0x7f28200732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f282619b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 0x7f2820074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f282599a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2820102080 0x7f28200732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45426/0 (socket says 172.21.15.57:45426) 2023-10-06T22:14:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f282699c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2820102a40 0x7f28200737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42410/0 (socket says 172.21.15.57:42410) 2023-10-06T22:14:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f282599a700 1 -- 172.21.15.57:0/2513043702 learned_addr learned my addr 172.21.15.57:0/2513043702 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.806+0000 7f282619b700 1 -- 172.21.15.57:0/2513043702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2820102080 msgr2=0x7f28200732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.807+0000 7f282619b700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2820102080 0x7f28200732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.807+0000 7f282619b700 1 -- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2820102a40 msgr2=0x7f28200737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:18.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.807+0000 7f282619b700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2820102a40 0x7f28200737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:18.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.807+0000 7f282619b700 1 -- 172.21.15.57:0/2513043702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0095f0 con 0x7f2820100ea0 2023-10-06T22:14:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.807+0000 7f282619b700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 0x7f2820074c60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b680 tx=0x7f281c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.808+0000 7f28177fe700 1 -- 172.21.15.57:0/2513043702 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c01d070 con 0x7f2820100ea0 2023-10-06T22:14:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.808+0000 7f28177fe700 1 -- 172.21.15.57:0/2513043702 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f281c0054e0 con 0x7f2820100ea0 2023-10-06T22:14:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.808+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28201a0ec0 con 0x7f2820100ea0 2023-10-06T22:14:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.808+0000 7f28177fe700 1 -- 172.21.15.57:0/2513043702 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c004bd0 con 0x7f2820100ea0 2023-10-06T22:14:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.808+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28201a1280 con 0x7f2820100ea0 2023-10-06T22:14:18.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.809+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282010c710 con 0x7f2820100ea0 2023-10-06T22:14:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.811+0000 7f28177fe700 1 -- 172.21.15.57:0/2513043702 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f281c00fb90 con 0x7f2820100ea0 2023-10-06T22:14:18.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.812+0000 7f28177fe700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f280c0a1550 0x7f280c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:18.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.812+0000 7f28177fe700 1 -- 172.21.15.57:0/2513043702 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f281c0cde80 con 0x7f2820100ea0 2023-10-06T22:14:18.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.812+0000 7f282599a700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f280c0a1550 0x7f280c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:18.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.813+0000 7f282599a700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f280c0a1550 0x7f280c0a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f281000bd70 tx=0x7f281000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:18.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:18.814+0000 7f28177fe700 1 -- 172.21.15.57:0/2513043702 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f281c097540 con 0x7f2820100ea0 2023-10-06T22:14:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:18 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/379807796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:18 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3775792207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:19.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.024+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2820107e90 con 0x7f2820100ea0 2023-10-06T22:14:19.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.026+0000 7f28177fe700 1 -- 172.21.15.57:0/2513043702 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7f281c09ab60 con 0x7f2820100ea0 2023-10-06T22:14:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:19.029 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:14:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f280c0a1550 msgr2=0x7f280c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f280c0a1550 0x7f280c0a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f281000bd70 tx=0x7f281000b4a0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 msgr2=0x7f2820074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 0x7f2820074c60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b680 tx=0x7f281c004730 comp rx=0 tx=0).stop 2023-10-06T22:14:19.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 shutdown_connections 2023-10-06T22:14:19.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f280c0a1550 0x7f280c0a3a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2820102a40 0x7f28200737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2820100ea0 0x7f2820074c60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 --2- 172.21.15.57:0/2513043702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2820102080 0x7f28200732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.028+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 >> 172.21.15.57:0/2513043702 conn(0x7f28200fc420 msgr2=0x7f28200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.029+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 shutdown_connections 2023-10-06T22:14:19.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.029+0000 7f28283ff700 1 -- 172.21.15.57:0/2513043702 wait complete. 2023-10-06T22:14:19.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-06T22:12:43.212955+0000', 'modified': '2023-10-06T22:12:50.650753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24553, 'mds_1': 24551, 'mds_2': 24529, 'mds_3': 24517, 'mds_4': 24509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14523': {'gid': 14523, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6839/3033019918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6838', 'nonce': 3033019918}, {'type': 'v1', 'addr': '172.21.15.57:6839', 'nonce': 3033019918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14535': {'gid': 14535, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6841/490100321', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6840', 'nonce': 490100321}, {'type': 'v1', 'addr': '172.21.15.57:6841', 'nonce': 490100321}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6833/1142570288', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6832', 'nonce': 1142570288}, {'type': 'v1', 'addr': '172.21.15.156:6833', 'nonce': 1142570288}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/1813464152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 1813464152}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 1813464152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2797676784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2797676784}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2797676784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6837/3773867071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3773867071}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3773867071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.190:6835/871306260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 871306260}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 871306260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.190:6837/3737066182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3737066182}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3737066182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.156:6839/1657784543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1657784543}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1657784543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6839/751138264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 751138264}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 751138264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-06T22:14:19.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-06T22:14:19.050 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-06T22:14:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:18 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/379807796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:18 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3775792207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:19.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:18 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/379807796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:19.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:18 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3775792207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:19.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 -- 172.21.15.57:0/990045032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56481042b0 msgr2=0x7f56481066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 --2- 172.21.15.57:0/990045032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56481042b0 0x7f56481066a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5638009990 tx=0x7f5638009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 -- 172.21.15.57:0/990045032 shutdown_connections 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 --2- 172.21.15.57:0/990045032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5648106be0 0x7f5648109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 --2- 172.21.15.57:0/990045032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101980 0x7f5648103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 --2- 172.21.15.57:0/990045032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56481042b0 0x7f56481066a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 -- 172.21.15.57:0/990045032 >> 172.21.15.57:0/990045032 conn(0x7f56480fb370 msgr2=0x7f56480fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 -- 172.21.15.57:0/990045032 shutdown_connections 2023-10-06T22:14:19.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.252+0000 7f56503ff700 1 -- 172.21.15.57:0/990045032 wait complete. 2023-10-06T22:14:19.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.253+0000 7f56503ff700 1 Processor -- start 2023-10-06T22:14:19.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.253+0000 7f56503ff700 1 -- start start 2023-10-06T22:14:19.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f56503ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101980 0x7f56481012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f56503ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56481042b0 0x7f56480ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f56503ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5648106be0 0x7f56480ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f56503ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648100610 con 0x7f5648106be0 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f56503ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648100780 con 0x7f56481042b0 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f56503ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56481a0c10 con 0x7f5648101980 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5648106be0 0x7f56480ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5648106be0 0x7f56480ffe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42436/0 (socket says 172.21.15.57:42436) 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 -- 172.21.15.57:0/3942562575 learned_addr learned my addr 172.21.15.57:0/3942562575 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564d99a700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56481042b0 0x7f56480ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e19b700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101980 0x7f56481012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 -- 172.21.15.57:0/3942562575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101980 msgr2=0x7f56481012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101980 0x7f56481012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 -- 172.21.15.57:0/3942562575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56481042b0 msgr2=0x7f56480ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56481042b0 0x7f56480ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.254+0000 7f564e99c700 1 -- 172.21.15.57:0/3942562575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56380095f0 con 0x7f5648106be0 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.255+0000 7f564e99c700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5648106be0 0x7f56480ffe70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f564000ef30 tx=0x7f564000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.255+0000 7f563f7fe700 1 -- 172.21.15.57:0/3942562575 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640018070 con 0x7f5648106be0 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.255+0000 7f563f7fe700 1 -- 172.21.15.57:0/3942562575 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f564000f040 con 0x7f5648106be0 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.255+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56481a0e90 con 0x7f5648106be0 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.255+0000 7f563f7fe700 1 -- 172.21.15.57:0/3942562575 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564000cd00 con 0x7f5648106be0 2023-10-06T22:14:19.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.255+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56481a1310 con 0x7f5648106be0 2023-10-06T22:14:19.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.258+0000 7f563f7fe700 1 -- 172.21.15.57:0/3942562575 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5640003a00 con 0x7f5648106be0 2023-10-06T22:14:19.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 Processor -- start 2023-10-06T22:14:19.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f563f7fe700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f56340a1500 0x7f56340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 -- start start 2023-10-06T22:14:19.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f563f7fe700 1 -- 172.21.15.57:0/3942562575 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f56400cc7b0 con 0x7f5648106be0 2023-10-06T22:14:19.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6210 0x55a2b8fe6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8fe7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fe7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8f4e410 con 0x55a2b8fe7a10 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8f3fbd0 con 0x55a2b8fe6b70 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f5663cffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8f3d4f0 con 0x55a2b8fe6210 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f564d99a700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f56340a1500 0x7f56340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f564e19b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6210 0x55a2b8fe6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.260+0000 7f564e99c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fe7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6210 0x55a2b8fe6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45452/0 (socket says 172.21.15.57:45452) 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8fe7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 -- 172.21.15.57:0/2217051394 learned_addr learned my addr 172.21.15.57:0/2217051394 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 -- 172.21.15.57:0/2217051394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 msgr2=0x55a2b8fe7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8fe7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 -- 172.21.15.57:0/2217051394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 msgr2=0x55a2b8fe7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fe7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 -- 172.21.15.57:0/2217051394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b8d63d90 con 0x55a2b8fe6210 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564d99a700 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8fe7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e99c700 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fe7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564e19b700 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6210 0x55a2b8fe6630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5644007d70 tx=0x7f56440106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d47eb58a4741ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f564d99a700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f56340a1500 0x7f56340a39c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5638005cd0 tx=0x7f5638005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f563cbfe700 1 -- 172.21.15.57:0/2217051394 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644010e10 con 0x55a2b8fe6210 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f563cbfe700 1 -- 172.21.15.57:0/2217051394 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5644015070 con 0x55a2b8fe6210 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.261+0000 7f563cbfe700 1 -- 172.21.15.57:0/2217051394 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644003e60 con 0x55a2b8fe6210 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 -- 172.21.15.57:0/2217051394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6210 msgr2=0x55a2b8fe6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6210 0x55a2b8fe6630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5644007d70 tx=0x7f56440106b0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 -- 172.21.15.57:0/2217051394 shutdown_connections 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fe7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6210 0x55a2b8fe6630 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2217051394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8fe7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 -- 172.21.15.57:0/2217051394 >> 172.21.15.57:0/2217051394 conn(0x55a2b8f5a210 msgr2=0x55a2b8fe10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 -- 172.21.15.57:0/2217051394 shutdown_connections 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.262+0000 7f5663cffb80 1 -- 172.21.15.57:0/2217051394 wait complete. 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.263+0000 7f5663cffb80 1 -- start start 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.263+0000 7f5663cffb80 1 Processor -- start 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe7a10 0x55a2b8ff7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f5663cffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8f38de0 con 0x55a2b8fe6210 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f5663cffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8ed56c0 con 0x55a2b8fe6b70 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f5663cffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8ed5360 con 0x55a2b8fe7a10 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f5663cffb80 10 client.0 _ll_register_callbacks cb 0x55a2b8d327b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f564e19b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f564e19b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42452/0 (socket says 172.21.15.57:42452) 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f564e19b700 1 -- 172.21.15.57:0/1755326552 learned_addr learned my addr 172.21.15.57:0/1755326552 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f564e99c700 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe7a10 0x55a2b8ff7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.264+0000 7f564d99a700 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f564e19b700 1 -- 172.21.15.57:0/1755326552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe7a10 msgr2=0x55a2b8ff7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f564e19b700 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe7a10 0x55a2b8ff7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f564e19b700 1 -- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 msgr2=0x55a2b8ff04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f564e19b700 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f564e19b700 1 -- 172.21.15.57:0/1755326552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5640084040 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f564e19b700 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feffa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5644005850 tx=0x7f5644003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394b33bb8b36a4c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f5631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f5631ffb700 1 -- 172.21.15.57:0/1755326552 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644020070 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f5631ffb700 1 -- 172.21.15.57:0/1755326552 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f5644005030 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2b8fd8ec0 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f5631ffb700 1 -- 172.21.15.57:0/1755326552 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5644010ea0 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f5663cffb80 10 client.15141 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.265+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f56440079a0 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5631ffb700 1 -- 172.21.15.57:0/1755326552 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2b8fd8ec0 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5631ffb700 1 -- 172.21.15.57:0/1755326552 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5644003be0 con 0x55a2b8fe6210 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5663cffb80 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:19.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5663cffb80 10 client.15141 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T22:14:19.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5663cffb80 20 client.15141 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:19.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5663cffb80 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b8ffbab0 0x55a2b8ffa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5663cffb80 4 client.15141 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:14:19.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.266+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2b8f31dc0 con 0x55a2b8ffbab0 2023-10-06T22:14:19.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.267+0000 7f564d99a700 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b8ffbab0 0x55a2b8ffa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.267+0000 7f564d99a700 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b8ffbab0 0x55a2b8ffa5c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8d27f746414e412 server_cookie=3662775e1b8c5641 in_seq=0 out_seq=0 2023-10-06T22:14:19.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.267+0000 7f5631ffb700 10 client.15141 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:14:19.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5631ffb700 1 -- 172.21.15.57:0/1755326552 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a2b8f31dc0 con 0x55a2b8ffbab0 2023-10-06T22:14:19.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5631ffb700 10 client.15141 handle_command_reply: tid=0 2023-10-06T22:14:19.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 client.15141 shutdown 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 msgr2=0x55a2b8feffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feffa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5644005850 tx=0x7f5644003710 comp rx=0 tx=0).stop 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 shutdown_connections 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b8ffbab0 0x55a2b8ffa5c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe7a10 0x55a2b8ff7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feffa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 --2- 172.21.15.57:0/1755326552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5631ffb700 0 client.15141 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.268+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 >> 172.21.15.57:0/1755326552 conn(0x55a2b8f5a210 msgr2=0x55a2b8fe0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.269+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 shutdown_connections 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.269+0000 7f5663cffb80 1 -- 172.21.15.57:0/1755326552 wait complete. 2023-10-06T22:14:19.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.269+0000 7f5663cffb80 20 client.15141 trim_cache size 0 max 16384 2023-10-06T22:14:19.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.287+0000 7f5663cffb80 1 Processor -- start 2023-10-06T22:14:19.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.287+0000 7f5663cffb80 1 -- start start 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6210 0x55a2b8feecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f5663cffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8db98d0 con 0x55a2b8fe7a10 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f5663cffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8f34ed0 con 0x55a2b8fe6210 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f5663cffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8edf4d0 con 0x55a2b8fe6b70 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fea940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42456/0 (socket says 172.21.15.57:42456) 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 -- 172.21.15.57:0/2369598182 learned_addr learned my addr 172.21.15.57:0/2369598182 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564d99a700 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e19b700 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6210 0x55a2b8feecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 -- 172.21.15.57:0/2369598182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 msgr2=0x55a2b8ff74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 -- 172.21.15.57:0/2369598182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6210 msgr2=0x55a2b8feecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6210 0x55a2b8feecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.288+0000 7f564e99c700 1 -- 172.21.15.57:0/2369598182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b8d4cc10 con 0x55a2b8fe7a10 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f564e99c700 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fea940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f564000ef00 tx=0x7f56400b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ee95b98198a21a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f563d3ff700 1 -- 172.21.15.57:0/2369598182 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564001b320 con 0x55a2b8fe7a10 2023-10-06T22:14:19.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f563d3ff700 1 -- 172.21.15.57:0/2369598182 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f564009a230 con 0x55a2b8fe7a10 2023-10-06T22:14:19.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f563d3ff700 1 -- 172.21.15.57:0/2369598182 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400b2260 con 0x55a2b8fe7a10 2023-10-06T22:14:19.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f5663cffb80 1 -- 172.21.15.57:0/2369598182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 msgr2=0x55a2b8fea940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fea940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f564000ef00 tx=0x7f56400b9040 comp rx=0 tx=0).stop 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f5663cffb80 1 -- 172.21.15.57:0/2369598182 shutdown_connections 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe7a10 0x55a2b8fea940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe6210 0x55a2b8feecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f5663cffb80 1 --2- 172.21.15.57:0/2369598182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.289+0000 7f5663cffb80 1 -- 172.21.15.57:0/2369598182 >> 172.21.15.57:0/2369598182 conn(0x55a2b8f5a210 msgr2=0x55a2b9004410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.290+0000 7f5663cffb80 1 -- 172.21.15.57:0/2369598182 shutdown_connections 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.290+0000 7f5663cffb80 1 -- 172.21.15.57:0/2369598182 wait complete. 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.290+0000 7f5663cffb80 1 -- start start 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.290+0000 7f5663cffb80 1 Processor -- start 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f5663cffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe7a10 0x55a2b8febf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f5663cffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8fed630 con 0x55a2b8fe6210 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f5663cffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8fec4d0 con 0x55a2b8fe7a10 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f5663cffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b8fec690 con 0x55a2b8fe6b70 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f5663cffb80 10 client.0 _ll_register_callbacks cb 0x7f5640095750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f564e19b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f564e19b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42460/0 (socket says 172.21.15.57:42460) 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f564e19b700 1 -- 172.21.15.57:0/731430699 learned_addr learned my addr 172.21.15.57:0/731430699 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f564d99a700 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.291+0000 7f564e99c700 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe7a10 0x55a2b8febf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f564e19b700 1 -- 172.21.15.57:0/731430699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 msgr2=0x55a2b8ff74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f564e19b700 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f564e19b700 1 -- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe7a10 msgr2=0x55a2b8febf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f564e19b700 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe7a10 0x55a2b8febf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f564e19b700 1 -- 172.21.15.57:0/731430699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56481a1310 con 0x55a2b8fe6210 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f564e19b700 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feba50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5644007560 tx=0x7f56440036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880a8af038275076 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f5631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f5631ffb700 1 -- 172.21.15.57:0/731430699 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644020040 con 0x55a2b8fe6210 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f5631ffb700 1 -- 172.21.15.57:0/731430699 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f5644000e40 con 0x55a2b8fe6210 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5644015070 con 0x55a2b8fe6210 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f5631ffb700 1 -- 172.21.15.57:0/731430699 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5644010ea0 con 0x55a2b8fe6210 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f5663cffb80 10 client.15147 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:19.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.292+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a2b8d8a200 con 0x55a2b8fe6210 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5631ffb700 1 -- 172.21.15.57:0/731430699 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5644015070 con 0x55a2b8fe6210 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5631ffb700 1 -- 172.21.15.57:0/731430699 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56440047d0 con 0x55a2b8fe6210 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5663cffb80 10 client.15147 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5663cffb80 10 client.15147 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5663cffb80 20 client.15147 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5663cffb80 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b9053fc0 0x55a2b9050400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5663cffb80 4 client.15147 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a2b8f31ae0 con 0x55a2b9053fc0 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.293+0000 7f564d99a700 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b9053fc0 0x55a2b9050400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f564d99a700 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b9053fc0 0x55a2b9050400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1c3af94256cd406 server_cookie=1fabe7449f098b16 in_seq=0 out_seq=0 2023-10-06T22:14:19.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5631ffb700 10 client.15147 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:14:19.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5631ffb700 1 -- 172.21.15.57:0/731430699 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a2b8f31ae0 con 0x55a2b9053fc0 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5631ffb700 10 client.15147 handle_command_reply: tid=0 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 client.15147 shutdown 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 msgr2=0x55a2b8feba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feba50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5644007560 tx=0x7f56440036a0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 shutdown_connections 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a2b9053fc0 0x55a2b9050400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2b8fe7a10 0x55a2b8febf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a2b8fe6210 0x55a2b8feba50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 --2- 172.21.15.57:0/731430699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2b8fe6b70 0x55a2b8ff74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 >> 172.21.15.57:0/731430699 conn(0x55a2b8f5a210 msgr2=0x55a2b8feddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5631ffb700 0 client.15147 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:14:19.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 shutdown_connections 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 1 -- 172.21.15.57:0/731430699 wait complete. 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.294+0000 7f5663cffb80 20 client.15147 trim_cache size 0 max 16384 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 4, 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24509, 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:14:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:14:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 89.527378389000006, 2023-10-06T22:14:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:14:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:14:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:14:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 112.01137790200001 2023-10-06T22:14:19.303 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f56340a1500 msgr2=0x7f56340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f56340a1500 0x7f56340a39c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5638005cd0 tx=0x7f5638005bc0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5648106be0 msgr2=0x7f56480ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5648106be0 0x7f56480ffe70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f564000ef30 tx=0x7f564000c3a0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 shutdown_connections 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f56340a1500 0x7f56340a39c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5648106be0 0x7f56480ffe70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101980 0x7f56481012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 --2- 172.21.15.57:0/3942562575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56481042b0 0x7f56480ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 >> 172.21.15.57:0/3942562575 conn(0x7f56480fb370 msgr2=0x7f56481025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 shutdown_connections 2023-10-06T22:14:19.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.295+0000 7f56503ff700 1 -- 172.21.15.57:0/3942562575 wait complete. 2023-10-06T22:14:19.315 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-06T22:14:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 -- 172.21.15.57:0/1672329650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102070 msgr2=0x7f17c01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 --2- 172.21.15.57:0/1672329650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102070 0x7f17c01024f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009990 tx=0x7f17b4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 -- 172.21.15.57:0/1672329650 shutdown_connections 2023-10-06T22:14:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 --2- 172.21.15.57:0/1672329650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17c0102a30 0x7f17c0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 --2- 172.21.15.57:0/1672329650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 0x7f17c01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 --2- 172.21.15.57:0/1672329650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102070 0x7f17c01024f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 -- 172.21.15.57:0/1672329650 >> 172.21.15.57:0/1672329650 conn(0x7f17c00fc430 msgr2=0x7f17c00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 -- 172.21.15.57:0/1672329650 shutdown_connections 2023-10-06T22:14:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.518+0000 7f17c63ff700 1 -- 172.21.15.57:0/1672329650 wait complete. 2023-10-06T22:14:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.519+0000 7f17c63ff700 1 Processor -- start 2023-10-06T22:14:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c63ff700 1 -- start start 2023-10-06T22:14:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c63ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 0x7f17c0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c63ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17c0102070 0x7f17c0199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c63ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102a30 0x7f17c01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c63ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01a09c0 con 0x7f17c0102a30 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c63ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01a0b00 con 0x7f17c0102070 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c63ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01a0c90 con 0x7f17c0100e90 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17bffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 0x7f17c0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17c499c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102a30 0x7f17c01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17bffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 0x7f17c0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45484/0 (socket says 172.21.15.57:45484) 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17bffff700 1 -- 172.21.15.57:0/963154857 learned_addr learned my addr 172.21.15.57:0/963154857 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.520+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17c0102070 0x7f17c0199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bffff700 1 -- 172.21.15.57:0/963154857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17c0102070 msgr2=0x7f17c0199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bffff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17c0102070 0x7f17c0199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bffff700 1 -- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102a30 msgr2=0x7f17c01a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bffff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102a30 0x7f17c01a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bffff700 1 -- 172.21.15.57:0/963154857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b40095f0 con 0x7f17c0100e90 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17c499c700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102a30 0x7f17c01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17c0102070 0x7f17c0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bffff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 0x7f17c0198e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17b000e970 tx=0x7f17b000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bd7fa700 1 -- 172.21.15.57:0/963154857 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0009790 con 0x7f17c0100e90 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c01a0f70 con 0x7f17c0100e90 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bd7fa700 1 -- 172.21.15.57:0/963154857 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b000c7e0 con 0x7f17c0100e90 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17bd7fa700 1 -- 172.21.15.57:0/963154857 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0003c70 con 0x7f17c0100e90 2023-10-06T22:14:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.521+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c01a1390 con 0x7f17c0100e90 2023-10-06T22:14:19.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.525+0000 7f17bd7fa700 1 -- 172.21.15.57:0/963154857 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f17b000cc50 con 0x7f17c0100e90 2023-10-06T22:14:19.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 Processor -- start 2023-10-06T22:14:19.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 -- start start 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810322630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x558810323430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810323a10 0x558810323e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881028a410 con 0x558810322b70 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881027bbd0 con 0x558810322210 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588102794f0 con 0x558810323a10 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x558810323430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810323a10 0x558810323e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17c499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810322630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.526+0000 7f17bd7fa700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f17a80a1480 0x7f17a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810323a10 0x558810323e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45496/0 (socket says 172.21.15.57:45496) 2023-10-06T22:14:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 -- 172.21.15.57:0/3078392908 learned_addr learned my addr 172.21.15.57:0/3078392908 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bd7fa700 1 -- 172.21.15.57:0/963154857 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f17b0014070 con 0x7f17c0100e90 2023-10-06T22:14:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 -- 172.21.15.57:0/3078392908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 msgr2=0x558810322630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17c499c700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f17a80a1480 0x7f17a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/3078392908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810322630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 -- 172.21.15.57:0/3078392908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 msgr2=0x558810323430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/3078392908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x558810323430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 -- 172.21.15.57:0/3078392908 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881009fd90 con 0x558810323a10 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/3078392908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810323a10 0x558810323e90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17c0100bb0 tx=0x7f17b4005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab14fe6b32613df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.527+0000 7f17c499c700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f17a80a1480 0x7f17a80a3940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f17b80060b0 tx=0x7f17b8004b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17a67fc700 1 -- 172.21.15.57:0/3078392908 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b401d050 con 0x558810323a10 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17a67fc700 1 -- 172.21.15.57:0/3078392908 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b401e040 con 0x558810323a10 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17a67fc700 1 -- 172.21.15.57:0/3078392908 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4005690 con 0x558810323a10 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3078392908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810323a10 msgr2=0x558810323e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3078392908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810323a10 0x558810323e90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17c0100bb0 tx=0x7f17b4005ea0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3078392908 shutdown_connections 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3078392908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810323a10 0x558810323e90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3078392908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810322630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3078392908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x558810323430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3078392908 >> 172.21.15.57:0/3078392908 conn(0x558810296210 msgr2=0x55881031d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3078392908 shutdown_connections 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.528+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3078392908 wait complete. 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.529+0000 7f17d9d64b80 1 -- start start 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.529+0000 7f17d9d64b80 1 Processor -- start 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.529+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322210 0x55881032bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x55881032c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x5588103334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558810274de0 con 0x558810322b70 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588102116c0 con 0x558810323a10 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558810211360 con 0x558810322210 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17d9d64b80 10 client.0 _ll_register_callbacks cb 0x55881006e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322210 0x55881032bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x55881032c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322210 0x55881032bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45504/0 (socket says 172.21.15.57:45504) 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 -- 172.21.15.57:0/3347096679 learned_addr learned my addr 172.21.15.57:0/3347096679 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17c499c700 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x5588103334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 -- 172.21.15.57:0/3347096679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 msgr2=0x5588103334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x5588103334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 -- 172.21.15.57:0/3347096679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 msgr2=0x55881032c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x55881032c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17bffff700 1 -- 172.21.15.57:0/3347096679 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c0109c80 con 0x558810322210 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.530+0000 7f17c499c700 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x5588103334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f17bffff700 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322210 0x55881032bf70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f17b00a2010 tx=0x7f17b0099b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d841a73d1b91fcf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f179f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f179f7fe700 1 -- 172.21.15.57:0/3347096679 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b009a610 con 0x558810322210 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558810314ec0 con 0x558810322210 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f179f7fe700 1 -- 172.21.15.57:0/3347096679 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f17b009a1b0 con 0x558810322210 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f179f7fe700 1 -- 172.21.15.57:0/3347096679 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b009e300 con 0x558810322210 2023-10-06T22:14:19.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f17d9d64b80 10 client.25189 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558810088c10 con 0x558810322210 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.531+0000 7f179f7fe700 1 -- 172.21.15.57:0/3347096679 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b0063020 con 0x558810322210 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f179f7fe700 1 -- 172.21.15.57:0/3347096679 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f17b00a20b0 con 0x558810322210 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f17d9d64b80 10 client.25189 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f17d9d64b80 10 client.25189 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f17d9d64b80 20 client.25189 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810337ba0 0x558810336640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f17d9d64b80 4 client.25189 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:14:19.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55881026ddc0 con 0x558810337ba0 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.532+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810337ba0 0x558810336640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.533+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810337ba0 0x558810336640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ce9a0e7e8e1a78f server_cookie=516264f9601e36b3 in_seq=0 out_seq=0 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.533+0000 7f179f7fe700 10 client.25189 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f179f7fe700 1 -- 172.21.15.57:0/3347096679 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55881026ddc0 con 0x558810337ba0 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f179f7fe700 10 client.25189 handle_command_reply: tid=0 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 client.25189 shutdown 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322210 msgr2=0x55881032bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322210 0x55881032bf70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f17b00a2010 tx=0x7f17b0099b10 comp rx=0 tx=0).stop 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 shutdown_connections 2023-10-06T22:14:19.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810337ba0 0x558810336640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x5588103334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322210 0x55881032bf70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/3347096679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322b70 0x55881032c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f179f7fe700 0 client.25189 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 >> 172.21.15.57:0/3347096679 conn(0x558810296210 msgr2=0x55881031cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 shutdown_connections 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 1 -- 172.21.15.57:0/3347096679 wait complete. 2023-10-06T22:14:19.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.534+0000 7f17d9d64b80 20 client.25189 trim_cache size 0 max 16384 2023-10-06T22:14:19.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.552+0000 7f17d9d64b80 1 Processor -- start 2023-10-06T22:14:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17d9d64b80 1 -- start start 2023-10-06T22:14:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810336640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x5588103334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810323a10 0x558810336b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881032b980 con 0x558810323a10 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588100f58d0 con 0x558810322210 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558810270ed0 con 0x558810322b70 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17c499c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810323a10 0x558810336b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17c499c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810323a10 0x558810336b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42510/0 (socket says 172.21.15.57:42510) 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17c499c700 1 -- 172.21.15.57:0/2190623958 learned_addr learned my addr 172.21.15.57:0/2190623958 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x5588103334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.553+0000 7f17bffff700 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810336640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17c499c700 1 -- 172.21.15.57:0/2190623958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 msgr2=0x5588103334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17c499c700 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x5588103334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17c499c700 1 -- 172.21.15.57:0/2190623958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 msgr2=0x558810336640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17c499c700 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810336640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17c499c700 1 -- 172.21.15.57:0/2190623958 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588100c6200 con 0x558810323a10 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x5588103334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17bffff700 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810336640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17c499c700 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810323a10 0x558810336b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e5a0 tx=0x7f17b8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c445a5b35fefd88e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17a6ffd700 1 -- 172.21.15.57:0/2190623958 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801b650 con 0x558810323a10 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17a6ffd700 1 -- 172.21.15.57:0/2190623958 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b8023040 con 0x558810323a10 2023-10-06T22:14:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.554+0000 7f17a6ffd700 1 -- 172.21.15.57:0/2190623958 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801ab90 con 0x558810323a10 2023-10-06T22:14:19.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 -- 172.21.15.57:0/2190623958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810323a10 msgr2=0x558810336b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810323a10 0x558810336b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e5a0 tx=0x7f17b8009920 comp rx=0 tx=0).stop 2023-10-06T22:14:19.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 -- 172.21.15.57:0/2190623958 shutdown_connections 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810323a10 0x558810336b80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810322210 0x558810336640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/2190623958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x5588103334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 -- 172.21.15.57:0/2190623958 >> 172.21.15.57:0/2190623958 conn(0x558810296210 msgr2=0x558810326880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 -- 172.21.15.57:0/2190623958 shutdown_connections 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 -- 172.21.15.57:0/2190623958 wait complete. 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.555+0000 7f17d9d64b80 1 -- start start 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 1 Processor -- start 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322210 0x55881032e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x55881032e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x55881038b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588103335d0 con 0x558810322210 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558810333740 con 0x558810323a10 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881038bad0 con 0x558810322b70 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17d9d64b80 10 client.0 _ll_register_callbacks cb 0x7f17b009a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322210 0x55881032e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322210 0x55881032e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42512/0 (socket says 172.21.15.57:42512) 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17bffff700 1 -- 172.21.15.57:0/175102161 learned_addr learned my addr 172.21.15.57:0/175102161 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17c499c700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x55881038b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.556+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x55881032e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17bffff700 1 -- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 msgr2=0x55881032e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17bffff700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x55881032e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17bffff700 1 -- 172.21.15.57:0/175102161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 msgr2=0x55881038b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17bffff700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x55881038b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17bffff700 1 -- 172.21.15.57:0/175102161 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b8009280 con 0x558810322210 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17c499c700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x55881038b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17bffff700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322210 0x55881032e060 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f17b0096c20 tx=0x7f17b00a22e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c67bb62ab1adb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f179f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f179f7fe700 1 -- 172.21.15.57:0/175102161 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b00b84c0 con 0x558810322210 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f179f7fe700 1 -- 172.21.15.57:0/175102161 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f17b0064080 con 0x558810322210 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17b401e040 con 0x558810322210 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f179f7fe700 1 -- 172.21.15.57:0/175102161 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b0003dd0 con 0x558810322210 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17d9d64b80 10 client.15165 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.557+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55881009e260 con 0x558810322210 2023-10-06T22:14:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f179f7fe700 1 -- 172.21.15.57:0/175102161 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b401e040 con 0x558810322210 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f179f7fe700 1 -- 172.21.15.57:0/175102161 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f17b0095fa0 con 0x558810322210 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f17d9d64b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f17d9d64b80 10 client.15165 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f17d9d64b80 20 client.15165 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810390710 0x55881038cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f17d9d64b80 4 client.15165 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55881026dae0 con 0x558810390710 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.558+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810390710 0x55881038cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f179f7fe700 10 client.15165 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:14:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17bf7fe700 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810390710 0x55881038cbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f589ff4e97b0ad0 server_cookie=5587a77ca3ec7dd1 in_seq=0 out_seq=0 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f179f7fe700 1 -- 172.21.15.57:0/175102161 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55881026dae0 con 0x558810390710 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f179f7fe700 10 client.15165 handle_command_reply: tid=0 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 client.15165 shutdown 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322210 msgr2=0x55881032e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322210 0x55881032e060 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f17b0096c20 tx=0x7f17b00a22e0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 shutdown_connections 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558810390710 0x55881038cbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558810323a10 0x55881038b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558810322210 0x55881032e060 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f179f7fe700 0 client.15165 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:14:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 --2- 172.21.15.57:0/175102161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558810322b70 0x55881032e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 >> 172.21.15.57:0/175102161 conn(0x558810296210 msgr2=0x558810336640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 shutdown_connections 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.559+0000 7f17d9d64b80 1 -- 172.21.15.57:0/175102161 wait complete. 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17d9d64b80 20 client.15165 trim_cache size 0 max 16384 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 3, 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24517, 2023-10-06T22:14:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 90.801212406999994, 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 116.763841536 2023-10-06T22:14:19.569 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:14:19.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f17a80a1480 msgr2=0x7f17a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f17a80a1480 0x7f17a80a3940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f17b80060b0 tx=0x7f17b8004b40 comp rx=0 tx=0).stop 2023-10-06T22:14:19.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 msgr2=0x7f17c0198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 0x7f17c0198e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17b000e970 tx=0x7f17b000ec80 comp rx=0 tx=0).stop 2023-10-06T22:14:19.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 shutdown_connections 2023-10-06T22:14:19.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f17a80a1480 0x7f17a80a3940 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17c0102a30 0x7f17c01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c0100e90 0x7f17c0198e40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 --2- 172.21.15.57:0/963154857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17c0102070 0x7f17c0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 >> 172.21.15.57:0/963154857 conn(0x7f17c00fc430 msgr2=0x7f17c0107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 shutdown_connections 2023-10-06T22:14:19.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.560+0000 7f17c63ff700 1 -- 172.21.15.57:0/963154857 wait complete. 2023-10-06T22:14:19.580 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T22:14:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.787+0000 7f2bdddff700 1 -- 172.21.15.57:0/911634542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 msgr2=0x7f2bd8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.787+0000 7f2bdddff700 1 --2- 172.21.15.57:0/911634542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 0x7f2bd8109350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009f90 tx=0x7f2bd000b330 comp rx=0 tx=0).stop 2023-10-06T22:14:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.787+0000 7f2bdddff700 1 -- 172.21.15.57:0/911634542 shutdown_connections 2023-10-06T22:14:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.787+0000 7f2bdddff700 1 --2- 172.21.15.57:0/911634542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 0x7f2bd8109350 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.787+0000 7f2bdddff700 1 --2- 172.21.15.57:0/911634542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 0x7f2bd81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.787+0000 7f2bdddff700 1 --2- 172.21.15.57:0/911634542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd8102070 0x7f2bd81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.787+0000 7f2bdddff700 1 -- 172.21.15.57:0/911634542 >> 172.21.15.57:0/911634542 conn(0x7f2bd80fc430 msgr2=0x7f2bd80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.788+0000 7f2bdddff700 1 -- 172.21.15.57:0/911634542 shutdown_connections 2023-10-06T22:14:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.788+0000 7f2bdddff700 1 -- 172.21.15.57:0/911634542 wait complete. 2023-10-06T22:14:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.789+0000 7f2bdddff700 1 Processor -- start 2023-10-06T22:14:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.789+0000 7f2bdddff700 1 -- start start 2023-10-06T22:14:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bdddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 0x7f2bd8196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bdddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd8102070 0x7f2bd8197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bdddff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 0x7f2bd819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bdddff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd819e860 con 0x7f2bd8102a30 2023-10-06T22:14:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bdddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd819e9d0 con 0x7f2bd8102070 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bdddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd819eb90 con 0x7f2bd8100e90 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 0x7f2bd8196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 0x7f2bd819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 0x7f2bd8196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45530/0 (socket says 172.21.15.57:45530) 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/2740142610 learned_addr learned my addr 172.21.15.57:0/2740142610 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bd77fe700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd8102070 0x7f2bd8197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2740142610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 msgr2=0x7f2bd8196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 0x7f2bd8196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2740142610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd8102070 msgr2=0x7f2bd8197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd8102070 0x7f2bd8197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2740142610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0009c40 con 0x7f2bd8102a30 2023-10-06T22:14:19.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.790+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 0x7f2bd8196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:19.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.791+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 0x7f2bd819e290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000b6c0 tx=0x7f2bd0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.791+0000 7f2bd57fa700 1 -- 172.21.15.57:0/2740142610 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000e040 con 0x7f2bd8102a30 2023-10-06T22:14:19.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.791+0000 7f2bd57fa700 1 -- 172.21.15.57:0/2740142610 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd000bbf0 con 0x7f2bd8102a30 2023-10-06T22:14:19.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.791+0000 7f2bd57fa700 1 -- 172.21.15.57:0/2740142610 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0005040 con 0x7f2bd8102a30 2023-10-06T22:14:19.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.791+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd819ee10 con 0x7f2bd8102a30 2023-10-06T22:14:19.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.791+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd819f180 con 0x7f2bd8102a30 2023-10-06T22:14:19.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.794+0000 7f2bd57fa700 1 -- 172.21.15.57:0/2740142610 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2bd001d470 con 0x7f2bd8102a30 2023-10-06T22:14:19.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 Processor -- start 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bd57fa700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bc40a1480 0x7f2bc40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 -- start start 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bd57fa700 1 -- 172.21.15.57:0/2740142610 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f2bd00ce030 con 0x7f2bd8102a30 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bd77fe700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bc40a1480 0x7f2bc40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e37630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e38430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e38e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5d9f410 con 0x55d0a5e37b70 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5d90bd0 con 0x55d0a5e37210 2023-10-06T22:14:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.796+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5d8e4f0 con 0x55d0a5e38a10 2023-10-06T22:14:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e37630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e38e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e37630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59448/0 (socket says 172.21.15.57:59448) 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/2212867703 learned_addr learned my addr 172.21.15.57:0/2212867703 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd77fe700 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e38430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2212867703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 msgr2=0x55d0a5e37630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e37630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2212867703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 msgr2=0x55d0a5e38430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e38430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd77fe700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bc40a1480 0x7f2bc40a3940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8100bb0 tx=0x7f2bc0011720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2212867703 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a5bb4d90 con 0x55d0a5e38a10 2023-10-06T22:14:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e38e90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009f90 tx=0x7f2bd0006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911987e601ee5add server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.797+0000 7f2bcdffb700 1 -- 172.21.15.57:0/2212867703 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd009b410 con 0x55d0a5e38a10 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bcdffb700 1 -- 172.21.15.57:0/2212867703 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd009f560 con 0x55d0a5e38a10 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bcdffb700 1 -- 172.21.15.57:0/2212867703 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd009b8d0 con 0x55d0a5e38a10 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2212867703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 msgr2=0x55d0a5e38e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e38e90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009f90 tx=0x7f2bd0006000 comp rx=0 tx=0).stop 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2212867703 shutdown_connections 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e38e90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e37630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2212867703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e38430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2212867703 >> 172.21.15.57:0/2212867703 conn(0x55d0a5dab210 msgr2=0x55d0a5e320f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2212867703 shutdown_connections 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.798+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2212867703 wait complete. 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.799+0000 7f2bf1690b80 1 -- start start 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.799+0000 7f2bf1690b80 1 Processor -- start 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e37210 0x55d0a5e40e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e38a10 0x55d0a5e48380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5d89de0 con 0x55d0a5e38a10 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5d266c0 con 0x55d0a5e37b70 2023-10-06T22:14:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5d26360 con 0x55d0a5e37210 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bf1690b80 10 client.0 _ll_register_callbacks cb 0x55d0a5b837b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e38a10 0x55d0a5e48380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e38a10 0x55d0a5e48380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42546/0 (socket says 172.21.15.57:42546) 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2505396511 learned_addr learned my addr 172.21.15.57:0/2505396511 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e37210 0x55d0a5e40e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd77fe700 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e37210 msgr2=0x55d0a5e40e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e37210 0x55d0a5e40e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2505396511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 msgr2=0x55d0a5e41340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e41340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.800+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2505396511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd819f180 con 0x55d0a5e38a10 2023-10-06T22:14:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e38a10 0x55d0a5e48380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00cbed0 tx=0x7f2bd00094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ed605ab0ece132 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2505396511 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00a3710 con 0x55d0a5e38a10 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2505396511 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f2bd00b3e40 con 0x55d0a5e38a10 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0a5e29ec0 con 0x55d0a5e38a10 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2505396511 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd009b410 con 0x55d0a5e38a10 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bf1690b80 10 client.15177 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d0a5b9dc10 con 0x55d0a5e38a10 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2505396511 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd0080020 con 0x55d0a5e38a10 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.801+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2505396511 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2bd001d720 con 0x55d0a5e38a10 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bf1690b80 10 client.15177 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bf1690b80 10 client.15177 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bf1690b80 20 client.15177 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5e4c450 0x55d0a5e4b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bf1690b80 4 client.15177 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0a5d82dc0 con 0x55d0a5e4c450 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5e4c450 0x55d0a5e4b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5e4c450 0x55d0a5e4b2b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=446725748dfcbdf0 server_cookie=c49c6d6be138fdbc in_seq=0 out_seq=0 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.802+0000 7f2bb6ffd700 10 client.15177 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2505396511 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d0a5d82dc0 con 0x55d0a5e4c450 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bb6ffd700 10 client.15177 handle_command_reply: tid=0 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 client.15177 shutdown 2023-10-06T22:14:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e38a10 msgr2=0x55d0a5e48380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e38a10 0x55d0a5e48380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00cbed0 tx=0x7f2bd00094f0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 shutdown_connections 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5e4c450 0x55d0a5e4b2b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e38a10 0x55d0a5e48380 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e37210 0x55d0a5e40e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bb6ffd700 0 client.15177 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2505396511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e41340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.803+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 >> 172.21.15.57:0/2505396511 conn(0x55d0a5dab210 msgr2=0x55d0a5e31f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.804+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 shutdown_connections 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.804+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2505396511 wait complete. 2023-10-06T22:14:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.804+0000 7f2bf1690b80 20 client.15177 trim_cache size 0 max 16384 2023-10-06T22:14:19.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.823+0000 7f2bf1690b80 1 Processor -- start 2023-10-06T22:14:19.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bf1690b80 1 -- start start 2023-10-06T22:14:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e3fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e48340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5c0a8d0 con 0x55d0a5e37b70 2023-10-06T22:14:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5e3f8b0 con 0x55d0a5e37210 2023-10-06T22:14:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5e3f610 con 0x55d0a5e38a10 2023-10-06T22:14:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bd77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e48340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bd77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e48340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42560/0 (socket says 172.21.15.57:42560) 2023-10-06T22:14:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bd77fe700 1 -- 172.21.15.57:0/830722227 learned_addr learned my addr 172.21.15.57:0/830722227 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.824+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e3fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bd7fff700 1 -- 172.21.15.57:0/830722227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 msgr2=0x55d0a5e3fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e3fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bd7fff700 1 -- 172.21.15.57:0/830722227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 msgr2=0x55d0a5e48340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e48340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bd7fff700 1 -- 172.21.15.57:0/830722227 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a5bdb200 con 0x55d0a5e38a10 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bd77fe700 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e48340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e3ba10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009c10 tx=0x7f2bd0006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcccd9b4e6a262c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bce7fc700 1 -- 172.21.15.57:0/830722227 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0099890 con 0x55d0a5e38a10 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bce7fc700 1 -- 172.21.15.57:0/830722227 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd0080070 con 0x55d0a5e38a10 2023-10-06T22:14:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.825+0000 7f2bce7fc700 1 -- 172.21.15.57:0/830722227 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00a3280 con 0x55d0a5e38a10 2023-10-06T22:14:19.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 -- 172.21.15.57:0/830722227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 msgr2=0x55d0a5e3ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e3ba10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009c10 tx=0x7f2bd0006000 comp rx=0 tx=0).stop 2023-10-06T22:14:19.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 -- 172.21.15.57:0/830722227 shutdown_connections 2023-10-06T22:14:19.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e3ba10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37210 0x55d0a5e3fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/830722227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37b70 0x55d0a5e48340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 -- 172.21.15.57:0/830722227 >> 172.21.15.57:0/830722227 conn(0x55d0a5dab210 msgr2=0x55d0a5e3ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 -- 172.21.15.57:0/830722227 shutdown_connections 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.826+0000 7f2bf1690b80 1 -- 172.21.15.57:0/830722227 wait complete. 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 -- start start 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 Processor -- start 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37210 0x55d0a5e47f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e44ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5e48720 con 0x55d0a5e37210 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5e45450 con 0x55d0a5e37b70 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a5e45610 con 0x55d0a5e38a10 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bf1690b80 10 client.0 _ll_register_callbacks cb 0x7f2bc0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bd77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.827+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37210 0x55d0a5e47f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e44ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e449a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59478/0 (socket says 172.21.15.57:59478) 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37210 0x55d0a5e47f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42568/0 (socket says 172.21.15.57:42568) 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd77fe700 1 -- 172.21.15.57:0/2499759758 learned_addr learned my addr 172.21.15.57:0/2499759758 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2499759758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 msgr2=0x55d0a5e449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2499759758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37210 msgr2=0x55d0a5e47f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37210 0x55d0a5e47f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd7fff700 1 -- 172.21.15.57:0/2499759758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0a5b9dc10 con 0x55d0a5e38a10 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37210 0x55d0a5e47f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bd7fff700 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e44ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00978c0 tx=0x7f2bd001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=692c44040d9b6948 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2499759758 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd009b410 con 0x55d0a5e38a10 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2499759758 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f2bd00b3e40 con 0x55d0a5e38a10 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.828+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd009f560 con 0x55d0a5e38a10 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2499759758 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd00a1310 con 0x55d0a5e38a10 2023-10-06T22:14:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 10 client.25213 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d0a5bb3260 con 0x55d0a5e38a10 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2499759758 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd009f560 con 0x55d0a5e38a10 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2499759758 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2bd009a370 con 0x55d0a5e38a10 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 10 client.25213 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 10 client.25213 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 20 client.25213 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5ea4a40 0x55d0a5ea0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 4 client.25213 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d0a5d82ae0 con 0x55d0a5ea4a40 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.829+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5ea4a40 0x55d0a5ea0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5ea4a40 0x55d0a5ea0f30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c6729f9004cebc3 server_cookie=821b4d919498318f in_seq=0 out_seq=0 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bb6ffd700 10 client.25213 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bb6ffd700 1 -- 172.21.15.57:0/2499759758 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d0a5d82ae0 con 0x55d0a5ea4a40 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bb6ffd700 10 client.25213 handle_command_reply: tid=0 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 client.25213 shutdown 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 msgr2=0x55d0a5e44ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e44ee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00978c0 tx=0x7f2bd001a000 comp rx=0 tx=0).stop 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 shutdown_connections 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55d0a5ea4a40 0x55d0a5ea0f30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0a5e38a10 0x55d0a5e44ee0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0a5e37210 0x55d0a5e47f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bb6ffd700 0 client.25213 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 --2- 172.21.15.57:0/2499759758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0a5e37b70 0x55d0a5e449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 >> 172.21.15.57:0/2499759758 conn(0x55d0a5dab210 msgr2=0x55d0a5e3ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 shutdown_connections 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.830+0000 7f2bf1690b80 1 -- 172.21.15.57:0/2499759758 wait complete. 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bf1690b80 20 client.25213 trim_cache size 0 max 16384 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:14:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24529, 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 92.089502015999997, 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 114.751382404 2023-10-06T22:14:19.842 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:14:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bc40a1480 msgr2=0x7f2bc40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bc40a1480 0x7f2bc40a3940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8100bb0 tx=0x7f2bc0011720 comp rx=0 tx=0).stop 2023-10-06T22:14:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 msgr2=0x7f2bd819e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 0x7f2bd819e290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000b6c0 tx=0x7f2bd0004c60 comp rx=0 tx=0).stop 2023-10-06T22:14:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 shutdown_connections 2023-10-06T22:14:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2bc40a1480 0x7f2bc40a3940 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bd8102a30 0x7f2bd819e290 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd8100e90 0x7f2bd8196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 --2- 172.21.15.57:0/2740142610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd8102070 0x7f2bd8197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 >> 172.21.15.57:0/2740142610 conn(0x7f2bd80fc430 msgr2=0x7f2bd80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 shutdown_connections 2023-10-06T22:14:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:19.831+0000 7f2bdddff700 1 -- 172.21.15.57:0/2740142610 wait complete. 2023-10-06T22:14:19.849 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:19 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2513043702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:19.849 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:19 smithi057 ceph-mon[98492]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-06T22:14:19.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 -- 172.21.15.57:0/744024435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 msgr2=0x7f1cf81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/744024435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf81012b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009990 tx=0x7f1ce8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 -- 172.21.15.57:0/744024435 shutdown_connections 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/744024435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cf8102a30 0x7f1cf8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/744024435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf81012b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/744024435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf8102070 0x7f1cf81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 -- 172.21.15.57:0/744024435 >> 172.21.15.57:0/744024435 conn(0x7f1cf80fc430 msgr2=0x7f1cf80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 -- 172.21.15.57:0/744024435 shutdown_connections 2023-10-06T22:14:20.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.053+0000 7f1cfcbff700 1 -- 172.21.15.57:0/744024435 wait complete. 2023-10-06T22:14:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.054+0000 7f1cfcbff700 1 Processor -- start 2023-10-06T22:14:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.054+0000 7f1cfcbff700 1 -- start start 2023-10-06T22:14:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cfcbff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cfcbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf8102070 0x7f1cf80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cfcbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cf8102a30 0x7f1cf80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cfcbff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8073f80 con 0x7f1cf8100e90 2023-10-06T22:14:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cfcbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80740f0 con 0x7f1cf8102a30 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cfcbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf81a0ce0 con 0x7f1cf8102070 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf8074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42574/0 (socket says 172.21.15.57:42574) 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 -- 172.21.15.57:0/3425392594 learned_addr learned my addr 172.21.15.57:0/3425392594 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cf8102a30 0x7f1cf80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf8102070 0x7f1cf80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 -- 172.21.15.57:0/3425392594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf8102070 msgr2=0x7f1cf80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf8102070 0x7f1cf80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 -- 172.21.15.57:0/3425392594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cf8102a30 msgr2=0x7f1cf80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cf8102a30 0x7f1cf80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf659c700 1 -- 172.21.15.57:0/3425392594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec009520 con 0x7f1cf8100e90 2023-10-06T22:14:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf8102070 0x7f1cf80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:14:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.055+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cf8102a30 0x7f1cf80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.056+0000 7f1cf659c700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf8074c60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005720 tx=0x7f1ce8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.056+0000 7f1ce77fe700 1 -- 172.21.15.57:0/3425392594 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce801d070 con 0x7f1cf8100e90 2023-10-06T22:14:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.056+0000 7f1ce77fe700 1 -- 172.21.15.57:0/3425392594 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce8021410 con 0x7f1cf8100e90 2023-10-06T22:14:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.056+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce80095f0 con 0x7f1cf8100e90 2023-10-06T22:14:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.056+0000 7f1ce77fe700 1 -- 172.21.15.57:0/3425392594 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8017650 con 0x7f1cf8100e90 2023-10-06T22:14:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.056+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf81a11a0 con 0x7f1cf8100e90 2023-10-06T22:14:20.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.059+0000 7f1ce77fe700 1 -- 172.21.15.57:0/3425392594 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1ce8021580 con 0x7f1cf8100e90 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.060+0000 7f1d105e3b80 1 Processor -- start 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1ce77fe700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1ce00a1480 0x7f1ce00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1d105e3b80 1 -- start start 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1ce77fe700 1 -- 172.21.15.57:0/3425392594 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1ce80ce190 con 0x7f1cf8100e90 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1ce00a1480 0x7f1ce00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd5b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd5c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd5ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfcc3410 con 0x55c9dfd5ca10 2023-10-06T22:14:20.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfcb4bd0 con 0x55c9dfd5b210 2023-10-06T22:14:20.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfcb24f0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1cf6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd5ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1cf6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd5ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42588/0 (socket says 172.21.15.57:42588) 2023-10-06T22:14:20.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1cf6d9d700 1 -- 172.21.15.57:0/2444882461 learned_addr learned my addr 172.21.15.57:0/2444882461 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd5c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.061+0000 7f1cf659c700 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd5b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf659c700 1 -- 172.21.15.57:0/2444882461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5bb70 msgr2=0x55c9dfd5c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf659c700 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd5c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf659c700 1 -- 172.21.15.57:0/2444882461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 msgr2=0x55c9dfd5ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf659c700 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd5ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf659c700 1 -- 172.21.15.57:0/2444882461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9dfad8d90 con 0x55c9dfd5b210 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd5c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd5ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1ce00a1480 0x7f1ce00a3940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00094f0 tx=0x7f1cf00092c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1cf659c700 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd5b630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80cbc60 tx=0x7f1ce8097980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f403f16c0db1d9a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1ce4ff9700 1 -- 172.21.15.57:0/2444882461 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce809bc10 con 0x55c9dfd5b210 2023-10-06T22:14:20.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1ce4ff9700 1 -- 172.21.15.57:0/2444882461 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce809b510 con 0x55c9dfd5b210 2023-10-06T22:14:20.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.062+0000 7f1ce4ff9700 1 -- 172.21.15.57:0/2444882461 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce809b9d0 con 0x55c9dfd5b210 2023-10-06T22:14:20.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 -- 172.21.15.57:0/2444882461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 msgr2=0x55c9dfd5b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd5b630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80cbc60 tx=0x7f1ce8097980 comp rx=0 tx=0).stop 2023-10-06T22:14:20.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 -- 172.21.15.57:0/2444882461 shutdown_connections 2023-10-06T22:14:20.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd5ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd5b630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/2444882461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd5c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 -- 172.21.15.57:0/2444882461 >> 172.21.15.57:0/2444882461 conn(0x55c9dfccf210 msgr2=0x55c9dfd560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 -- 172.21.15.57:0/2444882461 shutdown_connections 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.063+0000 7f1d105e3b80 1 -- 172.21.15.57:0/2444882461 wait complete. 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 -- start start 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 Processor -- start 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfcadde0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfc4a6c0 con 0x55c9dfd5b210 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.064+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfc4a360 con 0x55c9dfd5ca10 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1d105e3b80 10 client.0 _ll_register_callbacks cb 0x55c9dfaa77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd651e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42592/0 (socket says 172.21.15.57:42592) 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 -- 172.21.15.57:0/3616353762 learned_addr learned my addr 172.21.15.57:0/3616353762 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf659c700 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 -- 172.21.15.57:0/3616353762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 msgr2=0x55c9dfd6c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 -- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 msgr2=0x55c9dfd64ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd64ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 -- 172.21.15.57:0/3616353762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec0099d0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd651e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1cec005c50 tx=0x7f1cec013cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38242e8d3e489a4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cddffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.065+0000 7f1cddffb700 1 -- 172.21.15.57:0/3616353762 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec011ce0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9dfd4dec0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1cddffb700 1 -- 172.21.15.57:0/3616353762 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1cec011e40 con 0x55c9dfd5bb70 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1cddffb700 1 -- 172.21.15.57:0/3616353762 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cec004830 con 0x55c9dfd5bb70 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 10 client.15195 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f1cf0007de0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1cddffb700 1 -- 172.21.15.57:0/3616353762 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9dfd4dec0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1cddffb700 1 -- 172.21.15.57:0/3616353762 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1cec024050 con 0x55c9dfd5bb70 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 10 client.15195 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 10 client.15195 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 20 client.15195 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfd70950 0x55c9dfd6f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 4 client.15195 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.066+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9dfca6dc0 con 0x55c9dfd70950 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.067+0000 7f1cf659c700 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfd70950 0x55c9dfd6f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.067+0000 7f1cf659c700 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfd70950 0x55c9dfd6f3d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86c635bc62cd1221 server_cookie=60c1a5f88fe9ced3 in_seq=0 out_seq=0 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.067+0000 7f1cddffb700 10 client.15195 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1cddffb700 1 -- 172.21.15.57:0/3616353762 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9dfca6dc0 con 0x55c9dfd70950 2023-10-06T22:14:20.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1cddffb700 10 client.15195 handle_command_reply: tid=0 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 client.15195 shutdown 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5bb70 msgr2=0x55c9dfd651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd651e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1cec005c50 tx=0x7f1cec013cd0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 shutdown_connections 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfd70950 0x55c9dfd6f3d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5b210 0x55c9dfd64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/3616353762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd651e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1cddffb700 0 client.15195 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 >> 172.21.15.57:0/3616353762 conn(0x55c9dfccf210 msgr2=0x55c9dfd55f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 shutdown_connections 2023-10-06T22:14:20.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.068+0000 7f1d105e3b80 1 -- 172.21.15.57:0/3616353762 wait complete. 2023-10-06T22:14:20.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.069+0000 7f1d105e3b80 20 client.15195 trim_cache size 0 max 16384 2023-10-06T22:14:20.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:19 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2513043702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:20.078 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:19 smithi190 ceph-mon[134891]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-06T22:14:20.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 Processor -- start 2023-10-06T22:14:20.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 -- start start 2023-10-06T22:14:20.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5b210 0x55c9dfd6ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd62e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfd646b0 con 0x55c9dfd5b210 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfb2e8d0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfc544d0 con 0x55c9dfd5ca10 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5b210 0x55c9dfd6ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5b210 0x55c9dfd6ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42600/0 (socket says 172.21.15.57:42600) 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1cf659c700 1 -- 172.21.15.57:0/419987601 learned_addr learned my addr 172.21.15.57:0/419987601 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.087+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd62e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1cf659c700 1 -- 172.21.15.57:0/419987601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 msgr2=0x55c9dfd62e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1cf659c700 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd62e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1cf659c700 1 -- 172.21.15.57:0/419987601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 msgr2=0x55c9dfd6c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1cf659c700 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1cf659c700 1 -- 172.21.15.57:0/419987601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf81a11a0 con 0x55c9dfd5b210 2023-10-06T22:14:20.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1cf659c700 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5b210 0x55c9dfd6ca50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8097a10 tx=0x7f1ce8021c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669ea401d7c9de70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1ce57fa700 1 -- 172.21.15.57:0/419987601 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80b9040 con 0x55c9dfd5b210 2023-10-06T22:14:20.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1ce57fa700 1 -- 172.21.15.57:0/419987601 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce8099650 con 0x55c9dfd5b210 2023-10-06T22:14:20.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.088+0000 7f1ce57fa700 1 -- 172.21.15.57:0/419987601 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce809b770 con 0x55c9dfd5b210 2023-10-06T22:14:20.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 -- 172.21.15.57:0/419987601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5b210 msgr2=0x55c9dfd6ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5b210 0x55c9dfd6ca50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8097a10 tx=0x7f1ce8021c20 comp rx=0 tx=0).stop 2023-10-06T22:14:20.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 -- 172.21.15.57:0/419987601 shutdown_connections 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd62e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5b210 0x55c9dfd6ca50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/419987601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 -- 172.21.15.57:0/419987601 >> 172.21.15.57:0/419987601 conn(0x55c9dfccf210 msgr2=0x55c9dfd55f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 -- 172.21.15.57:0/419987601 shutdown_connections 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 -- 172.21.15.57:0/419987601 wait complete. 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.089+0000 7f1d105e3b80 1 -- start start 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 1 Processor -- start 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5b210 0x55c9dfd69170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfd6c180 con 0x55c9dfd5ca10 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfd6c2c0 con 0x55c9dfd5bb70 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9dfd6c480 con 0x55c9dfd5b210 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1d105e3b80 10 client.0 _ll_register_callbacks cb 0x7f1cec0134f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1cf6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1cf6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42612/0 (socket says 172.21.15.57:42612) 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5b210 0x55c9dfd69170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1cf6d9d700 1 -- 172.21.15.57:0/1704022804 learned_addr learned my addr 172.21.15.57:0/1704022804 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.090+0000 7f1cf5d9b700 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cf6d9d700 1 -- 172.21.15.57:0/1704022804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5b210 msgr2=0x55c9dfd69170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5b210 0x55c9dfd69170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cf6d9d700 1 -- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 msgr2=0x55c9dfd6b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cf6d9d700 1 -- 172.21.15.57:0/1704022804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9dfad8d90 con 0x55c9dfd5ca10 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cf6d9d700 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6bc00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1cf001b6e0 tx=0x7f1cf0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dad3cc9fa5a864 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cddffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cddffb700 1 -- 172.21.15.57:0/1704022804 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000e050 con 0x55c9dfd5ca10 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ce809b510 con 0x55c9dfd5ca10 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cddffb700 1 -- 172.21.15.57:0/1704022804 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1cf0021040 con 0x55c9dfd5ca10 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1cddffb700 1 -- 172.21.15.57:0/1704022804 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf0007600 con 0x55c9dfd5ca10 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1d105e3b80 10 client.15201 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:20.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.091+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c9dfac1c10 con 0x55c9dfd5ca10 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1cddffb700 1 -- 172.21.15.57:0/1704022804 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce809b510 con 0x55c9dfd5ca10 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1cddffb700 1 -- 172.21.15.57:0/1704022804 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1cf0027c70 con 0x55c9dfd5ca10 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1d105e3b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1d105e3b80 10 client.15201 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1d105e3b80 20 client.15201 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfdc9430 0x55c9dfdc5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1d105e3b80 4 client.15201 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c9dfca6ae0 con 0x55c9dfdc9430 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.092+0000 7f1cf659c700 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfdc9430 0x55c9dfdc5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1cf659c700 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfdc9430 0x55c9dfdc5890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba6915f9a8a214ab server_cookie=ef3369bbf3fb8bd7 in_seq=0 out_seq=0 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1cddffb700 10 client.15201 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:14:20.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1cddffb700 1 -- 172.21.15.57:0/1704022804 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c9dfca6ae0 con 0x55c9dfdc9430 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1cddffb700 10 client.15201 handle_command_reply: tid=0 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 client.15201 shutdown 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 msgr2=0x55c9dfd6bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6bc00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1cf001b6e0 tx=0x7f1cf0004b40 comp rx=0 tx=0).stop 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 shutdown_connections 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55c9dfdc9430 0x55c9dfdc5890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9dfd5ca10 0x55c9dfd6bc00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c9dfd5b210 0x55c9dfd69170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 --2- 172.21.15.57:0/1704022804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9dfd5bb70 0x55c9dfd6b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1cddffb700 0 client.15201 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 >> 172.21.15.57:0/1704022804 conn(0x55c9dfccf210 msgr2=0x55c9dfd62e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 shutdown_connections 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.093+0000 7f1d105e3b80 1 -- 172.21.15.57:0/1704022804 wait complete. 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1d105e3b80 20 client.15201 trim_cache size 0 max 16384 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2023-10-06T22:14:20.103 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24551, 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 93.362469279999999, 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 101.74168697899999 2023-10-06T22:14:20.104 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:14:20.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1ce00a1480 msgr2=0x7f1ce00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1ce00a1480 0x7f1ce00a3940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00094f0 tx=0x7f1cf00092c0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 msgr2=0x7f1cf8074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf8074c60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005720 tx=0x7f1ce8009580 comp rx=0 tx=0).stop 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 shutdown_connections 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f1ce00a1480 0x7f1ce00a3940 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cf8102a30 0x7f1cf80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cf8100e90 0x7f1cf8074c60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 --2- 172.21.15.57:0/3425392594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf8102070 0x7f1cf80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 >> 172.21.15.57:0/3425392594 conn(0x7f1cf80fc430 msgr2=0x7f1cf80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 shutdown_connections 2023-10-06T22:14:20.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.094+0000 7f1cfcbff700 1 -- 172.21.15.57:0/3425392594 wait complete. 2023-10-06T22:14:20.114 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-06T22:14:20.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:19 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2513043702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:20.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:19 smithi156 ceph-mon[134358]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-06T22:14:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 -- 172.21.15.57:0/2604756798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc0738b0 msgr2=0x7f29cc073d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 --2- 172.21.15.57:0/2604756798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc0738b0 0x7f29cc073d30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009990 tx=0x7f29bc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 -- 172.21.15.57:0/2604756798 shutdown_connections 2023-10-06T22:14:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 --2- 172.21.15.57:0/2604756798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29cc108dc0 0x7f29cc10b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 --2- 172.21.15.57:0/2604756798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29cc074e90 0x7f29cc0732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 --2- 172.21.15.57:0/2604756798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc0738b0 0x7f29cc073d30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 -- 172.21.15.57:0/2604756798 >> 172.21.15.57:0/2604756798 conn(0x7f29cc0fc460 msgr2=0x7f29cc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 -- 172.21.15.57:0/2604756798 shutdown_connections 2023-10-06T22:14:20.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.315+0000 7f29d45ff700 1 -- 172.21.15.57:0/2604756798 wait complete. 2023-10-06T22:14:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.316+0000 7f29d45ff700 1 Processor -- start 2023-10-06T22:14:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.316+0000 7f29d45ff700 1 -- start start 2023-10-06T22:14:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d45ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29cc0738b0 0x7f29cc198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d45ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc074e90 0x7f29cc199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d45ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29cc108dc0 0x7f29cc1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d45ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc1a0960 con 0x7f29cc074e90 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d45ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc1a0aa0 con 0x7f29cc108dc0 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d45ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc1a0c30 con 0x7f29cc0738b0 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc074e90 0x7f29cc199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc074e90 0x7f29cc199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42626/0 (socket says 172.21.15.57:42626) 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3589856176 learned_addr learned my addr 172.21.15.57:0/3589856176 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d239b700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29cc0738b0 0x7f29cc198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.317+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29cc108dc0 0x7f29cc1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3589856176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29cc0738b0 msgr2=0x7f29cc198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29cc0738b0 0x7f29cc198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3589856176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29cc108dc0 msgr2=0x7f29cc1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29cc108dc0 0x7f29cc1a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3589856176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0095f0 con 0x7f29cc074e90 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d239b700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29cc0738b0 0x7f29cc198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc074e90 0x7f29cc199320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00b680 tx=0x7f29bc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29c37fe700 1 -- 172.21.15.57:0/3589856176 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc01d070 con 0x7f29cc074e90 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29c37fe700 1 -- 172.21.15.57:0/3589856176 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29bc022470 con 0x7f29cc074e90 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc1a0eb0 con 0x7f29cc074e90 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29c37fe700 1 -- 172.21.15.57:0/3589856176 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc017650 con 0x7f29cc074e90 2023-10-06T22:14:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.318+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc1a12d0 con 0x7f29cc074e90 2023-10-06T22:14:20.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.322+0000 7f29c37fe700 1 -- 172.21.15.57:0/3589856176 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f29bc0225e0 con 0x7f29cc074e90 2023-10-06T22:14:20.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 Processor -- start 2023-10-06T22:14:20.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 -- start start 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1210 0x55b657dd1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1b70 0x55b657dd2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657dd2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29c37fe700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f29b80a1500 0x7f29b80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657d39410 con 0x55b657dd1210 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657d2abd0 con 0x55b657dd1b70 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657d284f0 con 0x55b657dd2a10 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.323+0000 7f29c37fe700 1 -- 172.21.15.57:0/3589856176 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f29bc0ce120 con 0x7f29cc074e90 2023-10-06T22:14:20.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1210 0x55b657dd1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1b70 0x55b657dd2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1210 0x55b657dd1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42634/0 (socket says 172.21.15.57:42634) 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d239b700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f29b80a1500 0x7f29b80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 -- 172.21.15.57:0/1980099483 learned_addr learned my addr 172.21.15.57:0/1980099483 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657dd2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 -- 172.21.15.57:0/1980099483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 msgr2=0x55b657dd2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657dd2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 -- 172.21.15.57:0/1980099483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1b70 msgr2=0x55b657dd2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1b70 0x55b657dd2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 -- 172.21.15.57:0/1980099483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b657b4ed90 con 0x55b657dd1210 2023-10-06T22:14:20.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1b70 0x55b657dd2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:20.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1210 0x55b657dd1630 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009fd0 tx=0x7f29c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f71aa63010cadfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.324+0000 7f29d239b700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f29b80a1500 0x7f29b80a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29c8011490 tx=0x7f29c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29c0bfe700 1 -- 172.21.15.57:0/1980099483 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c401f070 con 0x55b657dd1210 2023-10-06T22:14:20.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29c0bfe700 1 -- 172.21.15.57:0/1980099483 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c400f040 con 0x55b657dd1210 2023-10-06T22:14:20.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29c0bfe700 1 -- 172.21.15.57:0/1980099483 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400e040 con 0x55b657dd1210 2023-10-06T22:14:20.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1980099483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1210 msgr2=0x55b657dd1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1210 0x55b657dd1630 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009fd0 tx=0x7f29c4009920 comp rx=0 tx=0).stop 2023-10-06T22:14:20.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1980099483 shutdown_connections 2023-10-06T22:14:20.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657dd2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1210 0x55b657dd1630 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1980099483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1b70 0x55b657dd2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1980099483 >> 172.21.15.57:0/1980099483 conn(0x55b657d45210 msgr2=0x55b657dcc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1980099483 shutdown_connections 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.325+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1980099483 wait complete. 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.326+0000 7f29e7e8ab80 1 -- start start 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.326+0000 7f29e7e8ab80 1 Processor -- start 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657ddb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1b70 0x55b657ddb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657de2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657d23de0 con 0x55b657dd1b70 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657cc06c0 con 0x55b657dd1210 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657cc0360 con 0x55b657dd2a10 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29e7e8ab80 10 client.0 _ll_register_callbacks cb 0x55b657b1d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1b70 0x55b657ddb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1b70 0x55b657ddb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42642/0 (socket says 172.21.15.57:42642) 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3213395507 learned_addr learned my addr 172.21.15.57:0/3213395507 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d239b700 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657ddb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657de2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 msgr2=0x55b657de2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657de2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3213395507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 msgr2=0x55b657ddb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657ddb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 -- 172.21.15.57:0/3213395507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc1a12d0 con 0x55b657dd1b70 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.327+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1b70 0x55b657ddb540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f29bc022890 tx=0x7f29bc022bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9a7213ed08fa9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29b5ffb700 1 -- 172.21.15.57:0/3213395507 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc09f370 con 0x55b657dd1b70 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b657dc3ec0 con 0x55b657dd1b70 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29b5ffb700 1 -- 172.21.15.57:0/3213395507 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f29bc09b4a0 con 0x55b657dd1b70 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29b5ffb700 1 -- 172.21.15.57:0/3213395507 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29bc09f5f0 con 0x55b657dd1b70 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29e7e8ab80 10 client.15213 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f29c4013ba0 con 0x55b657dd1b70 2023-10-06T22:14:20.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29b5ffb700 1 -- 172.21.15.57:0/3213395507 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b657dc3ec0 con 0x55b657dd1b70 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.328+0000 7f29b5ffb700 1 -- 172.21.15.57:0/3213395507 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29bc0a3310 con 0x55b657dd1b70 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29e7e8ab80 10 client.15213 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29e7e8ab80 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29e7e8ab80 20 client.15213 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657de6770 0x55b657de54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29e7e8ab80 4 client.15213 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b657d1cdc0 con 0x55b657de6770 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29d239b700 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657de6770 0x55b657de54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29d239b700 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657de6770 0x55b657de54b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af259585dc638b95 server_cookie=52930b1c9d322709 in_seq=0 out_seq=0 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.329+0000 7f29b5ffb700 10 client.15213 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:20.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29b5ffb700 1 -- 172.21.15.57:0/3213395507 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b657d1cdc0 con 0x55b657de6770 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29b5ffb700 10 client.15213 handle_command_reply: tid=0 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 client.15213 shutdown 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1b70 msgr2=0x55b657ddb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1b70 0x55b657ddb540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f29bc022890 tx=0x7f29bc022bc0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 shutdown_connections 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657de6770 0x55b657de54b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd2a10 0x55b657de2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657ddb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29b5ffb700 0 client.15213 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.330+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/3213395507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd1b70 0x55b657ddb540 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.331+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 >> 172.21.15.57:0/3213395507 conn(0x55b657d45210 msgr2=0x55b657dcbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.331+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 shutdown_connections 2023-10-06T22:14:20.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.331+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/3213395507 wait complete. 2023-10-06T22:14:20.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.331+0000 7f29e7e8ab80 20 client.15213 trim_cache size 0 max 16384 2023-10-06T22:14:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.349+0000 7f29e7e8ab80 1 Processor -- start 2023-10-06T22:14:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.349+0000 7f29e7e8ab80 1 -- start start 2023-10-06T22:14:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657de59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657de2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657ba48d0 con 0x55b657dd2a10 2023-10-06T22:14:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657dcb2b0 con 0x55b657dd1210 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657dc62b0 con 0x55b657dd1b70 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d2b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657de2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59594/0 (socket says 172.21.15.57:59594) 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657de59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d239b700 1 -- 172.21.15.57:0/777313014 learned_addr learned my addr 172.21.15.57:0/777313014 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d2b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657de2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42652/0 (socket says 172.21.15.57:42652) 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d2b9c700 1 -- 172.21.15.57:0/777313014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 msgr2=0x55b657de59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657de59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d2b9c700 1 -- 172.21.15.57:0/777313014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 msgr2=0x55b657de54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.350+0000 7f29d2b9c700 1 -- 172.21.15.57:0/777313014 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b657b37c10 con 0x55b657dd2a10 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.351+0000 7f29d239b700 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.351+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657de59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.351+0000 7f29d2b9c700 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657de2540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55b657dc6670 tx=0x7f29c4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18228929a62c7e8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.351+0000 7f29c13ff700 1 -- 172.21.15.57:0/777313014 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400e070 con 0x55b657dd2a10 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.351+0000 7f29c13ff700 1 -- 172.21.15.57:0/777313014 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c40135d0 con 0x55b657dd2a10 2023-10-06T22:14:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.351+0000 7f29c13ff700 1 -- 172.21.15.57:0/777313014 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c401f040 con 0x55b657dd2a10 2023-10-06T22:14:20.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/777313014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 msgr2=0x55b657de2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657de2540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55b657dc6670 tx=0x7f29c4009ba0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/777313014 shutdown_connections 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657de2540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/777313014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657de59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/777313014 >> 172.21.15.57:0/777313014 conn(0x55b657d45210 msgr2=0x55b657dd9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/777313014 shutdown_connections 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/777313014 wait complete. 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.352+0000 7f29e7e8ab80 1 -- start start 2023-10-06T22:14:20.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 1 Processor -- start 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657ddeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657ddf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657ddfb50 con 0x55b657dd2a10 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657e429a0 con 0x55b657dd1210 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b657e42b60 con 0x55b657dd1b70 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.353+0000 7f29e7e8ab80 10 client.0 _ll_register_callbacks cb 0x7f29bc097890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d2b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657ddf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657ddeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657ddeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45662/0 (socket says 172.21.15.57:45662) 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 -- 172.21.15.57:0/1742442178 learned_addr learned my addr 172.21.15.57:0/1742442178 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 -- 172.21.15.57:0/1742442178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 msgr2=0x55b657de2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 -- 172.21.15.57:0/1742442178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 msgr2=0x55b657ddf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657ddf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 -- 172.21.15.57:0/1742442178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c8017070 con 0x55b657dd1b70 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29d1b9a700 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657ddeee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0a3710 tx=0x7f29bc0cbc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74c5461ab41a59a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29b5ffb700 1 -- 172.21.15.57:0/1742442178 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc09f370 con 0x55b657dd1b70 2023-10-06T22:14:20.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29c400f040 con 0x55b657dd1b70 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.354+0000 7f29b5ffb700 1 -- 172.21.15.57:0/1742442178 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f29bc09b4a0 con 0x55b657dd1b70 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29b5ffb700 1 -- 172.21.15.57:0/1742442178 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29bc09f5f0 con 0x55b657dd1b70 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 10 client.25246 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f29c4013ba0 con 0x55b657dd1b70 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29b5ffb700 1 -- 172.21.15.57:0/1742442178 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c400f040 con 0x55b657dd1b70 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29b5ffb700 1 -- 172.21.15.57:0/1742442178 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29bc097320 con 0x55b657dd1b70 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 10 client.25246 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 10 client.25246 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 20 client.25246 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657e47aa0 0x55b657e43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 4 client.25246 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T22:14:20.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.355+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b657d1cae0 con 0x55b657e47aa0 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29d239b700 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657e47aa0 0x55b657e43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29d239b700 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657e47aa0 0x55b657e43ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9622086cd0289c0d server_cookie=2b4f30ec0aed97f1 in_seq=0 out_seq=0 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29b5ffb700 10 client.25246 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29b5ffb700 1 -- 172.21.15.57:0/1742442178 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b657d1cae0 con 0x55b657e47aa0 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29b5ffb700 10 client.25246 handle_command_reply: tid=0 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29e7e8ab80 1 client.25246 shutdown 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 msgr2=0x55b657ddeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.356+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657ddeee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0a3710 tx=0x7f29bc0cbc70 comp rx=0 tx=0).stop 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 shutdown_connections 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b657e47aa0 0x55b657e43ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b657dd2a10 0x55b657ddf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b657dd1210 0x55b657de2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 --2- 172.21.15.57:0/1742442178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b657dd1b70 0x55b657ddeee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29b5ffb700 0 client.25246 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 >> 172.21.15.57:0/1742442178 conn(0x55b657d45210 msgr2=0x55b657de54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 shutdown_connections 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 1 -- 172.21.15.57:0/1742442178 wait complete. 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29e7e8ab80 20 client.25246 trim_cache size 0 max 16384 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:14:20.365 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "495b2048-6494-11ee-8db5-212e2dc638e7", 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24553, 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 96.643121210000004, 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 26, 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 85, 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 110.712712991 2023-10-06T22:14:20.366 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:14:20.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f29b80a1500 msgr2=0x7f29b80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29d45ff700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f29b80a1500 0x7f29b80a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29c8011490 tx=0x7f29c8009260 comp rx=0 tx=0).stop 2023-10-06T22:14:20.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc074e90 msgr2=0x7f29cc199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.357+0000 7f29d45ff700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc074e90 0x7f29cc199320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00b680 tx=0x7f29bc005740 comp rx=0 tx=0).stop 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 shutdown_connections 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f29b80a1500 0x7f29b80a39c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29cc108dc0 0x7f29cc1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29cc0738b0 0x7f29cc198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 --2- 172.21.15.57:0/3589856176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29cc074e90 0x7f29cc199320 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 >> 172.21.15.57:0/3589856176 conn(0x7f29cc0fc460 msgr2=0x7f29cc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 shutdown_connections 2023-10-06T22:14:20.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:20.358+0000 7f29d45ff700 1 -- 172.21.15.57:0/3589856176 wait complete. 2023-10-06T22:14:20.378 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T22:14:20.403 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T22:14:20.403 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-06T22:14:20.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:20.404 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-06T22:14:20.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:20.449 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-06T22:14:20.562 INFO:teuthology.orchestra.run.smithi057.stderr:Cannot find device "ceph-brx" 2023-10-06T22:14:20.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:14:20.565 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-10-06T22:14:20.565 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-06T22:14:20.565 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:20.566 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-06T22:14:21.302 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:14:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 -- 172.21.15.57:0/1761878570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc106f90 msgr2=0x7f38fc107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 --2- 172.21.15.57:0/1761878570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc106f90 0x7f38fc107370 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38f80099c0 tx=0x7f38f8009cd0 comp rx=0 tx=0).stop 2023-10-06T22:14:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 -- 172.21.15.57:0/1761878570 shutdown_connections 2023-10-06T22:14:22.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 --2- 172.21.15.57:0/1761878570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc101940 0x7f38fc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 --2- 172.21.15.57:0/1761878570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc106f90 0x7f38fc107370 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 --2- 172.21.15.57:0/1761878570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38fc100f80 0x7f38fc101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 -- 172.21.15.57:0/1761878570 >> 172.21.15.57:0/1761878570 conn(0x7f38fc0780a0 msgr2=0x7f38fc0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.816+0000 7f3903767700 1 -- 172.21.15.57:0/1761878570 shutdown_connections 2023-10-06T22:14:22.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.817+0000 7f3903767700 1 -- 172.21.15.57:0/1761878570 wait complete. 2023-10-06T22:14:22.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 Processor -- start 2023-10-06T22:14:22.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 -- start start 2023-10-06T22:14:22.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc100f80 0x7f38fc0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:22.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc101940 0x7f38fc0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38fc106f90 0x7f38fc073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc1a53f0 con 0x7f38fc101940 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc1a5560 con 0x7f38fc100f80 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.818+0000 7f3903767700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc1a5720 con 0x7f38fc106f90 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc100f80 0x7f38fc0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc100f80 0x7f38fc0731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:42946/0 (socket says 172.21.15.57:42946) 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 -- 172.21.15.57:0/2141730939 learned_addr learned my addr 172.21.15.57:0/2141730939 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3900d02700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc101940 0x7f38fc0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901d04700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38fc106f90 0x7f38fc073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 -- 172.21.15.57:0/2141730939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38fc106f90 msgr2=0x7f38fc073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38fc106f90 0x7f38fc073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:22.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 -- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc101940 msgr2=0x7f38fc0736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:22.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc101940 0x7f38fc0736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:22.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 -- 172.21.15.57:0/2141730939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80095f0 con 0x7f38fc100f80 2023-10-06T22:14:22.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.819+0000 7f3901503700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc100f80 0x7f38fc0731b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38f800b580 tx=0x7f38f800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:22.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.820+0000 7f38f27fc700 1 -- 172.21.15.57:0/2141730939 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f801d070 con 0x7f38fc100f80 2023-10-06T22:14:22.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.820+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc1a59a0 con 0x7f38fc100f80 2023-10-06T22:14:22.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.820+0000 7f38f27fc700 1 -- 172.21.15.57:0/2141730939 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38f80054f0 con 0x7f38fc100f80 2023-10-06T22:14:22.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.820+0000 7f38f27fc700 1 -- 172.21.15.57:0/2141730939 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8004b30 con 0x7f38fc100f80 2023-10-06T22:14:22.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.820+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc1a5e10 con 0x7f38fc100f80 2023-10-06T22:14:22.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.821+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc04ebb0 con 0x7f38fc100f80 2023-10-06T22:14:22.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.823+0000 7f38f27fc700 1 -- 172.21.15.57:0/2141730939 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f38f800fb80 con 0x7f38fc100f80 2023-10-06T22:14:22.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.824+0000 7f38f27fc700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f38e80a1500 0x7f38e80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:22.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.825+0000 7f38f27fc700 1 -- 172.21.15.57:0/2141730939 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f38f80cda80 con 0x7f38fc100f80 2023-10-06T22:14:22.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.825+0000 7f3900d02700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f38e80a1500 0x7f38e80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:22.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.825+0000 7f3900d02700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f38e80a1500 0x7f38e80a39c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38fc074950 tx=0x7f38ec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:22.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:22.827+0000 7f38f27fc700 1 -- 172.21.15.57:0/2141730939 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f38f8097140 con 0x7f38fc100f80 2023-10-06T22:14:22.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:22 smithi057 ceph-mon[98492]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-06T22:14:23.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:22 smithi156 ceph-mon[134358]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-06T22:14:23.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:22 smithi190 ceph-mon[134891]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-06T22:14:23.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.080+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f38fc074690 con 0x7f38fc100f80 2023-10-06T22:14:23.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.081+0000 7f38f27fc700 1 -- 172.21.15.57:0/2141730939 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f38f809a760 con 0x7f38fc100f80 2023-10-06T22:14:23.084 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T22:14:23.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.083+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f38e80a1500 msgr2=0x7f38e80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:23.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.083+0000 7f3903767700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f38e80a1500 0x7f38e80a39c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38fc074950 tx=0x7f38ec00b410 comp rx=0 tx=0).stop 2023-10-06T22:14:23.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc100f80 msgr2=0x7f38fc0731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:23.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc100f80 0x7f38fc0731b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38f800b580 tx=0x7f38f800f690 comp rx=0 tx=0).stop 2023-10-06T22:14:23.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 shutdown_connections 2023-10-06T22:14:23.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f38e80a1500 0x7f38e80a39c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:23.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38fc106f90 0x7f38fc073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:23.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38fc100f80 0x7f38fc0731b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:23.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 --2- 172.21.15.57:0/2141730939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38fc101940 0x7f38fc0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:23.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 >> 172.21.15.57:0/2141730939 conn(0x7f38fc0780a0 msgr2=0x7f38fc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:23.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 shutdown_connections 2023-10-06T22:14:23.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:23.084+0000 7f3903767700 1 -- 172.21.15.57:0/2141730939 wait complete. 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T22:14:23.580 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T22:14:23.581 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T22:14:23.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:23.581 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-06T22:14:23.625 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T22:14:23.625 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2023-10-06T22:14:23.626 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41584sec preferred_lft 41584sec 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-06T22:14:23.627 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2023-10-06T22:14:23.628 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T22:14:23.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:23.628 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-06T22:14:23.629 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-06T22:14:23.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2023-10-06T22:14:23.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2023-10-06T22:14:23.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2023-10-06T22:14:23.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T22:14:23.630 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-06T22:14:23.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:23 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2141730939' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T22:14:24.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:23 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2141730939' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T22:14:24.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:23 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2141730939' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T22:14:24.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:24.123 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T22:14:24.193 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-06T22:14:24.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:24.197 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-06T22:14:24.271 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-06T22:14:24.271 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T22:14:24.271 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T22:14:24.271 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T22:14:24.272 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T22:14:24.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:24.273 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-06T22:14:24.273 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-06T22:14:24.273 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T22:14:24.273 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T22:14:24.273 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T22:14:24.273 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-06T22:14:24.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:24 smithi057 ceph-mon[98492]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-06T22:14:24.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:24.979 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-06T22:14:25.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:24 smithi156 ceph-mon[134358]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-06T22:14:25.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:25.024 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2023-10-06T22:14:25.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:24 smithi190 ceph-mon[134891]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-06T22:14:25.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:25.108 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-06T22:14:25.108 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-06T22:14:25.108 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T22:14:25.109 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T22:14:25.109 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-06T22:14:25.295 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T22:14:25.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:25.295 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-06T22:14:25.295 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-06T22:14:25.295 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T22:14:25.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T22:14:25.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T22:14:25.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T22:14:25.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T22:14:25.296 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-06T22:14:25.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:25.582 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-06T22:14:25.582 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-06T22:14:25.582 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-10-06T22:14:25.582 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T22:14:25.582 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-06T22:14:25.760 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T22:14:25.760 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:25.804 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T22:14:25.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T22:14:25.805 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:25.889 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-06T22:14:25.889 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-06T22:14:25.889 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc 2023-10-06T22:14:25.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:25.889 DEBUG:teuthology.orchestra.run.smithi057:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2023-10-06T22:14:26.049 INFO:teuthology.orchestra.run.smithi057.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2023-10-06T22:14:26.049 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2023-10-06T22:14:26.049 INFO:teuthology.orchestra.run.smithi057.stdout:invalid new device string format 2023-10-06T22:14:26.519 INFO:teuthology.orchestra.run.smithi057.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2023-10-06T22:14:26.519 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2023-10-06T22:14:26.519 INFO:teuthology.orchestra.run.smithi057.stdout:invalid new device string format 2023-10-06T22:14:26.519 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: resolved to: "172.21.15.57:6789,172.21.15.156:6789,172.21.15.190:6789" 2023-10-06T22:14:26.519 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.57:6789,172.21.15.156:6789,172.21.15.190:6789:/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc 2023-10-06T22:14:26.519 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=495b2048-6494-11ee-8db5-212e2dc638e7" will pass to kernel 2023-10-06T22:14:26.525 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T22:14:26.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:26.525 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:26.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:26.602 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T22:14:26.694 INFO:teuthology.orchestra.run.smithi057.stdout:client15228 2023-10-06T22:14:26.695 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:14:26.695 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T22:14:26.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:26.740 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T22:14:26.868 INFO:teuthology.orchestra.run.smithi057.stdout:client15228 2023-10-06T22:14:26.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:26.869 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T22:14:26.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:26 smithi057 ceph-mon[98492]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:14:26.957 INFO:teuthology.orchestra.run.smithi057.stdout:client15228 2023-10-06T22:14:26.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:14:26.958 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/495b2048-6494-11ee-8db5-212e2dc638e7.client15228/status') 2023-10-06T22:14:27.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:26 smithi156 ceph-mon[134358]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:14:27.067 INFO:teuthology.orchestra.run.smithi057.stdout:instance: client.15228 (3)192.168.0.1:0/335089958 2023-10-06T22:14:27.067 INFO:teuthology.orchestra.run.smithi057.stdout:blocklisted: false 2023-10-06T22:14:27.067 INFO:teuthology.orchestra.run.smithi057.stderr:0+1 records in 2023-10-06T22:14:27.067 INFO:teuthology.orchestra.run.smithi057.stderr:0+1 records out 2023-10-06T22:14:27.067 INFO:teuthology.orchestra.run.smithi057.stderr:69 bytes copied, 5.5951e-05 s, 1.2 MB/s 2023-10-06T22:14:27.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:26 smithi190 ceph-mon[134891]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-06T22:14:27.070 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-06T22:14:27.083 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-06T22:14:27.137 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:14:27.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.338+0000 7f19a83ff700 1 -- 172.21.15.57:0/1082745661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102a40 msgr2=0x7f19a0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.338+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1082745661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102a40 0x7f19a0109360 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1998009f90 tx=0x7f199800b330 comp rx=0 tx=0).stop 2023-10-06T22:14:27.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.338+0000 7f19a83ff700 1 -- 172.21.15.57:0/1082745661 shutdown_connections 2023-10-06T22:14:27.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.338+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1082745661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102a40 0x7f19a0109360 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.338+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1082745661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a0100ea0 0x7f19a01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.339+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1082745661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0102080 0x7f19a0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.339+0000 7f19a83ff700 1 -- 172.21.15.57:0/1082745661 >> 172.21.15.57:0/1082745661 conn(0x7f19a00fc420 msgr2=0x7f19a00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.339+0000 7f19a83ff700 1 -- 172.21.15.57:0/1082745661 shutdown_connections 2023-10-06T22:14:27.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.339+0000 7f19a83ff700 1 -- 172.21.15.57:0/1082745661 wait complete. 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 Processor -- start 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 -- start start 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0100ea0 0x7f19a0196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102080 0x7f19a0197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a0102a40 0x7f19a019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a019e870 con 0x7f19a0102080 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a019e9e0 con 0x7f19a0102a40 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a83ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a019eba0 con 0x7f19a0100ea0 2023-10-06T22:14:27.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a599a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102080 0x7f19a0197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a599a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102080 0x7f19a0197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38180/0 (socket says 172.21.15.57:38180) 2023-10-06T22:14:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a619b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0100ea0 0x7f19a0196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.340+0000 7f19a599a700 1 -- 172.21.15.57:0/1601898178 learned_addr learned my addr 172.21.15.57:0/1601898178 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a699c700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a0102a40 0x7f19a019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a599a700 1 -- 172.21.15.57:0/1601898178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0100ea0 msgr2=0x7f19a0196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a599a700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0100ea0 0x7f19a0196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a599a700 1 -- 172.21.15.57:0/1601898178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a0102a40 msgr2=0x7f19a019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a599a700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a0102a40 0x7f19a019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a599a700 1 -- 172.21.15.57:0/1601898178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0095f0 con 0x7f19a0102080 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a599a700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102080 0x7f19a0197270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f199000e950 tx=0x7f199000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19977fe700 1 -- 172.21.15.57:0/1601898178 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990009770 con 0x7f19a0102080 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1998009c40 con 0x7f19a0102080 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19977fe700 1 -- 172.21.15.57:0/1601898178 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f199000c7e0 con 0x7f19a0102080 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.341+0000 7f19977fe700 1 -- 172.21.15.57:0/1601898178 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990003cf0 con 0x7f19a0102080 2023-10-06T22:14:27.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.342+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a019f1b0 con 0x7f19a0102080 2023-10-06T22:14:27.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.343+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a0108440 con 0x7f19a0102080 2023-10-06T22:14:27.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.344+0000 7f19977fe700 1 -- 172.21.15.57:0/1601898178 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f199000c950 con 0x7f19a0102080 2023-10-06T22:14:27.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.347+0000 7f19977fe700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f198c0a1550 0x7f198c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.347+0000 7f19977fe700 1 -- 172.21.15.57:0/1601898178 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f1990014070 con 0x7f19a0102080 2023-10-06T22:14:27.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.347+0000 7f19a619b700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f198c0a1550 0x7f198c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.348+0000 7f19a619b700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f198c0a1550 0x7f198c0a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f19a0198200 tx=0x7f199c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:27.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.351+0000 7f19977fe700 1 -- 172.21.15.57:0/1601898178 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f19900964e0 con 0x7f19a0102080 2023-10-06T22:14:27.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.563+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19a010c790 con 0x7f19a0102080 2023-10-06T22:14:27.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.565+0000 7f19977fe700 1 -- 172.21.15.57:0/1601898178 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7f1990099b00 con 0x7f19a0102080 2023-10-06T22:14:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:27.568 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:14:27.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.567+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f198c0a1550 msgr2=0x7f198c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.567+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f198c0a1550 0x7f198c0a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f19a0198200 tx=0x7f199c009310 comp rx=0 tx=0).stop 2023-10-06T22:14:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.567+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102080 msgr2=0x7f19a0197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.567+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102080 0x7f19a0197270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f199000e950 tx=0x7f199000ec60 comp rx=0 tx=0).stop 2023-10-06T22:14:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 shutdown_connections 2023-10-06T22:14:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f198c0a1550 0x7f198c0a3a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a0102a40 0x7f19a019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19a0100ea0 0x7f19a0196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 --2- 172.21.15.57:0/1601898178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19a0102080 0x7f19a0197270 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 >> 172.21.15.57:0/1601898178 conn(0x7f19a00fc420 msgr2=0x7f19a0105580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:27.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 shutdown_connections 2023-10-06T22:14:27.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.568+0000 7f19a83ff700 1 -- 172.21.15.57:0/1601898178 wait complete. 2023-10-06T22:14:27.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:14:27.590 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:14:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 -- 172.21.15.57:0/3463213827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148100e90 msgr2=0x7fa1481012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 --2- 172.21.15.57:0/3463213827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148100e90 0x7fa1481012b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa138009990 tx=0x7fa138009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 -- 172.21.15.57:0/3463213827 shutdown_connections 2023-10-06T22:14:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 --2- 172.21.15.57:0/3463213827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148102a30 0x7fa148109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 --2- 172.21.15.57:0/3463213827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148100e90 0x7fa1481012b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 --2- 172.21.15.57:0/3463213827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa148102070 0x7fa1481024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 -- 172.21.15.57:0/3463213827 >> 172.21.15.57:0/3463213827 conn(0x7fa1480fc430 msgr2=0x7fa1480fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 -- 172.21.15.57:0/3463213827 shutdown_connections 2023-10-06T22:14:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.802+0000 7fa14dfff700 1 -- 172.21.15.57:0/3463213827 wait complete. 2023-10-06T22:14:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.803+0000 7fa14dfff700 1 Processor -- start 2023-10-06T22:14:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa14dfff700 1 -- start start 2023-10-06T22:14:27.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa14dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa148100e90 0x7fa148198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa14dfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148102070 0x7fa148199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa14dfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148102a30 0x7fa1481a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa14dfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1481a09b0 con 0x7fa148102070 2023-10-06T22:14:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa14dfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1481a0af0 con 0x7fa148102a30 2023-10-06T22:14:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa14dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1481a0c80 con 0x7fa148100e90 2023-10-06T22:14:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa146ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148102070 0x7fa148199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa1477fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa148100e90 0x7fa148198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa146ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148102070 0x7fa148199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38206/0 (socket says 172.21.15.57:38206) 2023-10-06T22:14:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa146ffd700 1 -- 172.21.15.57:0/625759152 learned_addr learned my addr 172.21.15.57:0/625759152 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:27.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.804+0000 7fa147fff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148102a30 0x7fa1481a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa146ffd700 1 -- 172.21.15.57:0/625759152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa148100e90 msgr2=0x7fa148198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa146ffd700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa148100e90 0x7fa148198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa146ffd700 1 -- 172.21.15.57:0/625759152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148102a30 msgr2=0x7fa1481a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa146ffd700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148102a30 0x7fa1481a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa146ffd700 1 -- 172.21.15.57:0/625759152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c009520 con 0x7fa148102070 2023-10-06T22:14:27.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa147fff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148102a30 0x7fa1481a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:27.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa1477fe700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa148100e90 0x7fa148198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:27.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa146ffd700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148102070 0x7fa148199370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00ea50 tx=0x7fa13c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:27.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa144ff9700 1 -- 172.21.15.57:0/625759152 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00cbd0 con 0x7fa148102070 2023-10-06T22:14:27.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa140009c40 con 0x7fa148102070 2023-10-06T22:14:27.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa144ff9700 1 -- 172.21.15.57:0/625759152 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa13c00cd30 con 0x7fa148102070 2023-10-06T22:14:27.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa144ff9700 1 -- 172.21.15.57:0/625759152 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c018680 con 0x7fa148102070 2023-10-06T22:14:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.805+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1380095f0 con 0x7fa148102070 2023-10-06T22:14:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.807+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14810c700 con 0x7fa148102070 2023-10-06T22:14:27.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.808+0000 7fa144ff9700 1 -- 172.21.15.57:0/625759152 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa13c0188e0 con 0x7fa148102070 2023-10-06T22:14:27.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.810+0000 7fa144ff9700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa1300a1480 0x7fa1300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:27.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.810+0000 7fa144ff9700 1 -- 172.21.15.57:0/625759152 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fa13c014070 con 0x7fa148102070 2023-10-06T22:14:27.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.810+0000 7fa1477fe700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa1300a1480 0x7fa1300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:27.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.811+0000 7fa1477fe700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa1300a1480 0x7fa1300a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b680 tx=0x7fa138005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:27.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:27.814+0000 7fa144ff9700 1 -- 172.21.15.57:0/625759152 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa13c0968b0 con 0x7fa148102070 2023-10-06T22:14:27.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:27 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1601898178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:28.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:27 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1601898178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:28.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.030+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1481a12e0 con 0x7fa148102070 2023-10-06T22:14:28.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.031+0000 7fa144ff9700 1 -- 172.21.15.57:0/625759152 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa13c099ed0 con 0x7fa148102070 2023-10-06T22:14:28.033 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa1300a1480 msgr2=0x7fa1300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa1300a1480 0x7fa1300a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b680 tx=0x7fa138005fd0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148102070 msgr2=0x7fa148199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148102070 0x7fa148199370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00ea50 tx=0x7fa13c00ed60 comp rx=0 tx=0).stop 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 shutdown_connections 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fa1300a1480 0x7fa1300a3940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148102a30 0x7fa1481a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa148100e90 0x7fa148198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 --2- 172.21.15.57:0/625759152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa148102070 0x7fa148199370 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.033+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 >> 172.21.15.57:0/625759152 conn(0x7fa1480fc430 msgr2=0x7fa1480fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:28.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.034+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 shutdown_connections 2023-10-06T22:14:28.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.034+0000 7fa14dfff700 1 -- 172.21.15.57:0/625759152 wait complete. 2023-10-06T22:14:28.054 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:14:28.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:27 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1601898178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.258+0000 7f22c2fff700 1 -- 172.21.15.57:0/90221630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc100f70 msgr2=0x7f22bc1013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.258+0000 7f22c2fff700 1 --2- 172.21.15.57:0/90221630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc100f70 0x7f22bc1013f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22a80098f0 tx=0x7f22a8009c00 comp rx=0 tx=0).stop 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.259+0000 7f22c2fff700 1 -- 172.21.15.57:0/90221630 shutdown_connections 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.259+0000 7f22c2fff700 1 --2- 172.21.15.57:0/90221630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc101930 0x7f22bc108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.259+0000 7f22c2fff700 1 --2- 172.21.15.57:0/90221630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22bc0ffe20 0x7f22bc100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.259+0000 7f22c2fff700 1 --2- 172.21.15.57:0/90221630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc100f70 0x7f22bc1013f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.259+0000 7f22c2fff700 1 -- 172.21.15.57:0/90221630 >> 172.21.15.57:0/90221630 conn(0x7f22bc0fb370 msgr2=0x7f22bc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.259+0000 7f22c2fff700 1 -- 172.21.15.57:0/90221630 shutdown_connections 2023-10-06T22:14:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.259+0000 7f22c2fff700 1 -- 172.21.15.57:0/90221630 wait complete. 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.260+0000 7f22c2fff700 1 Processor -- start 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c2fff700 1 -- start start 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c2fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22bc0ffe20 0x7f22bc198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c2fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc100f70 0x7f22bc1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c2fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc101930 0x7f22bc1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c2fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc1a09f0 con 0x7f22bc100f70 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c2fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc1a0b30 con 0x7f22bc101930 2023-10-06T22:14:28.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c2fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc1a0cc0 con 0x7f22bc0ffe20 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc100f70 0x7f22bc1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc100f70 0x7f22bc1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38222/0 (socket says 172.21.15.57:38222) 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22bbfff700 1 -- 172.21.15.57:0/582997020 learned_addr learned my addr 172.21.15.57:0/582997020 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.261+0000 7f22c0d9b700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22bc0ffe20 0x7f22bc198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22c159c700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc101930 0x7f22bc1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22bbfff700 1 -- 172.21.15.57:0/582997020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22bc0ffe20 msgr2=0x7f22bc198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22bbfff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22bc0ffe20 0x7f22bc198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22bbfff700 1 -- 172.21.15.57:0/582997020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc101930 msgr2=0x7f22bc1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22bbfff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc101930 0x7f22bc1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22bbfff700 1 -- 172.21.15.57:0/582997020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b4009d00 con 0x7f22bc100f70 2023-10-06T22:14:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22bbfff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc100f70 0x7f22bc1993b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005e90 tx=0x7f22a800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22b9ffb700 1 -- 172.21.15.57:0/582997020 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a801c070 con 0x7f22bc100f70 2023-10-06T22:14:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a8009520 con 0x7f22bc100f70 2023-10-06T22:14:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22b9ffb700 1 -- 172.21.15.57:0/582997020 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22a800f7f0 con 0x7f22bc100f70 2023-10-06T22:14:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22b9ffb700 1 -- 172.21.15.57:0/582997020 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8004b30 con 0x7f22bc100f70 2023-10-06T22:14:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.262+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc1a11d0 con 0x7f22bc100f70 2023-10-06T22:14:28.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.264+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc10b670 con 0x7f22bc100f70 2023-10-06T22:14:28.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.265+0000 7f22b9ffb700 1 -- 172.21.15.57:0/582997020 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f22a8020070 con 0x7f22bc100f70 2023-10-06T22:14:28.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.268+0000 7f22b9ffb700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22ac0a1500 0x7f22ac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.268+0000 7f22b9ffb700 1 -- 172.21.15.57:0/582997020 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f22a80cdad0 con 0x7f22bc100f70 2023-10-06T22:14:28.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.268+0000 7f22c0d9b700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22ac0a1500 0x7f22ac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.268+0000 7f22c0d9b700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22ac0a1500 0x7f22ac0a39c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009c60 tx=0x7f22b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:28.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.272+0000 7f22b9ffb700 1 -- 172.21.15.57:0/582997020 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f22a8097190 con 0x7f22bc100f70 2023-10-06T22:14:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.480+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22bc1a13c0 con 0x7f22bc100f70 2023-10-06T22:14:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.481+0000 7f22b9ffb700 1 -- 172.21.15.57:0/582997020 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22a809a7b0 con 0x7f22bc100f70 2023-10-06T22:14:28.483 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:14:28.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22ac0a1500 msgr2=0x7f22ac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22ac0a1500 0x7f22ac0a39c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009c60 tx=0x7f22b0009260 comp rx=0 tx=0).stop 2023-10-06T22:14:28.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc100f70 msgr2=0x7f22bc1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc100f70 0x7f22bc1993b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005e90 tx=0x7f22a800b730 comp rx=0 tx=0).stop 2023-10-06T22:14:28.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 shutdown_connections 2023-10-06T22:14:28.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f22ac0a1500 0x7f22ac0a39c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22bc101930 0x7f22bc1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22bc0ffe20 0x7f22bc198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 --2- 172.21.15.57:0/582997020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22bc100f70 0x7f22bc1993b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 >> 172.21.15.57:0/582997020 conn(0x7f22bc0fb370 msgr2=0x7f22bc069030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 shutdown_connections 2023-10-06T22:14:28.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.483+0000 7f22c2fff700 1 -- 172.21.15.57:0/582997020 wait complete. 2023-10-06T22:14:28.504 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:14:28.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 -- 172.21.15.57:0/2557677553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0102a30 msgr2=0x7f8df0109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2557677553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0102a30 0x7f8df0109350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8de800b0a0 tx=0x7f8de800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 -- 172.21.15.57:0/2557677553 shutdown_connections 2023-10-06T22:14:28.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2557677553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0102a30 0x7f8df0109350 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2557677553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8df0100e90 0x7f8df01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2557677553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df0102070 0x7f8df01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 -- 172.21.15.57:0/2557677553 >> 172.21.15.57:0/2557677553 conn(0x7f8df00fc430 msgr2=0x7f8df00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:28.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.707+0000 7f8df5dff700 1 -- 172.21.15.57:0/2557677553 shutdown_connections 2023-10-06T22:14:28.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.708+0000 7f8df5dff700 1 -- 172.21.15.57:0/2557677553 wait complete. 2023-10-06T22:14:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.709+0000 7f8df5dff700 1 Processor -- start 2023-10-06T22:14:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.709+0000 7f8df5dff700 1 -- start start 2023-10-06T22:14:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8df5dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0100e90 0x7f8df0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8df5dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df0102070 0x7f8df00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8df5dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8df0102a30 0x7f8df00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8df5dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0073f80 con 0x7f8df0102a30 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8df5dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00740f0 con 0x7f8df0102070 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8df5dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01a0d60 con 0x7f8df0100e90 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0100e90 0x7f8df0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8deeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df0102070 0x7f8df00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0100e90 0x7f8df0074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58814/0 (socket says 172.21.15.57:58814) 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 -- 172.21.15.57:0/2612052906 learned_addr learned my addr 172.21.15.57:0/2612052906 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:28.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8deffff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8df0102a30 0x7f8df00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 -- 172.21.15.57:0/2612052906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df0102070 msgr2=0x7f8df00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df0102070 0x7f8df00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 -- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8df0102a30 msgr2=0x7f8df00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8df0102a30 0x7f8df00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8def7fe700 1 -- 172.21.15.57:0/2612052906 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8009d00 con 0x7f8df0100e90 2023-10-06T22:14:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.710+0000 7f8deffff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8df0102a30 0x7f8df00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:28.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.711+0000 7f8deeffd700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df0102070 0x7f8df00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:28.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.711+0000 7f8def7fe700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0100e90 0x7f8df0074c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e970 tx=0x7f8de000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.715+0000 7f8decff9700 1 -- 172.21.15.57:0/2612052906 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0009790 con 0x7f8df0100e90 2023-10-06T22:14:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.715+0000 7f8decff9700 1 -- 172.21.15.57:0/2612052906 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de000c7e0 con 0x7f8df0100e90 2023-10-06T22:14:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.715+0000 7f8decff9700 1 -- 172.21.15.57:0/2612052906 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003c70 con 0x7f8df0100e90 2023-10-06T22:14:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.715+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df01a0fe0 con 0x7f8df0100e90 2023-10-06T22:14:28.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.715+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df01a1400 con 0x7f8df0100e90 2023-10-06T22:14:28.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.716+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df010c7c0 con 0x7f8df0100e90 2023-10-06T22:14:28.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.718+0000 7f8decff9700 1 -- 172.21.15.57:0/2612052906 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8de000cc50 con 0x7f8df0100e90 2023-10-06T22:14:28.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.719+0000 7f8decff9700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8dd80a1500 0x7f8dd80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:28.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.719+0000 7f8decff9700 1 -- 172.21.15.57:0/2612052906 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f8de0014070 con 0x7f8df0100e90 2023-10-06T22:14:28.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.719+0000 7f8deeffd700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8dd80a1500 0x7f8dd80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:28.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.720+0000 7f8deeffd700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8dd80a1500 0x7f8dd80a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8df0100bb0 tx=0x7f8de4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:28.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.723+0000 7f8decff9700 1 -- 172.21.15.57:0/2612052906 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8de00964a0 con 0x7f8df0100e90 2023-10-06T22:14:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:28 smithi057 ceph-mon[98492]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:14:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:28 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/625759152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:28.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:28 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/582997020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:28.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.935+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8df0107da0 con 0x7f8df0100e90 2023-10-06T22:14:28.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.936+0000 7f8decff9700 1 -- 172.21.15.57:0/2612052906 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8de0003dd0 con 0x7f8df0100e90 2023-10-06T22:14:28.938 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:14:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8dd80a1500 msgr2=0x7f8dd80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8dd80a1500 0x7f8dd80a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8df0100bb0 tx=0x7f8de4009260 comp rx=0 tx=0).stop 2023-10-06T22:14:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0100e90 msgr2=0x7f8df0074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:28.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0100e90 0x7f8df0074c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e970 tx=0x7f8de000ec80 comp rx=0 tx=0).stop 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 shutdown_connections 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f8dd80a1500 0x7f8dd80a39c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8df0102a30 0x7f8df00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8df0100e90 0x7f8df0074c60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 --2- 172.21.15.57:0/2612052906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df0102070 0x7f8df00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 >> 172.21.15.57:0/2612052906 conn(0x7f8df00fc430 msgr2=0x7f8df00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 shutdown_connections 2023-10-06T22:14:28.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:28.939+0000 7f8df5dff700 1 -- 172.21.15.57:0/2612052906 wait complete. 2023-10-06T22:14:28.960 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:14:29.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:28 smithi156 ceph-mon[134358]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:14:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:28 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/625759152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:29.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:28 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/582997020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:29.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:28 smithi190 ceph-mon[134891]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-06T22:14:29.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:28 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/625759152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:29.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:28 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/582997020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:29.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 -- 172.21.15.57:0/2583654050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 msgr2=0x7f6e2c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 --2- 172.21.15.57:0/2583654050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c102500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009990 tx=0x7f6e20009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 -- 172.21.15.57:0/2583654050 shutdown_connections 2023-10-06T22:14:29.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 --2- 172.21.15.57:0/2583654050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e2c102a40 0x7f6e2c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 --2- 172.21.15.57:0/2583654050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e2c100ea0 0x7f6e2c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 --2- 172.21.15.57:0/2583654050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c102500 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 -- 172.21.15.57:0/2583654050 >> 172.21.15.57:0/2583654050 conn(0x7f6e2c0fc420 msgr2=0x7f6e2c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:29.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 -- 172.21.15.57:0/2583654050 shutdown_connections 2023-10-06T22:14:29.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.163+0000 7f6e31fff700 1 -- 172.21.15.57:0/2583654050 wait complete. 2023-10-06T22:14:29.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 Processor -- start 2023-10-06T22:14:29.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 -- start start 2023-10-06T22:14:29.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e2c100ea0 0x7f6e2c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e2c102a40 0x7f6e2c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c073f80 con 0x7f6e2c102080 2023-10-06T22:14:29.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0740f0 con 0x7f6e2c100ea0 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.164+0000 7f6e31fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c1a0c00 con 0x7f6e2c102a40 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38256/0 (socket says 172.21.15.57:38256) 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e2c100ea0 0x7f6e2c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 -- 172.21.15.57:0/3439227774 learned_addr learned my addr 172.21.15.57:0/3439227774 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2bfff700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e2c102a40 0x7f6e2c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 -- 172.21.15.57:0/3439227774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e2c102a40 msgr2=0x7f6e2c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e2c102a40 0x7f6e2c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 -- 172.21.15.57:0/3439227774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e2c100ea0 msgr2=0x7f6e2c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e2c100ea0 0x7f6e2c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 -- 172.21.15.57:0/3439227774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e200095f0 con 0x7f6e2c102080 2023-10-06T22:14:29.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.165+0000 7f6e2affd700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c0732a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e20004710 tx=0x7f6e20009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:29.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.166+0000 7f6e28ff9700 1 -- 172.21.15.57:0/3439227774 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2001d070 con 0x7f6e2c102080 2023-10-06T22:14:29.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.166+0000 7f6e28ff9700 1 -- 172.21.15.57:0/3439227774 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e20021410 con 0x7f6e2c102080 2023-10-06T22:14:29.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.166+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c1a0e80 con 0x7f6e2c102080 2023-10-06T22:14:29.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.166+0000 7f6e28ff9700 1 -- 172.21.15.57:0/3439227774 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20017650 con 0x7f6e2c102080 2023-10-06T22:14:29.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.166+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c1a11f0 con 0x7f6e2c102080 2023-10-06T22:14:29.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.167+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c10c710 con 0x7f6e2c102080 2023-10-06T22:14:29.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.169+0000 7f6e28ff9700 1 -- 172.21.15.57:0/3439227774 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6e20021580 con 0x7f6e2c102080 2023-10-06T22:14:29.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.170+0000 7f6e28ff9700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6e140a14d0 0x7f6e140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.170+0000 7f6e28ff9700 1 -- 172.21.15.57:0/3439227774 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f6e200ce300 con 0x7f6e2c102080 2023-10-06T22:14:29.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.170+0000 7f6e2b7fe700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6e140a14d0 0x7f6e140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.171+0000 7f6e2b7fe700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6e140a14d0 0x7f6e140a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c101bf0 tx=0x7f6e1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.172+0000 7f6e28ff9700 1 -- 172.21.15.57:0/3439227774 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e20097a20 con 0x7f6e2c102080 2023-10-06T22:14:29.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.387+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e2c107e50 con 0x7f6e2c102080 2023-10-06T22:14:29.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.388+0000 7f6e28ff9700 1 -- 172.21.15.57:0/3439227774 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7f6e2009b040 con 0x7f6e2c102080 2023-10-06T22:14:29.391 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:29.391 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:14:29.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6e140a14d0 msgr2=0x7f6e140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6e140a14d0 0x7f6e140a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c101bf0 tx=0x7f6e1c009260 comp rx=0 tx=0).stop 2023-10-06T22:14:29.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 msgr2=0x7f6e2c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c0732a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e20004710 tx=0x7f6e20009580 comp rx=0 tx=0).stop 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 shutdown_connections 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f6e140a14d0 0x7f6e140a3990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e2c102a40 0x7f6e2c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e2c100ea0 0x7f6e2c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 --2- 172.21.15.57:0/3439227774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e2c102080 0x7f6e2c0732a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 >> 172.21.15.57:0/3439227774 conn(0x7f6e2c0fc420 msgr2=0x7f6e2c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 shutdown_connections 2023-10-06T22:14:29.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.391+0000 7f6e31fff700 1 -- 172.21.15.57:0/3439227774 wait complete. 2023-10-06T22:14:29.412 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:14:29.413 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:14:29.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 -- 172.21.15.57:0/2921256913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8108da0 msgr2=0x7f9ee810b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/2921256913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8108da0 0x7f9ee810b230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000b0a0 tx=0x7f9ee000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 -- 172.21.15.57:0/2921256913 shutdown_connections 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/2921256913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8108da0 0x7f9ee810b230 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/2921256913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8074ef0 0x7f9ee8073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/2921256913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ee8073910 0x7f9ee8073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 -- 172.21.15.57:0/2921256913 >> 172.21.15.57:0/2921256913 conn(0x7f9ee80fc430 msgr2=0x7f9ee80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 -- 172.21.15.57:0/2921256913 shutdown_connections 2023-10-06T22:14:29.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.618+0000 7f9eee9ff700 1 -- 172.21.15.57:0/2921256913 wait complete. 2023-10-06T22:14:29.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.619+0000 7f9eee9ff700 1 Processor -- start 2023-10-06T22:14:29.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.619+0000 7f9eee9ff700 1 -- start start 2023-10-06T22:14:29.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.619+0000 7f9eee9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8073910 0x7f9ee8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.619+0000 7f9eee9ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8074ef0 0x7f9ee8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.619+0000 7f9eee9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ee8108da0 0x7f9ee81a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9eee9ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a0aa0 con 0x7f9ee8074ef0 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9eee9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a0be0 con 0x7f9ee8073910 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9eee9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a0d70 con 0x7f9ee8108da0 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8073910 0x7f9ee8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8074ef0 0x7f9ee8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8073910 0x7f9ee8198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:43068/0 (socket says 172.21.15.57:43068) 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 -- 172.21.15.57:0/243971636 learned_addr learned my addr 172.21.15.57:0/243971636 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9eecf9c700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ee8108da0 0x7f9ee81a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 -- 172.21.15.57:0/243971636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ee8108da0 msgr2=0x7f9ee81a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ee8108da0 0x7f9ee81a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 -- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8074ef0 msgr2=0x7f9ee8199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8074ef0 0x7f9ee8199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 -- 172.21.15.57:0/243971636 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0009d00 con 0x7f9ee8073910 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee77fe700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8074ef0 0x7f9ee8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.620+0000 7f9ee7fff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8073910 0x7f9ee8198f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800e9a0 tx=0x7f9ed800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.621+0000 7f9ee57fa700 1 -- 172.21.15.57:0/243971636 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8009790 con 0x7f9ee8073910 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.621+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee81a0ff0 con 0x7f9ee8073910 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.621+0000 7f9ee57fa700 1 -- 172.21.15.57:0/243971636 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ed8010430 con 0x7f9ee8073910 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.621+0000 7f9ee57fa700 1 -- 172.21.15.57:0/243971636 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8018680 con 0x7f9ee8073910 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.621+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee81a14c0 con 0x7f9ee8073910 2023-10-06T22:14:29.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.622+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee8107450 con 0x7f9ee8073910 2023-10-06T22:14:29.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.624+0000 7f9ee57fa700 1 -- 172.21.15.57:0/243971636 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9ed80105a0 con 0x7f9ee8073910 2023-10-06T22:14:29.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.625+0000 7f9ee57fa700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9ed00a1480 0x7f9ed00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:29.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.626+0000 7f9ee57fa700 1 -- 172.21.15.57:0/243971636 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f9ed8014070 con 0x7f9ee8073910 2023-10-06T22:14:29.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.626+0000 7f9ee77fe700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9ed00a1480 0x7f9ed00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:29.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.626+0000 7f9ee77fe700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9ed00a1480 0x7f9ed00a3940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8074c10 tx=0x7f9edc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:29.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.627+0000 7f9ee57fa700 1 -- 172.21.15.57:0/243971636 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9ed80966f0 con 0x7f9ee8073910 2023-10-06T22:14:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.844+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ee80008d0 con 0x7f9ee8073910 2023-10-06T22:14:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.845+0000 7f9ee57fa700 1 -- 172.21.15.57:0/243971636 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ed8099d10 con 0x7f9ee8073910 2023-10-06T22:14:29.847 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:14:29.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9ed00a1480 msgr2=0x7f9ed00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9ed00a1480 0x7f9ed00a3940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8074c10 tx=0x7f9edc009260 comp rx=0 tx=0).stop 2023-10-06T22:14:29.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8073910 msgr2=0x7f9ee8198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:29.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8073910 0x7f9ee8198f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800e9a0 tx=0x7f9ed800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 shutdown_connections 2023-10-06T22:14:29.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f9ed00a1480 0x7f9ed00a3940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ee8108da0 0x7f9ee81a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ee8073910 0x7f9ee8198f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 --2- 172.21.15.57:0/243971636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee8074ef0 0x7f9ee8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:29.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.847+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 >> 172.21.15.57:0/243971636 conn(0x7f9ee80fc430 msgr2=0x7f9ee80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:29.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.848+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 shutdown_connections 2023-10-06T22:14:29.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:29.848+0000 7f9eee9ff700 1 -- 172.21.15.57:0/243971636 wait complete. 2023-10-06T22:14:29.869 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:14:30.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:29 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2612052906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:30.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:29 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3439227774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:30.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:29 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2612052906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:30.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:29 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3439227774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 -- 172.21.15.57:0/2310595163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102070 msgr2=0x7f2bb41024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/2310595163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102070 0x7f2bb41024f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40098f0 tx=0x7f2ba4009c00 comp rx=0 tx=0).stop 2023-10-06T22:14:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 -- 172.21.15.57:0/2310595163 shutdown_connections 2023-10-06T22:14:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/2310595163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bb4102a30 0x7f2bb4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/2310595163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4100e90 0x7f2bb41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/2310595163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102070 0x7f2bb41024f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 -- 172.21.15.57:0/2310595163 >> 172.21.15.57:0/2310595163 conn(0x7f2bb40fc430 msgr2=0x7f2bb40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:30.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 -- 172.21.15.57:0/2310595163 shutdown_connections 2023-10-06T22:14:30.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.075+0000 7f2bbbfff700 1 -- 172.21.15.57:0/2310595163 wait complete. 2023-10-06T22:14:30.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.076+0000 7f2bbbfff700 1 Processor -- start 2023-10-06T22:14:30.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.076+0000 7f2bbbfff700 1 -- start start 2023-10-06T22:14:30.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.076+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bb4100e90 0x7f2bb4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.076+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4102070 0x7f2bb40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.076+0000 7f2bbbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102a30 0x7f2bb40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bbbfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4073f80 con 0x7f2bb4102070 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bbbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40740f0 con 0x7f2bb4102a30 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bbbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb41a0cb0 con 0x7f2bb4100e90 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4102070 0x7f2bb40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb9d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bb4100e90 0x7f2bb4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb9d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bb4100e90 0x7f2bb4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58870/0 (socket says 172.21.15.57:58870) 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb9d9b700 1 -- 172.21.15.57:0/48159059 learned_addr learned my addr 172.21.15.57:0/48159059 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4102070 0x7f2bb40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38296/0 (socket says 172.21.15.57:38296) 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bba59c700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102a30 0x7f2bb40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 -- 172.21.15.57:0/48159059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bb4100e90 msgr2=0x7f2bb4074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bb4100e90 0x7f2bb4074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 -- 172.21.15.57:0/48159059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102a30 msgr2=0x7f2bb40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102a30 0x7f2bb40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 -- 172.21.15.57:0/48159059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb00095f0 con 0x7f2bb4102070 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bba59c700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102a30 0x7f2bb40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.077+0000 7f2bb959a700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4102070 0x7f2bb40732a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.078+0000 7f2baaffd700 1 -- 172.21.15.57:0/48159059 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba401c070 con 0x7f2bb4102070 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.078+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4009520 con 0x7f2bb4102070 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.078+0000 7f2baaffd700 1 -- 172.21.15.57:0/48159059 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ba400f7d0 con 0x7f2bb4102070 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.078+0000 7f2baaffd700 1 -- 172.21.15.57:0/48159059 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4003a40 con 0x7f2bb4102070 2023-10-06T22:14:30.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.078+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb41a11d0 con 0x7f2bb4102070 2023-10-06T22:14:30.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.079+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb410c700 con 0x7f2bb4102070 2023-10-06T22:14:30.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.081+0000 7f2baaffd700 1 -- 172.21.15.57:0/48159059 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2ba4020070 con 0x7f2bb4102070 2023-10-06T22:14:30.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.082+0000 7f2baaffd700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2ba00a1480 0x7f2ba00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.082+0000 7f2baaffd700 1 -- 172.21.15.57:0/48159059 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f2ba40cd860 con 0x7f2bb4102070 2023-10-06T22:14:30.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.083+0000 7f2bb9d9b700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2ba00a1480 0x7f2ba00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.083+0000 7f2bb9d9b700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2ba00a1480 0x7f2ba00a3940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0005950 tx=0x7f2bb0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:30.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.086+0000 7f2baaffd700 1 -- 172.21.15.57:0/48159059 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ba4096f20 con 0x7f2bb4102070 2023-10-06T22:14:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:29 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2612052906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:30.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:29 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3439227774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:30.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.298+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bb4107ec0 con 0x7f2bb4102070 2023-10-06T22:14:30.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.298+0000 7f2baaffd700 1 -- 172.21.15.57:0/48159059 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ba409a540 con 0x7f2bb4102070 2023-10-06T22:14:30.301 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:14:30.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2ba00a1480 msgr2=0x7f2ba00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2ba00a1480 0x7f2ba00a3940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0005950 tx=0x7f2bb0009310 comp rx=0 tx=0).stop 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4102070 msgr2=0x7f2bb40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4102070 0x7f2bb40732a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba400b790 comp rx=0 tx=0).stop 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 shutdown_connections 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2ba00a1480 0x7f2ba00a3940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bb4102a30 0x7f2bb40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bb4100e90 0x7f2bb4074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 --2- 172.21.15.57:0/48159059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bb4102070 0x7f2bb40732a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 >> 172.21.15.57:0/48159059 conn(0x7f2bb40fc430 msgr2=0x7f2bb40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:30.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 shutdown_connections 2023-10-06T22:14:30.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.301+0000 7f2bbbfff700 1 -- 172.21.15.57:0/48159059 wait complete. 2023-10-06T22:14:30.322 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:14:30.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 -- 172.21.15.57:0/2369633621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0100e90 msgr2=0x7fc3f01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/2369633621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0100e90 0x7fc3f01012b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009990 tx=0x7fc3e0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 -- 172.21.15.57:0/2369633621 shutdown_connections 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/2369633621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f0102a30 0x7fc3f0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/2369633621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0100e90 0x7fc3f01012b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/2369633621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0102070 0x7fc3f01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 -- 172.21.15.57:0/2369633621 >> 172.21.15.57:0/2369633621 conn(0x7fc3f00fc430 msgr2=0x7fc3f00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 -- 172.21.15.57:0/2369633621 shutdown_connections 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.526+0000 7fc3f5dff700 1 -- 172.21.15.57:0/2369633621 wait complete. 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.527+0000 7fc3f5dff700 1 Processor -- start 2023-10-06T22:14:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.527+0000 7fc3f5dff700 1 -- start start 2023-10-06T22:14:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3f5dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0100e90 0x7fc3f0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3f5dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0102070 0x7fc3f00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3f5dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f0102a30 0x7fc3f00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3f5dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f0073f80 con 0x7fc3f0102070 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3f5dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00740f0 con 0x7fc3f0100e90 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3f5dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f01a0c90 con 0x7fc3f0102a30 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3eeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0102070 0x7fc3f00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3eeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0102070 0x7fc3f00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38314/0 (socket says 172.21.15.57:38314) 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3eeffd700 1 -- 172.21.15.57:0/838916634 learned_addr learned my addr 172.21.15.57:0/838916634 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3ef7fe700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0100e90 0x7fc3f0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3effff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f0102a30 0x7fc3f00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.528+0000 7fc3eeffd700 1 -- 172.21.15.57:0/838916634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f0102a30 msgr2=0x7fc3f00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3eeffd700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f0102a30 0x7fc3f00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3eeffd700 1 -- 172.21.15.57:0/838916634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0100e90 msgr2=0x7fc3f0074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3eeffd700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0100e90 0x7fc3f0074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3eeffd700 1 -- 172.21.15.57:0/838916634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e00095f0 con 0x7fc3f0102070 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3effff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f0102a30 0x7fc3f00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3eeffd700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0102070 0x7fc3f00732a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400e970 tx=0x7fc3e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3ecff9700 1 -- 172.21.15.57:0/838916634 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4009790 con 0x7fc3f0102070 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f01a0f70 con 0x7fc3f0102070 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3ecff9700 1 -- 172.21.15.57:0/838916634 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3e400eec0 con 0x7fc3f0102070 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3ecff9700 1 -- 172.21.15.57:0/838916634 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4003cf0 con 0x7fc3f0102070 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.529+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f01a1440 con 0x7fc3f0102070 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.530+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f0107e90 con 0x7fc3f0102070 2023-10-06T22:14:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.532+0000 7fc3ecff9700 1 -- 172.21.15.57:0/838916634 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc3e400ccd0 con 0x7fc3f0102070 2023-10-06T22:14:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.534+0000 7fc3ecff9700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3d80a1480 0x7fc3d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.534+0000 7fc3ecff9700 1 -- 172.21.15.57:0/838916634 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fc3e4014070 con 0x7fc3f0102070 2023-10-06T22:14:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.534+0000 7fc3ef7fe700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3d80a1480 0x7fc3d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.535+0000 7fc3ef7fe700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3d80a1480 0x7fc3d80a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000b2e0 tx=0x7fc3e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:30.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.537+0000 7fc3ecff9700 1 -- 172.21.15.57:0/838916634 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc3e40965c0 con 0x7fc3f0102070 2023-10-06T22:14:30.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.748+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3f002d270 con 0x7fc3f0102070 2023-10-06T22:14:30.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.748+0000 7fc3ecff9700 1 -- 172.21.15.57:0/838916634 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc3e4099be0 con 0x7fc3f0102070 2023-10-06T22:14:30.751 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:14:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3d80a1480 msgr2=0x7fc3d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3d80a1480 0x7fc3d80a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000b2e0 tx=0x7fc3e0005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0102070 msgr2=0x7fc3f00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0102070 0x7fc3f00732a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400e970 tx=0x7fc3e400ec80 comp rx=0 tx=0).stop 2023-10-06T22:14:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 shutdown_connections 2023-10-06T22:14:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc3d80a1480 0x7fc3d80a3940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3f0102a30 0x7fc3f00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f0100e90 0x7fc3f0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 --2- 172.21.15.57:0/838916634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc3f0102070 0x7fc3f00732a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 >> 172.21.15.57:0/838916634 conn(0x7fc3f00fc430 msgr2=0x7fc3f00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 shutdown_connections 2023-10-06T22:14:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.751+0000 7fc3f5dff700 1 -- 172.21.15.57:0/838916634 wait complete. 2023-10-06T22:14:30.772 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-06T22:14:30.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:14:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 -- 172.21.15.57:0/3789485981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102080 msgr2=0x7f0834102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 --2- 172.21.15.57:0/3789485981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102080 0x7f0834102500 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0828009990 tx=0x7f0828009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 -- 172.21.15.57:0/3789485981 shutdown_connections 2023-10-06T22:14:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 --2- 172.21.15.57:0/3789485981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0834102a40 0x7f0834109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 --2- 172.21.15.57:0/3789485981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0834100ea0 0x7f08341012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 --2- 172.21.15.57:0/3789485981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102080 0x7f0834102500 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 -- 172.21.15.57:0/3789485981 >> 172.21.15.57:0/3789485981 conn(0x7f08340fc420 msgr2=0x7f08340fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.975+0000 7f08395ff700 1 -- 172.21.15.57:0/3789485981 shutdown_connections 2023-10-06T22:14:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.976+0000 7f08395ff700 1 -- 172.21.15.57:0/3789485981 wait complete. 2023-10-06T22:14:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.977+0000 7f08395ff700 1 Processor -- start 2023-10-06T22:14:30.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.977+0000 7f08395ff700 1 -- start start 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08395ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0834100ea0 0x7f0834074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08395ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0834102080 0x7f08340732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08395ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102a40 0x7f08340737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08395ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0834073f80 con 0x7f0834102a40 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08395ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08340740f0 con 0x7f0834100ea0 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08395ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08341a0c50 con 0x7f0834102080 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102a40 0x7f08340737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102a40 0x7f08340737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38346/0 (socket says 172.21.15.57:38346) 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08337fe700 1 -- 172.21.15.57:0/2824817212 learned_addr learned my addr 172.21.15.57:0/2824817212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:30.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f0832ffd700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0834100ea0 0x7f0834074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.978+0000 7f08327fc700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0834102080 0x7f08340732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08337fe700 1 -- 172.21.15.57:0/2824817212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0834102080 msgr2=0x7f08340732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08337fe700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0834102080 0x7f08340732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08337fe700 1 -- 172.21.15.57:0/2824817212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0834100ea0 msgr2=0x7f0834074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08337fe700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0834100ea0 0x7f0834074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08337fe700 1 -- 172.21.15.57:0/2824817212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280095f0 con 0x7f0834102a40 2023-10-06T22:14:30.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08337fe700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102a40 0x7f08340737e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f082c00ef30 tx=0x7f082c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f081bfff700 1 -- 172.21.15.57:0/2824817212 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c018070 con 0x7f0834102a40 2023-10-06T22:14:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f081bfff700 1 -- 172.21.15.57:0/2824817212 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f082c00f040 con 0x7f0834102a40 2023-10-06T22:14:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08341a0ed0 con 0x7f0834102a40 2023-10-06T22:14:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f081bfff700 1 -- 172.21.15.57:0/2824817212 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c00cd00 con 0x7f0834102a40 2023-10-06T22:14:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.979+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08341a1300 con 0x7f0834102a40 2023-10-06T22:14:30.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.980+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083410c760 con 0x7f0834102a40 2023-10-06T22:14:30.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.982+0000 7f081bfff700 1 -- 172.21.15.57:0/2824817212 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f082c003a00 con 0x7f0834102a40 2023-10-06T22:14:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.984+0000 7f081bfff700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f081c0a14d0 0x7f081c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.984+0000 7f081bfff700 1 -- 172.21.15.57:0/2824817212 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f082c0cc840 con 0x7f0834102a40 2023-10-06T22:14:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.984+0000 7f0832ffd700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f081c0a14d0 0x7f081c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:30.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.984+0000 7f0832ffd700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f081c0a14d0 0x7f081c0a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0834101bf0 tx=0x7f0824009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:30.988+0000 7f081bfff700 1 -- 172.21.15.57:0/2824817212 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f082c095f00 con 0x7f0834102a40 2023-10-06T22:14:31.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:30 smithi156 ceph-mon[134358]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2023-10-06T22:14:31.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:30 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/243971636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:31.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:30 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/48159059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:31.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:30 smithi190 ceph-mon[134891]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2023-10-06T22:14:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:30 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/243971636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:31.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:30 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/48159059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:31.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:30 smithi057 ceph-mon[98492]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2023-10-06T22:14:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:30 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/243971636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:30 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/48159059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:31.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.248+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0834107e40 con 0x7f0834102a40 2023-10-06T22:14:31.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.249+0000 7f081bfff700 1 -- 172.21.15.57:0/2824817212 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f082c099520 con 0x7f0834102a40 2023-10-06T22:14:31.251 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:31.251 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.251+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f081c0a14d0 msgr2=0x7f081c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.251+0000 7f08395ff700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f081c0a14d0 0x7f081c0a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0834101bf0 tx=0x7f0824009260 comp rx=0 tx=0).stop 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.251+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102a40 msgr2=0x7f08340737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.251+0000 7f08395ff700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102a40 0x7f08340737e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f082c00ef30 tx=0x7f082c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 shutdown_connections 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f081c0a14d0 0x7f081c0a3990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0834102a40 0x7f08340737e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0834100ea0 0x7f0834074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 --2- 172.21.15.57:0/2824817212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0834102080 0x7f08340732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 >> 172.21.15.57:0/2824817212 conn(0x7f08340fc420 msgr2=0x7f08340fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:31.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 shutdown_connections 2023-10-06T22:14:31.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.252+0000 7f08395ff700 1 -- 172.21.15.57:0/2824817212 wait complete. 2023-10-06T22:14:31.255 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:14:31.273 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-06T22:14:31.273 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-06T22:14:31.273 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T22:14:31.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.474+0000 7fcadd3ff700 1 -- 172.21.15.57:0/3789335562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8100ea0 msgr2=0x7fcad81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.474+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/3789335562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8100ea0 0x7fcad81012c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009990 tx=0x7fcac8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.475+0000 7fcadd3ff700 1 -- 172.21.15.57:0/3789335562 shutdown_connections 2023-10-06T22:14:31.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.475+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/3789335562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcad8102a40 0x7fcad8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.475+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/3789335562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8100ea0 0x7fcad81012c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.475+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/3789335562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcad8102080 0x7fcad8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.475+0000 7fcadd3ff700 1 -- 172.21.15.57:0/3789335562 >> 172.21.15.57:0/3789335562 conn(0x7fcad80fc420 msgr2=0x7fcad80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:31.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.475+0000 7fcadd3ff700 1 -- 172.21.15.57:0/3789335562 shutdown_connections 2023-10-06T22:14:31.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.475+0000 7fcadd3ff700 1 -- 172.21.15.57:0/3789335562 wait complete. 2023-10-06T22:14:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 Processor -- start 2023-10-06T22:14:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 -- start start 2023-10-06T22:14:31.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcad8100ea0 0x7fcad8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcad8102080 0x7fcad8197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8102a40 0x7fcad819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad819e760 con 0x7fcad8102a40 2023-10-06T22:14:31.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad819e8d0 con 0x7fcad8102080 2023-10-06T22:14:31.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcadd3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad819ea90 con 0x7fcad8100ea0 2023-10-06T22:14:31.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.476+0000 7fcad77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8102a40 0x7fcad819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcad8100ea0 0x7fcad8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8102a40 0x7fcad819e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38376/0 (socket says 172.21.15.57:38376) 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 -- 172.21.15.57:0/1447520136 learned_addr learned my addr 172.21.15.57:0/1447520136 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad67fc700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcad8102080 0x7fcad8197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 -- 172.21.15.57:0/1447520136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcad8100ea0 msgr2=0x7fcad8196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcad8100ea0 0x7fcad8196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 -- 172.21.15.57:0/1447520136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcad8102080 msgr2=0x7fcad8197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcad8102080 0x7fcad8197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 -- 172.21.15.57:0/1447520136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80095f0 con 0x7fcad8102a40 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcad77fe700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8102a40 0x7fcad819e190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcad000ef30 tx=0x7fcad000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcabffff700 1 -- 172.21.15.57:0/1447520136 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0018070 con 0x7fcad8102a40 2023-10-06T22:14:31.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcabffff700 1 -- 172.21.15.57:0/1447520136 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcad000f040 con 0x7fcad8102a40 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad80730e0 con 0x7fcad8102a40 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.477+0000 7fcabffff700 1 -- 172.21.15.57:0/1447520136 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad000cd00 con 0x7fcad8102a40 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.478+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad80735a0 con 0x7fcad8102a40 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.479+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad810c760 con 0x7fcad8102a40 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.480+0000 7fcabffff700 1 -- 172.21.15.57:0/1447520136 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcad0003a00 con 0x7fcad8102a40 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.482+0000 7fcabffff700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcac00a14d0 0x7fcac00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.482+0000 7fcabffff700 1 -- 172.21.15.57:0/1447520136 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fcad00cc8a0 con 0x7fcad8102a40 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.482+0000 7fcad6ffd700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcac00a14d0 0x7fcac00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.483+0000 7fcad6ffd700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcac00a14d0 0x7fcac00a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:31.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.486+0000 7fcabffff700 1 -- 172.21.15.57:0/1447520136 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcad0095f60 con 0x7fcad8102a40 2023-10-06T22:14:31.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.693+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcad80612f0 con 0x7fcac00a14d0 2023-10-06T22:14:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.706+0000 7fcabffff700 1 -- 172.21.15.57:0/1447520136 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345234 (secure 0 0 0) 0x7fcad80612f0 con 0x7fcac00a14d0 2023-10-06T22:14:31.712 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:31.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcac00a14d0 msgr2=0x7fcac00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcac00a14d0 0x7fcac00a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac8005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8102a40 msgr2=0x7fcad819e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8102a40 0x7fcad819e190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcad000ef30 tx=0x7fcad000c3a0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 shutdown_connections 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fcac00a14d0 0x7fcac00a3990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad8102a40 0x7fcad819e190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcad8100ea0 0x7fcad8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 --2- 172.21.15.57:0/1447520136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcad8102080 0x7fcad8197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 >> 172.21.15.57:0/1447520136 conn(0x7fcad80fc420 msgr2=0x7fcad80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 shutdown_connections 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.710+0000 7fcadd3ff700 1 -- 172.21.15.57:0/1447520136 wait complete. 2023-10-06T22:14:31.716 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-06T22:14:31.732 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-10-06T22:14:31.647726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657018,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1150,"num_read_kb":694,"num_write":428,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337180,"kb_used_data":12988,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604644,"statfs":{"total":1151940427776,"available":1151595155456,"internally_reserved":0,"allocated":13299712,"data_stored":7285070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":52,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002915"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676898+0000","last_change":"2023-10-06T22:12:43.704484+0000","last_active":"2023-10-06T22:14:05.676898+0000","last_peered":"2023-10-06T22:14:05.676898+0000","last_clean":"2023-10-06T22:14:05.676898+0000","last_became_active":"2023-10-06T22:12:43.703740+0000","last_became_peered":"2023-10-06T22:12:43.703740+0000","last_unstale":"2023-10-06T22:14:05.676898+0000","last_undegraded":"2023-10-06T22:14:05.676898+0000","last_fullsized":"2023-10-06T22:14:05.676898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:25.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672032+0000","last_change":"2023-10-06T22:12:43.696968+0000","last_active":"2023-10-06T22:14:05.672032+0000","last_peered":"2023-10-06T22:14:05.672032+0000","last_clean":"2023-10-06T22:14:05.672032+0000","last_became_active":"2023-10-06T22:12:43.696532+0000","last_became_peered":"2023-10-06T22:12:43.696532+0000","last_unstale":"2023-10-06T22:14:05.672032+0000","last_undegraded":"2023-10-06T22:14:05.672032+0000","last_fullsized":"2023-10-06T22:14:05.672032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:59:50.754606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675300+0000","last_change":"2023-10-06T22:12:43.700845+0000","last_active":"2023-10-06T22:14:05.675300+0000","last_peered":"2023-10-06T22:14:05.675300+0000","last_clean":"2023-10-06T22:14:05.675300+0000","last_became_active":"2023-10-06T22:12:43.700531+0000","last_became_peered":"2023-10-06T22:12:43.700531+0000","last_unstale":"2023-10-06T22:14:05.675300+0000","last_undegraded":"2023-10-06T22:14:05.675300+0000","last_fullsized":"2023-10-06T22:14:05.675300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:18.518772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667765+0000","last_change":"2023-10-06T22:12:43.700806+0000","last_active":"2023-10-06T22:14:05.667765+0000","last_peered":"2023-10-06T22:14:05.667765+0000","last_clean":"2023-10-06T22:14:05.667765+0000","last_became_active":"2023-10-06T22:12:43.699901+0000","last_became_peered":"2023-10-06T22:12:43.699901+0000","last_unstale":"2023-10-06T22:14:05.667765+0000","last_undegraded":"2023-10-06T22:14:05.667765+0000","last_fullsized":"2023-10-06T22:14:05.667765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:41:35.906415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.674088+0000","last_change":"2023-10-06T22:12:43.694776+0000","last_active":"2023-10-06T22:14:05.674088+0000","last_peered":"2023-10-06T22:14:05.674088+0000","last_clean":"2023-10-06T22:14:05.674088+0000","last_became_active":"2023-10-06T22:12:43.694397+0000","last_became_peered":"2023-10-06T22:12:43.694397+0000","last_unstale":"2023-10-06T22:14:05.674088+0000","last_undegraded":"2023-10-06T22:14:05.674088+0000","last_fullsized":"2023-10-06T22:14:05.674088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:47:43.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665097+0000","last_change":"2023-10-06T22:12:43.703515+0000","last_active":"2023-10-06T22:14:05.665097+0000","last_peered":"2023-10-06T22:14:05.665097+0000","last_clean":"2023-10-06T22:14:05.665097+0000","last_became_active":"2023-10-06T22:12:43.703094+0000","last_became_peered":"2023-10-06T22:12:43.703094+0000","last_unstale":"2023-10-06T22:14:05.665097+0000","last_undegraded":"2023-10-06T22:14:05.665097+0000","last_fullsized":"2023-10-06T22:14:05.665097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:54:32.130131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667950+0000","last_change":"2023-10-06T22:12:43.697894+0000","last_active":"2023-10-06T22:14:05.667950+0000","last_peered":"2023-10-06T22:14:05.667950+0000","last_clean":"2023-10-06T22:14:05.667950+0000","last_became_active":"2023-10-06T22:12:43.697591+0000","last_became_peered":"2023-10-06T22:12:43.697591+0000","last_unstale":"2023-10-06T22:14:05.667950+0000","last_undegraded":"2023-10-06T22:14:05.667950+0000","last_fullsized":"2023-10-06T22:14:05.667950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:30.313447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668433+0000","last_change":"2023-10-06T22:13:06.927879+0000","last_active":"2023-10-06T22:14:05.668433+0000","last_peered":"2023-10-06T22:14:05.668433+0000","last_clean":"2023-10-06T22:14:05.668433+0000","last_became_active":"2023-10-06T22:13:06.927559+0000","last_became_peered":"2023-10-06T22:13:06.927559+0000","last_unstale":"2023-10-06T22:14:05.668433+0000","last_undegraded":"2023-10-06T22:14:05.668433+0000","last_fullsized":"2023-10-06T22:14:05.668433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:15.874159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664732+0000","last_change":"2023-10-06T22:12:43.696995+0000","last_active":"2023-10-06T22:14:05.664732+0000","last_peered":"2023-10-06T22:14:05.664732+0000","last_clean":"2023-10-06T22:14:05.664732+0000","last_became_active":"2023-10-06T22:12:43.696731+0000","last_became_peered":"2023-10-06T22:12:43.696731+0000","last_unstale":"2023-10-06T22:14:05.664732+0000","last_undegraded":"2023-10-06T22:14:05.664732+0000","last_fullsized":"2023-10-06T22:14:05.664732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:43:31.261922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663417+0000","last_change":"2023-10-06T22:12:43.704391+0000","last_active":"2023-10-06T22:14:05.663417+0000","last_peered":"2023-10-06T22:14:05.663417+0000","last_clean":"2023-10-06T22:14:05.663417+0000","last_became_active":"2023-10-06T22:12:43.702954+0000","last_became_peered":"2023-10-06T22:12:43.702954+0000","last_unstale":"2023-10-06T22:14:05.663417+0000","last_undegraded":"2023-10-06T22:14:05.663417+0000","last_fullsized":"2023-10-06T22:14:05.663417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:38.973781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666760+0000","last_change":"2023-10-06T22:12:43.696265+0000","last_active":"2023-10-06T22:14:05.666760+0000","last_peered":"2023-10-06T22:14:05.666760+0000","last_clean":"2023-10-06T22:14:05.666760+0000","last_became_active":"2023-10-06T22:12:43.695939+0000","last_became_peered":"2023-10-06T22:12:43.695939+0000","last_unstale":"2023-10-06T22:14:05.666760+0000","last_undegraded":"2023-10-06T22:14:05.666760+0000","last_fullsized":"2023-10-06T22:14:05.666760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:32.882083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668888+0000","last_change":"2023-10-06T22:13:06.812700+0000","last_active":"2023-10-06T22:14:05.668888+0000","last_peered":"2023-10-06T22:14:05.668888+0000","last_clean":"2023-10-06T22:14:05.668888+0000","last_became_active":"2023-10-06T22:13:06.812313+0000","last_became_peered":"2023-10-06T22:13:06.812313+0000","last_unstale":"2023-10-06T22:14:05.668888+0000","last_undegraded":"2023-10-06T22:14:05.668888+0000","last_fullsized":"2023-10-06T22:14:05.668888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:25.010233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676269+0000","last_change":"2023-10-06T22:12:43.694717+0000","last_active":"2023-10-06T22:14:05.676269+0000","last_peered":"2023-10-06T22:14:05.676269+0000","last_clean":"2023-10-06T22:14:05.676269+0000","last_became_active":"2023-10-06T22:12:43.694321+0000","last_became_peered":"2023-10-06T22:12:43.694321+0000","last_unstale":"2023-10-06T22:14:05.676269+0000","last_undegraded":"2023-10-06T22:14:05.676269+0000","last_fullsized":"2023-10-06T22:14:05.676269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:06.334726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669546+0000","last_change":"2023-10-06T22:12:43.703770+0000","last_active":"2023-10-06T22:14:05.669546+0000","last_peered":"2023-10-06T22:14:05.669546+0000","last_clean":"2023-10-06T22:14:05.669546+0000","last_became_active":"2023-10-06T22:12:43.703434+0000","last_became_peered":"2023-10-06T22:12:43.703434+0000","last_unstale":"2023-10-06T22:14:05.669546+0000","last_undegraded":"2023-10-06T22:14:05.669546+0000","last_fullsized":"2023-10-06T22:14:05.669546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:25.894280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676822+0000","last_change":"2023-10-06T22:12:43.700994+0000","last_active":"2023-10-06T22:14:05.676822+0000","last_peered":"2023-10-06T22:14:05.676822+0000","last_clean":"2023-10-06T22:14:05.676822+0000","last_became_active":"2023-10-06T22:12:43.700652+0000","last_became_peered":"2023-10-06T22:12:43.700652+0000","last_unstale":"2023-10-06T22:14:05.676822+0000","last_undegraded":"2023-10-06T22:14:05.676822+0000","last_fullsized":"2023-10-06T22:14:05.676822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:54.855597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676182+0000","last_change":"2023-10-06T22:12:43.702099+0000","last_active":"2023-10-06T22:14:05.676182+0000","last_peered":"2023-10-06T22:14:05.676182+0000","last_clean":"2023-10-06T22:14:05.676182+0000","last_became_active":"2023-10-06T22:12:43.701802+0000","last_became_peered":"2023-10-06T22:12:43.701802+0000","last_unstale":"2023-10-06T22:14:05.676182+0000","last_undegraded":"2023-10-06T22:14:05.676182+0000","last_fullsized":"2023-10-06T22:14:05.676182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:00.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677168+0000","last_change":"2023-10-06T22:12:43.704679+0000","last_active":"2023-10-06T22:14:05.677168+0000","last_peered":"2023-10-06T22:14:05.677168+0000","last_clean":"2023-10-06T22:14:05.677168+0000","last_became_active":"2023-10-06T22:12:43.704186+0000","last_became_peered":"2023-10-06T22:12:43.704186+0000","last_unstale":"2023-10-06T22:14:05.677168+0000","last_undegraded":"2023-10-06T22:14:05.677168+0000","last_fullsized":"2023-10-06T22:14:05.677168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:28.665217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668666+0000","last_change":"2023-10-06T22:12:43.695699+0000","last_active":"2023-10-06T22:14:05.668666+0000","last_peered":"2023-10-06T22:14:05.668666+0000","last_clean":"2023-10-06T22:14:05.668666+0000","last_became_active":"2023-10-06T22:12:43.695359+0000","last_became_peered":"2023-10-06T22:12:43.695359+0000","last_unstale":"2023-10-06T22:14:05.668666+0000","last_undegraded":"2023-10-06T22:14:05.668666+0000","last_fullsized":"2023-10-06T22:14:05.668666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:46.296768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672411+0000","last_change":"2023-10-06T22:12:43.707170+0000","last_active":"2023-10-06T22:14:05.672411+0000","last_peered":"2023-10-06T22:14:05.672411+0000","last_clean":"2023-10-06T22:14:05.672411+0000","last_became_active":"2023-10-06T22:12:43.706875+0000","last_became_peered":"2023-10-06T22:12:43.706875+0000","last_unstale":"2023-10-06T22:14:05.672411+0000","last_undegraded":"2023-10-06T22:14:05.672411+0000","last_fullsized":"2023-10-06T22:14:05.672411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:46.548458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":147,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:24.833047+0000","last_change":"2023-10-06T22:13:06.736382+0000","last_active":"2023-10-06T22:14:24.833047+0000","last_peered":"2023-10-06T22:14:24.833047+0000","last_clean":"2023-10-06T22:14:24.833047+0000","last_became_active":"2023-10-06T22:13:06.725703+0000","last_became_peered":"2023-10-06T22:13:06.725703+0000","last_unstale":"2023-10-06T22:14:24.833047+0000","last_undegraded":"2023-10-06T22:14:24.833047+0000","last_fullsized":"2023-10-06T22:14:24.833047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:10.056384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671657+0000","last_change":"2023-10-06T22:12:43.702574+0000","last_active":"2023-10-06T22:14:05.671657+0000","last_peered":"2023-10-06T22:14:05.671657+0000","last_clean":"2023-10-06T22:14:05.671657+0000","last_became_active":"2023-10-06T22:12:43.702238+0000","last_became_peered":"2023-10-06T22:12:43.702238+0000","last_unstale":"2023-10-06T22:14:05.671657+0000","last_undegraded":"2023-10-06T22:14:05.671657+0000","last_fullsized":"2023-10-06T22:14:05.671657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:34:09.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752842+0000","last_change":"2023-10-06T22:12:42.697256+0000","last_active":"2023-10-06T22:13:05.752842+0000","last_peered":"2023-10-06T22:13:05.752842+0000","last_clean":"2023-10-06T22:13:05.752842+0000","last_became_active":"2023-10-06T22:12:42.696724+0000","last_became_peered":"2023-10-06T22:12:42.696724+0000","last_unstale":"2023-10-06T22:13:05.752842+0000","last_undegraded":"2023-10-06T22:13:05.752842+0000","last_fullsized":"2023-10-06T22:13:05.752842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:50:01.378292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679610+0000","last_change":"2023-10-06T22:12:43.702050+0000","last_active":"2023-10-06T22:14:05.679610+0000","last_peered":"2023-10-06T22:14:05.679610+0000","last_clean":"2023-10-06T22:14:05.679610+0000","last_became_active":"2023-10-06T22:12:43.701432+0000","last_became_peered":"2023-10-06T22:12:43.701432+0000","last_unstale":"2023-10-06T22:14:05.679610+0000","last_undegraded":"2023-10-06T22:14:05.679610+0000","last_fullsized":"2023-10-06T22:14:05.679610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:30:37.909775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.948160+0000","last_change":"2023-10-06T22:13:06.948160+0000","last_active":"2023-10-06T22:13:06.948160+0000","last_peered":"2023-10-06T22:13:06.948160+0000","last_clean":"2023-10-06T22:13:06.948160+0000","last_became_active":"2023-10-06T22:13:06.927995+0000","last_became_peered":"2023-10-06T22:13:06.927995+0000","last_unstale":"2023-10-06T22:13:06.948160+0000","last_undegraded":"2023-10-06T22:13:06.948160+0000","last_fullsized":"2023-10-06T22:13:06.948160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:33:27.974776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.655518+0000","last_change":"2023-10-06T22:12:43.696511+0000","last_active":"2023-10-06T22:14:05.655518+0000","last_peered":"2023-10-06T22:14:05.655518+0000","last_clean":"2023-10-06T22:14:05.655518+0000","last_became_active":"2023-10-06T22:12:43.696087+0000","last_became_peered":"2023-10-06T22:12:43.696087+0000","last_unstale":"2023-10-06T22:14:05.655518+0000","last_undegraded":"2023-10-06T22:14:05.655518+0000","last_fullsized":"2023-10-06T22:14:05.655518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:12:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724948+0000","last_change":"2023-10-06T22:12:42.694963+0000","last_active":"2023-10-06T22:13:05.724948+0000","last_peered":"2023-10-06T22:13:05.724948+0000","last_clean":"2023-10-06T22:13:05.724948+0000","last_became_active":"2023-10-06T22:12:42.694517+0000","last_became_peered":"2023-10-06T22:12:42.694517+0000","last_unstale":"2023-10-06T22:13:05.724948+0000","last_undegraded":"2023-10-06T22:13:05.724948+0000","last_fullsized":"2023-10-06T22:13:05.724948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:07:33.404328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663561+0000","last_change":"2023-10-06T22:12:43.698077+0000","last_active":"2023-10-06T22:14:05.663561+0000","last_peered":"2023-10-06T22:14:05.663561+0000","last_clean":"2023-10-06T22:14:05.663561+0000","last_became_active":"2023-10-06T22:12:43.697676+0000","last_became_peered":"2023-10-06T22:12:43.697676+0000","last_unstale":"2023-10-06T22:14:05.663561+0000","last_undegraded":"2023-10-06T22:14:05.663561+0000","last_fullsized":"2023-10-06T22:14:05.663561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:59:56.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808989+0000","last_change":"2023-10-06T22:12:42.698067+0000","last_active":"2023-10-06T22:14:27.808989+0000","last_peered":"2023-10-06T22:14:27.808989+0000","last_clean":"2023-10-06T22:14:27.808989+0000","last_became_active":"2023-10-06T22:12:42.697671+0000","last_became_peered":"2023-10-06T22:12:42.697671+0000","last_unstale":"2023-10-06T22:14:27.808989+0000","last_undegraded":"2023-10-06T22:14:27.808989+0000","last_fullsized":"2023-10-06T22:14:27.808989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:33.607513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.661778+0000","last_change":"2023-10-06T22:12:43.696213+0000","last_active":"2023-10-06T22:14:05.661778+0000","last_peered":"2023-10-06T22:14:05.661778+0000","last_clean":"2023-10-06T22:14:05.661778+0000","last_became_active":"2023-10-06T22:12:43.695925+0000","last_became_peered":"2023-10-06T22:12:43.695925+0000","last_unstale":"2023-10-06T22:14:05.661778+0000","last_undegraded":"2023-10-06T22:14:05.661778+0000","last_fullsized":"2023-10-06T22:14:05.661778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:24.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.719695+0000","last_change":"2023-10-06T22:12:42.687188+0000","last_active":"2023-10-06T22:13:05.719695+0000","last_peered":"2023-10-06T22:13:05.719695+0000","last_clean":"2023-10-06T22:13:05.719695+0000","last_became_active":"2023-10-06T22:12:42.686747+0000","last_became_peered":"2023-10-06T22:12:42.686747+0000","last_unstale":"2023-10-06T22:13:05.719695+0000","last_undegraded":"2023-10-06T22:13:05.719695+0000","last_fullsized":"2023-10-06T22:13:05.719695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:21.326911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664388+0000","last_change":"2023-10-06T22:12:43.697988+0000","last_active":"2023-10-06T22:14:05.664388+0000","last_peered":"2023-10-06T22:14:05.664388+0000","last_clean":"2023-10-06T22:14:05.664388+0000","last_became_active":"2023-10-06T22:12:43.697641+0000","last_became_peered":"2023-10-06T22:12:43.697641+0000","last_unstale":"2023-10-06T22:14:05.664388+0000","last_undegraded":"2023-10-06T22:14:05.664388+0000","last_fullsized":"2023-10-06T22:14:05.664388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:33.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.423261+0000","last_change":"2023-10-06T22:12:42.688882+0000","last_active":"2023-10-06T22:13:06.423261+0000","last_peered":"2023-10-06T22:13:06.423261+0000","last_clean":"2023-10-06T22:13:06.423261+0000","last_became_active":"2023-10-06T22:12:42.687653+0000","last_became_peered":"2023-10-06T22:12:42.687653+0000","last_unstale":"2023-10-06T22:13:06.423261+0000","last_undegraded":"2023-10-06T22:13:06.423261+0000","last_fullsized":"2023-10-06T22:13:06.423261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:18.146081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667479+0000","last_change":"2023-10-06T22:12:43.696559+0000","last_active":"2023-10-06T22:14:05.667479+0000","last_peered":"2023-10-06T22:14:05.667479+0000","last_clean":"2023-10-06T22:14:05.667479+0000","last_became_active":"2023-10-06T22:12:43.696252+0000","last_became_peered":"2023-10-06T22:12:43.696252+0000","last_unstale":"2023-10-06T22:14:05.667479+0000","last_undegraded":"2023-10-06T22:14:05.667479+0000","last_fullsized":"2023-10-06T22:14:05.667479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:31:36.097000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722785+0000","last_change":"2023-10-06T22:12:42.694299+0000","last_active":"2023-10-06T22:13:05.722785+0000","last_peered":"2023-10-06T22:13:05.722785+0000","last_clean":"2023-10-06T22:13:05.722785+0000","last_became_active":"2023-10-06T22:12:42.693820+0000","last_became_peered":"2023-10-06T22:12:42.693820+0000","last_unstale":"2023-10-06T22:13:05.722785+0000","last_undegraded":"2023-10-06T22:13:05.722785+0000","last_fullsized":"2023-10-06T22:13:05.722785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:13.837379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672816+0000","last_change":"2023-10-06T22:12:43.695234+0000","last_active":"2023-10-06T22:14:05.672816+0000","last_peered":"2023-10-06T22:14:05.672816+0000","last_clean":"2023-10-06T22:14:05.672816+0000","last_became_active":"2023-10-06T22:12:43.694889+0000","last_became_peered":"2023-10-06T22:12:43.694889+0000","last_unstale":"2023-10-06T22:14:05.672816+0000","last_undegraded":"2023-10-06T22:14:05.672816+0000","last_fullsized":"2023-10-06T22:14:05.672816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:51.039264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808525+0000","last_change":"2023-10-06T22:12:42.695438+0000","last_active":"2023-10-06T22:14:27.808525+0000","last_peered":"2023-10-06T22:14:27.808525+0000","last_clean":"2023-10-06T22:14:27.808525+0000","last_became_active":"2023-10-06T22:12:42.695090+0000","last_became_peered":"2023-10-06T22:12:42.695090+0000","last_unstale":"2023-10-06T22:14:27.808525+0000","last_undegraded":"2023-10-06T22:14:27.808525+0000","last_fullsized":"2023-10-06T22:14:27.808525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:25:34.385961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663101+0000","last_change":"2023-10-06T22:12:43.697459+0000","last_active":"2023-10-06T22:14:05.663101+0000","last_peered":"2023-10-06T22:14:05.663101+0000","last_clean":"2023-10-06T22:14:05.663101+0000","last_became_active":"2023-10-06T22:12:43.696405+0000","last_became_peered":"2023-10-06T22:12:43.696405+0000","last_unstale":"2023-10-06T22:14:05.663101+0000","last_undegraded":"2023-10-06T22:14:05.663101+0000","last_fullsized":"2023-10-06T22:14:05.663101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:30.928083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727798+0000","last_change":"2023-10-06T22:12:42.692151+0000","last_active":"2023-10-06T22:13:05.727798+0000","last_peered":"2023-10-06T22:13:05.727798+0000","last_clean":"2023-10-06T22:13:05.727798+0000","last_became_active":"2023-10-06T22:12:42.691780+0000","last_became_peered":"2023-10-06T22:12:42.691780+0000","last_unstale":"2023-10-06T22:13:05.727798+0000","last_undegraded":"2023-10-06T22:13:05.727798+0000","last_fullsized":"2023-10-06T22:13:05.727798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:49:35.697733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679194+0000","last_change":"2023-10-06T22:12:43.701754+0000","last_active":"2023-10-06T22:14:05.679194+0000","last_peered":"2023-10-06T22:14:05.679194+0000","last_clean":"2023-10-06T22:14:05.679194+0000","last_became_active":"2023-10-06T22:12:43.700881+0000","last_became_peered":"2023-10-06T22:12:43.700881+0000","last_unstale":"2023-10-06T22:14:05.679194+0000","last_undegraded":"2023-10-06T22:14:05.679194+0000","last_fullsized":"2023-10-06T22:14:05.679194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:35:21.988307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723824+0000","last_change":"2023-10-06T22:12:42.699751+0000","last_active":"2023-10-06T22:13:05.723824+0000","last_peered":"2023-10-06T22:13:05.723824+0000","last_clean":"2023-10-06T22:13:05.723824+0000","last_became_active":"2023-10-06T22:12:42.699321+0000","last_became_peered":"2023-10-06T22:12:42.699321+0000","last_unstale":"2023-10-06T22:13:05.723824+0000","last_undegraded":"2023-10-06T22:13:05.723824+0000","last_fullsized":"2023-10-06T22:13:05.723824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:35:25.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.660837+0000","last_change":"2023-10-06T22:12:43.702426+0000","last_active":"2023-10-06T22:14:05.660837+0000","last_peered":"2023-10-06T22:14:05.660837+0000","last_clean":"2023-10-06T22:14:05.660837+0000","last_became_active":"2023-10-06T22:12:43.702101+0000","last_became_peered":"2023-10-06T22:12:43.702101+0000","last_unstale":"2023-10-06T22:14:05.660837+0000","last_undegraded":"2023-10-06T22:14:05.660837+0000","last_fullsized":"2023-10-06T22:14:05.660837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:31:09.133941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":252,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808520+0000","last_change":"2023-10-06T22:12:42.688815+0000","last_active":"2023-10-06T22:14:27.808520+0000","last_peered":"2023-10-06T22:14:27.808520+0000","last_clean":"2023-10-06T22:14:27.808520+0000","last_became_active":"2023-10-06T22:12:42.688447+0000","last_became_peered":"2023-10-06T22:12:42.688447+0000","last_unstale":"2023-10-06T22:14:27.808520+0000","last_undegraded":"2023-10-06T22:14:27.808520+0000","last_fullsized":"2023-10-06T22:14:27.808520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:26:25.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58073,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":204,"num_read_kb":159,"num_write":35,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666683+0000","last_change":"2023-10-06T22:12:43.698406+0000","last_active":"2023-10-06T22:14:05.666683+0000","last_peered":"2023-10-06T22:14:05.666683+0000","last_clean":"2023-10-06T22:14:05.666683+0000","last_became_active":"2023-10-06T22:12:43.697204+0000","last_became_peered":"2023-10-06T22:12:43.697204+0000","last_unstale":"2023-10-06T22:14:05.666683+0000","last_undegraded":"2023-10-06T22:14:05.666683+0000","last_fullsized":"2023-10-06T22:14:05.666683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:59.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727727+0000","last_change":"2023-10-06T22:12:42.691007+0000","last_active":"2023-10-06T22:13:05.727727+0000","last_peered":"2023-10-06T22:13:05.727727+0000","last_clean":"2023-10-06T22:13:05.727727+0000","last_became_active":"2023-10-06T22:12:42.690693+0000","last_became_peered":"2023-10-06T22:12:42.690693+0000","last_unstale":"2023-10-06T22:13:05.727727+0000","last_undegraded":"2023-10-06T22:13:05.727727+0000","last_fullsized":"2023-10-06T22:13:05.727727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:47.670091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671518+0000","last_change":"2023-10-06T22:12:43.705131+0000","last_active":"2023-10-06T22:14:05.671518+0000","last_peered":"2023-10-06T22:14:05.671518+0000","last_clean":"2023-10-06T22:14:05.671518+0000","last_became_active":"2023-10-06T22:12:43.704469+0000","last_became_peered":"2023-10-06T22:12:43.704469+0000","last_unstale":"2023-10-06T22:14:05.671518+0000","last_undegraded":"2023-10-06T22:14:05.671518+0000","last_fullsized":"2023-10-06T22:14:05.671518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:36:39.392274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723093+0000","last_change":"2023-10-06T22:12:42.698392+0000","last_active":"2023-10-06T22:13:05.723093+0000","last_peered":"2023-10-06T22:13:05.723093+0000","last_clean":"2023-10-06T22:13:05.723093+0000","last_became_active":"2023-10-06T22:12:42.698019+0000","last_became_peered":"2023-10-06T22:12:42.698019+0000","last_unstale":"2023-10-06T22:13:05.723093+0000","last_undegraded":"2023-10-06T22:13:05.723093+0000","last_fullsized":"2023-10-06T22:13:05.723093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:00.345300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677485+0000","last_change":"2023-10-06T22:12:43.693238+0000","last_active":"2023-10-06T22:14:05.677485+0000","last_peered":"2023-10-06T22:14:05.677485+0000","last_clean":"2023-10-06T22:14:05.677485+0000","last_became_active":"2023-10-06T22:12:43.692948+0000","last_became_peered":"2023-10-06T22:12:43.692948+0000","last_unstale":"2023-10-06T22:14:05.677485+0000","last_undegraded":"2023-10-06T22:14:05.677485+0000","last_fullsized":"2023-10-06T22:14:05.677485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:24:30.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.816392+0000","last_change":"2023-10-06T22:12:42.693213+0000","last_active":"2023-10-06T22:14:26.816392+0000","last_peered":"2023-10-06T22:14:26.816392+0000","last_clean":"2023-10-06T22:14:26.816392+0000","last_became_active":"2023-10-06T22:12:42.692894+0000","last_became_peered":"2023-10-06T22:12:42.692894+0000","last_unstale":"2023-10-06T22:14:26.816392+0000","last_undegraded":"2023-10-06T22:14:26.816392+0000","last_fullsized":"2023-10-06T22:14:26.816392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:40:21.159102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669294+0000","last_change":"2023-10-06T22:12:43.695527+0000","last_active":"2023-10-06T22:14:05.669294+0000","last_peered":"2023-10-06T22:14:05.669294+0000","last_clean":"2023-10-06T22:14:05.669294+0000","last_became_active":"2023-10-06T22:12:43.694364+0000","last_became_peered":"2023-10-06T22:12:43.694364+0000","last_unstale":"2023-10-06T22:14:05.669294+0000","last_undegraded":"2023-10-06T22:14:05.669294+0000","last_fullsized":"2023-10-06T22:14:05.669294+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723938+0000","last_change":"2023-10-06T22:12:42.697854+0000","last_active":"2023-10-06T22:13:05.723938+0000","last_peered":"2023-10-06T22:13:05.723938+0000","last_clean":"2023-10-06T22:13:05.723938+0000","last_became_active":"2023-10-06T22:12:42.697542+0000","last_became_peered":"2023-10-06T22:12:42.697542+0000","last_unstale":"2023-10-06T22:13:05.723938+0000","last_undegraded":"2023-10-06T22:13:05.723938+0000","last_fullsized":"2023-10-06T22:13:05.723938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:12:11.181398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665099+0000","last_change":"2023-10-06T22:12:43.698070+0000","last_active":"2023-10-06T22:14:05.665099+0000","last_peered":"2023-10-06T22:14:05.665099+0000","last_clean":"2023-10-06T22:14:05.665099+0000","last_became_active":"2023-10-06T22:12:43.697682+0000","last_became_peered":"2023-10-06T22:12:43.697682+0000","last_unstale":"2023-10-06T22:14:05.665099+0000","last_undegraded":"2023-10-06T22:14:05.665099+0000","last_fullsized":"2023-10-06T22:14:05.665099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:43.808866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724573+0000","last_change":"2023-10-06T22:12:42.691691+0000","last_active":"2023-10-06T22:13:05.724573+0000","last_peered":"2023-10-06T22:13:05.724573+0000","last_clean":"2023-10-06T22:13:05.724573+0000","last_became_active":"2023-10-06T22:12:42.691001+0000","last_became_peered":"2023-10-06T22:12:42.691001+0000","last_unstale":"2023-10-06T22:13:05.724573+0000","last_undegraded":"2023-10-06T22:13:05.724573+0000","last_fullsized":"2023-10-06T22:13:05.724573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:11:18.559570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.660998+0000","last_change":"2023-10-06T22:13:06.927620+0000","last_active":"2023-10-06T22:14:05.660998+0000","last_peered":"2023-10-06T22:14:05.660998+0000","last_clean":"2023-10-06T22:14:05.660998+0000","last_became_active":"2023-10-06T22:13:06.927206+0000","last_became_peered":"2023-10-06T22:13:06.927206+0000","last_unstale":"2023-10-06T22:14:05.660998+0000","last_undegraded":"2023-10-06T22:14:05.660998+0000","last_fullsized":"2023-10-06T22:14:05.660998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:50:30.780174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.726783+0000","last_change":"2023-10-06T22:13:06.726783+0000","last_active":"2023-10-06T22:13:06.726783+0000","last_peered":"2023-10-06T22:13:06.726783+0000","last_clean":"2023-10-06T22:13:06.726783+0000","last_became_active":"2023-10-06T22:13:06.726342+0000","last_became_peered":"2023-10-06T22:13:06.726342+0000","last_unstale":"2023-10-06T22:13:06.726783+0000","last_undegraded":"2023-10-06T22:13:06.726783+0000","last_fullsized":"2023-10-06T22:13:06.726783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:42.359819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672141+0000","last_change":"2023-10-06T22:12:43.702870+0000","last_active":"2023-10-06T22:14:05.672141+0000","last_peered":"2023-10-06T22:14:05.672141+0000","last_clean":"2023-10-06T22:14:05.672141+0000","last_became_active":"2023-10-06T22:12:43.702564+0000","last_became_peered":"2023-10-06T22:12:43.702564+0000","last_unstale":"2023-10-06T22:14:05.672141+0000","last_undegraded":"2023-10-06T22:14:05.672141+0000","last_fullsized":"2023-10-06T22:14:05.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:23:32.238268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.836399+0000","last_change":"2023-10-06T22:12:42.696603+0000","last_active":"2023-10-06T22:14:26.836399+0000","last_peered":"2023-10-06T22:14:26.836399+0000","last_clean":"2023-10-06T22:14:26.836399+0000","last_became_active":"2023-10-06T22:12:42.696313+0000","last_became_peered":"2023-10-06T22:12:42.696313+0000","last_unstale":"2023-10-06T22:14:26.836399+0000","last_undegraded":"2023-10-06T22:14:26.836399+0000","last_fullsized":"2023-10-06T22:14:26.836399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:15.719749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675740+0000","last_change":"2023-10-06T22:12:43.703356+0000","last_active":"2023-10-06T22:14:05.675740+0000","last_peered":"2023-10-06T22:14:05.675740+0000","last_clean":"2023-10-06T22:14:05.675740+0000","last_became_active":"2023-10-06T22:12:43.702670+0000","last_became_peered":"2023-10-06T22:12:43.702670+0000","last_unstale":"2023-10-06T22:14:05.675740+0000","last_undegraded":"2023-10-06T22:14:05.675740+0000","last_fullsized":"2023-10-06T22:14:05.675740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:15:05.681491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665212+0000","last_change":"2023-10-06T22:12:43.699083+0000","last_active":"2023-10-06T22:14:05.665212+0000","last_peered":"2023-10-06T22:14:05.665212+0000","last_clean":"2023-10-06T22:14:05.665212+0000","last_became_active":"2023-10-06T22:12:43.698826+0000","last_became_peered":"2023-10-06T22:12:43.698826+0000","last_unstale":"2023-10-06T22:14:05.665212+0000","last_undegraded":"2023-10-06T22:14:05.665212+0000","last_fullsized":"2023-10-06T22:14:05.665212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:04.225060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'7","reported_seq":120,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.515215+0000","last_change":"2023-10-06T22:12:42.699873+0000","last_active":"2023-10-06T22:14:26.515215+0000","last_peered":"2023-10-06T22:14:26.515215+0000","last_clean":"2023-10-06T22:14:26.515215+0000","last_became_active":"2023-10-06T22:12:42.699476+0000","last_became_peered":"2023-10-06T22:12:42.699476+0000","last_unstale":"2023-10-06T22:14:26.515215+0000","last_undegraded":"2023-10-06T22:14:26.515215+0000","last_fullsized":"2023-10-06T22:14:26.515215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:13:27.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.658075+0000","last_change":"2023-10-06T22:12:43.697720+0000","last_active":"2023-10-06T22:14:05.658075+0000","last_peered":"2023-10-06T22:14:05.658075+0000","last_clean":"2023-10-06T22:14:05.658075+0000","last_became_active":"2023-10-06T22:12:43.697430+0000","last_became_peered":"2023-10-06T22:12:43.697430+0000","last_unstale":"2023-10-06T22:14:05.658075+0000","last_undegraded":"2023-10-06T22:14:05.658075+0000","last_fullsized":"2023-10-06T22:14:05.658075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:29.633012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722760+0000","last_change":"2023-10-06T22:12:42.690781+0000","last_active":"2023-10-06T22:13:05.722760+0000","last_peered":"2023-10-06T22:13:05.722760+0000","last_clean":"2023-10-06T22:13:05.722760+0000","last_became_active":"2023-10-06T22:12:42.690274+0000","last_became_peered":"2023-10-06T22:12:42.690274+0000","last_unstale":"2023-10-06T22:13:05.722760+0000","last_undegraded":"2023-10-06T22:13:05.722760+0000","last_fullsized":"2023-10-06T22:13:05.722760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:26.653182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.724529+0000","last_change":"2023-10-06T22:11:36.409669+0000","last_active":"2023-10-06T22:13:06.724529+0000","last_peered":"2023-10-06T22:13:06.724529+0000","last_clean":"2023-10-06T22:13:06.724529+0000","last_became_active":"2023-10-06T22:11:36.409197+0000","last_became_peered":"2023-10-06T22:11:36.409197+0000","last_unstale":"2023-10-06T22:13:06.724529+0000","last_undegraded":"2023-10-06T22:13:06.724529+0000","last_fullsized":"2023-10-06T22:13:06.724529+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_clean_scrub_stamp":"2023-10-06T22:08:24.598743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:50:55.982571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672700+0000","last_change":"2023-10-06T22:12:43.698617+0000","last_active":"2023-10-06T22:14:05.672700+0000","last_peered":"2023-10-06T22:14:05.672700+0000","last_clean":"2023-10-06T22:14:05.672700+0000","last_became_active":"2023-10-06T22:12:43.697942+0000","last_became_peered":"2023-10-06T22:12:43.697942+0000","last_unstale":"2023-10-06T22:14:05.672700+0000","last_undegraded":"2023-10-06T22:14:05.672700+0000","last_fullsized":"2023-10-06T22:14:05.672700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:38.075730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727545+0000","last_change":"2023-10-06T22:12:42.695553+0000","last_active":"2023-10-06T22:13:05.727545+0000","last_peered":"2023-10-06T22:13:05.727545+0000","last_clean":"2023-10-06T22:13:05.727545+0000","last_became_active":"2023-10-06T22:12:42.695209+0000","last_became_peered":"2023-10-06T22:12:42.695209+0000","last_unstale":"2023-10-06T22:13:05.727545+0000","last_undegraded":"2023-10-06T22:13:05.727545+0000","last_fullsized":"2023-10-06T22:13:05.727545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:10.387710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724270+0000","last_change":"2023-10-06T22:12:42.699606+0000","last_active":"2023-10-06T22:13:05.724270+0000","last_peered":"2023-10-06T22:13:05.724270+0000","last_clean":"2023-10-06T22:13:05.724270+0000","last_became_active":"2023-10-06T22:12:42.699222+0000","last_became_peered":"2023-10-06T22:12:42.699222+0000","last_unstale":"2023-10-06T22:13:05.724270+0000","last_undegraded":"2023-10-06T22:13:05.724270+0000","last_fullsized":"2023-10-06T22:13:05.724270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:52.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675984+0000","last_change":"2023-10-06T22:12:43.697783+0000","last_active":"2023-10-06T22:14:05.675984+0000","last_peered":"2023-10-06T22:14:05.675984+0000","last_clean":"2023-10-06T22:14:05.675984+0000","last_became_active":"2023-10-06T22:12:43.697388+0000","last_became_peered":"2023-10-06T22:12:43.697388+0000","last_unstale":"2023-10-06T22:14:05.675984+0000","last_undegraded":"2023-10-06T22:14:05.675984+0000","last_fullsized":"2023-10-06T22:14:05.675984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:50.113458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.721888+0000","last_change":"2023-10-06T22:12:42.694180+0000","last_active":"2023-10-06T22:13:06.721888+0000","last_peered":"2023-10-06T22:13:06.721888+0000","last_clean":"2023-10-06T22:13:06.721888+0000","last_became_active":"2023-10-06T22:12:42.693861+0000","last_became_peered":"2023-10-06T22:12:42.693861+0000","last_unstale":"2023-10-06T22:13:06.721888+0000","last_undegraded":"2023-10-06T22:13:06.721888+0000","last_fullsized":"2023-10-06T22:13:06.721888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:28:02.511117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664374+0000","last_change":"2023-10-06T22:12:43.699118+0000","last_active":"2023-10-06T22:14:05.664374+0000","last_peered":"2023-10-06T22:14:05.664374+0000","last_clean":"2023-10-06T22:14:05.664374+0000","last_became_active":"2023-10-06T22:12:43.698540+0000","last_became_peered":"2023-10-06T22:12:43.698540+0000","last_unstale":"2023-10-06T22:14:05.664374+0000","last_undegraded":"2023-10-06T22:14:05.664374+0000","last_fullsized":"2023-10-06T22:14:05.664374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:31.192762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722961+0000","last_change":"2023-10-06T22:12:42.690617+0000","last_active":"2023-10-06T22:13:05.722961+0000","last_peered":"2023-10-06T22:13:05.722961+0000","last_clean":"2023-10-06T22:13:05.722961+0000","last_became_active":"2023-10-06T22:12:42.690238+0000","last_became_peered":"2023-10-06T22:12:42.690238+0000","last_unstale":"2023-10-06T22:13:05.722961+0000","last_undegraded":"2023-10-06T22:13:05.722961+0000","last_fullsized":"2023-10-06T22:13:05.722961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:09.819570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676115+0000","last_change":"2023-10-06T22:12:43.699064+0000","last_active":"2023-10-06T22:14:05.676115+0000","last_peered":"2023-10-06T22:14:05.676115+0000","last_clean":"2023-10-06T22:14:05.676115+0000","last_became_active":"2023-10-06T22:12:43.698727+0000","last_became_peered":"2023-10-06T22:12:43.698727+0000","last_unstale":"2023-10-06T22:14:05.676115+0000","last_undegraded":"2023-10-06T22:14:05.676115+0000","last_fullsized":"2023-10-06T22:14:05.676115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:48:14.493376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738711+0000","last_change":"2023-10-06T22:12:42.693779+0000","last_active":"2023-10-06T22:13:05.738711+0000","last_peered":"2023-10-06T22:13:05.738711+0000","last_clean":"2023-10-06T22:13:05.738711+0000","last_became_active":"2023-10-06T22:12:42.693257+0000","last_became_peered":"2023-10-06T22:12:42.693257+0000","last_unstale":"2023-10-06T22:13:05.738711+0000","last_undegraded":"2023-10-06T22:13:05.738711+0000","last_fullsized":"2023-10-06T22:13:05.738711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:56:48.945078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.631287+0000","last_change":"2023-10-06T22:12:43.700620+0000","last_active":"2023-10-06T22:14:05.631287+0000","last_peered":"2023-10-06T22:14:05.631287+0000","last_clean":"2023-10-06T22:14:05.631287+0000","last_became_active":"2023-10-06T22:12:43.700093+0000","last_became_peered":"2023-10-06T22:12:43.700093+0000","last_unstale":"2023-10-06T22:14:05.631287+0000","last_undegraded":"2023-10-06T22:14:05.631287+0000","last_fullsized":"2023-10-06T22:14:05.631287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:29.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751425+0000","last_change":"2023-10-06T22:12:42.689640+0000","last_active":"2023-10-06T22:13:05.751425+0000","last_peered":"2023-10-06T22:13:05.751425+0000","last_clean":"2023-10-06T22:13:05.751425+0000","last_became_active":"2023-10-06T22:12:42.688584+0000","last_became_peered":"2023-10-06T22:12:42.688584+0000","last_unstale":"2023-10-06T22:13:05.751425+0000","last_undegraded":"2023-10-06T22:13:05.751425+0000","last_fullsized":"2023-10-06T22:13:05.751425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:43.066396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676684+0000","last_change":"2023-10-06T22:12:43.700084+0000","last_active":"2023-10-06T22:14:05.676684+0000","last_peered":"2023-10-06T22:14:05.676684+0000","last_clean":"2023-10-06T22:14:05.676684+0000","last_became_active":"2023-10-06T22:12:43.699787+0000","last_became_peered":"2023-10-06T22:12:43.699787+0000","last_unstale":"2023-10-06T22:14:05.676684+0000","last_undegraded":"2023-10-06T22:14:05.676684+0000","last_fullsized":"2023-10-06T22:14:05.676684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:07.219800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727627+0000","last_change":"2023-10-06T22:12:42.688750+0000","last_active":"2023-10-06T22:13:05.727627+0000","last_peered":"2023-10-06T22:13:05.727627+0000","last_clean":"2023-10-06T22:13:05.727627+0000","last_became_active":"2023-10-06T22:12:42.688387+0000","last_became_peered":"2023-10-06T22:12:42.688387+0000","last_unstale":"2023-10-06T22:13:05.727627+0000","last_undegraded":"2023-10-06T22:13:05.727627+0000","last_fullsized":"2023-10-06T22:13:05.727627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:32.980631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676460+0000","last_change":"2023-10-06T22:12:43.706741+0000","last_active":"2023-10-06T22:14:05.676460+0000","last_peered":"2023-10-06T22:14:05.676460+0000","last_clean":"2023-10-06T22:14:05.676460+0000","last_became_active":"2023-10-06T22:12:43.706416+0000","last_became_peered":"2023-10-06T22:12:43.706416+0000","last_unstale":"2023-10-06T22:14:05.676460+0000","last_undegraded":"2023-10-06T22:14:05.676460+0000","last_fullsized":"2023-10-06T22:14:05.676460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:55:16.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724994+0000","last_change":"2023-10-06T22:12:42.692850+0000","last_active":"2023-10-06T22:13:05.724994+0000","last_peered":"2023-10-06T22:13:05.724994+0000","last_clean":"2023-10-06T22:13:05.724994+0000","last_became_active":"2023-10-06T22:12:42.692287+0000","last_became_peered":"2023-10-06T22:12:42.692287+0000","last_unstale":"2023-10-06T22:13:05.724994+0000","last_undegraded":"2023-10-06T22:13:05.724994+0000","last_fullsized":"2023-10-06T22:13:05.724994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:37:59.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667276+0000","last_change":"2023-10-06T22:12:43.700667+0000","last_active":"2023-10-06T22:14:05.667276+0000","last_peered":"2023-10-06T22:14:05.667276+0000","last_clean":"2023-10-06T22:14:05.667276+0000","last_became_active":"2023-10-06T22:12:43.698758+0000","last_became_peered":"2023-10-06T22:12:43.698758+0000","last_unstale":"2023-10-06T22:14:05.667276+0000","last_undegraded":"2023-10-06T22:14:05.667276+0000","last_fullsized":"2023-10-06T22:14:05.667276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:49:53.091578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.809273+0000","last_change":"2023-10-06T22:12:42.695964+0000","last_active":"2023-10-06T22:14:27.809273+0000","last_peered":"2023-10-06T22:14:27.809273+0000","last_clean":"2023-10-06T22:14:27.809273+0000","last_became_active":"2023-10-06T22:12:42.695431+0000","last_became_peered":"2023-10-06T22:12:42.695431+0000","last_unstale":"2023-10-06T22:14:27.809273+0000","last_undegraded":"2023-10-06T22:14:27.809273+0000","last_fullsized":"2023-10-06T22:14:27.809273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:21:42.761421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667361+0000","last_change":"2023-10-06T22:12:43.698088+0000","last_active":"2023-10-06T22:14:05.667361+0000","last_peered":"2023-10-06T22:14:05.667361+0000","last_clean":"2023-10-06T22:14:05.667361+0000","last_became_active":"2023-10-06T22:12:43.697765+0000","last_became_peered":"2023-10-06T22:12:43.697765+0000","last_unstale":"2023-10-06T22:14:05.667361+0000","last_undegraded":"2023-10-06T22:14:05.667361+0000","last_fullsized":"2023-10-06T22:14:05.667361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:26:22.958348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723709+0000","last_change":"2023-10-06T22:12:42.698586+0000","last_active":"2023-10-06T22:13:05.723709+0000","last_peered":"2023-10-06T22:13:05.723709+0000","last_clean":"2023-10-06T22:13:05.723709+0000","last_became_active":"2023-10-06T22:12:42.698198+0000","last_became_peered":"2023-10-06T22:12:42.698198+0000","last_unstale":"2023-10-06T22:13:05.723709+0000","last_undegraded":"2023-10-06T22:13:05.723709+0000","last_fullsized":"2023-10-06T22:13:05.723709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:03:43.898987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671926+0000","last_change":"2023-10-06T22:12:43.698615+0000","last_active":"2023-10-06T22:14:05.671926+0000","last_peered":"2023-10-06T22:14:05.671926+0000","last_clean":"2023-10-06T22:14:05.671926+0000","last_became_active":"2023-10-06T22:12:43.698302+0000","last_became_peered":"2023-10-06T22:12:43.698302+0000","last_unstale":"2023-10-06T22:14:05.671926+0000","last_undegraded":"2023-10-06T22:14:05.671926+0000","last_fullsized":"2023-10-06T22:14:05.671926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:35.973119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422877+0000","last_change":"2023-10-06T22:12:42.700325+0000","last_active":"2023-10-06T22:13:06.422877+0000","last_peered":"2023-10-06T22:13:06.422877+0000","last_clean":"2023-10-06T22:13:06.422877+0000","last_became_active":"2023-10-06T22:12:42.699426+0000","last_became_peered":"2023-10-06T22:12:42.699426+0000","last_unstale":"2023-10-06T22:13:06.422877+0000","last_undegraded":"2023-10-06T22:13:06.422877+0000","last_fullsized":"2023-10-06T22:13:06.422877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:09.649737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667113+0000","last_change":"2023-10-06T22:12:43.698620+0000","last_active":"2023-10-06T22:14:05.667113+0000","last_peered":"2023-10-06T22:14:05.667113+0000","last_clean":"2023-10-06T22:14:05.667113+0000","last_became_active":"2023-10-06T22:12:43.698335+0000","last_became_peered":"2023-10-06T22:12:43.698335+0000","last_unstale":"2023-10-06T22:14:05.667113+0000","last_undegraded":"2023-10-06T22:14:05.667113+0000","last_fullsized":"2023-10-06T22:14:05.667113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:49.222848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669528+0000","last_change":"2023-10-06T22:12:43.700477+0000","last_active":"2023-10-06T22:14:05.669528+0000","last_peered":"2023-10-06T22:14:05.669528+0000","last_clean":"2023-10-06T22:14:05.669528+0000","last_became_active":"2023-10-06T22:12:43.700177+0000","last_became_peered":"2023-10-06T22:12:43.700177+0000","last_unstale":"2023-10-06T22:14:05.669528+0000","last_undegraded":"2023-10-06T22:14:05.669528+0000","last_fullsized":"2023-10-06T22:14:05.669528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:11.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664348+0000","last_change":"2023-10-06T22:12:43.701792+0000","last_active":"2023-10-06T22:14:05.664348+0000","last_peered":"2023-10-06T22:14:05.664348+0000","last_clean":"2023-10-06T22:14:05.664348+0000","last_became_active":"2023-10-06T22:12:43.701316+0000","last_became_peered":"2023-10-06T22:12:43.701316+0000","last_unstale":"2023-10-06T22:14:05.664348+0000","last_undegraded":"2023-10-06T22:14:05.664348+0000","last_fullsized":"2023-10-06T22:14:05.664348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:03.085219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677454+0000","last_change":"2023-10-06T22:12:43.707871+0000","last_active":"2023-10-06T22:14:05.677454+0000","last_peered":"2023-10-06T22:14:05.677454+0000","last_clean":"2023-10-06T22:14:05.677454+0000","last_became_active":"2023-10-06T22:12:43.707659+0000","last_became_peered":"2023-10-06T22:12:43.707659+0000","last_unstale":"2023-10-06T22:14:05.677454+0000","last_undegraded":"2023-10-06T22:14:05.677454+0000","last_fullsized":"2023-10-06T22:14:05.677454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:43.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.673925+0000","last_change":"2023-10-06T22:12:43.697701+0000","last_active":"2023-10-06T22:14:05.673925+0000","last_peered":"2023-10-06T22:14:05.673925+0000","last_clean":"2023-10-06T22:14:05.673925+0000","last_became_active":"2023-10-06T22:12:43.697361+0000","last_became_peered":"2023-10-06T22:12:43.697361+0000","last_unstale":"2023-10-06T22:14:05.673925+0000","last_undegraded":"2023-10-06T22:14:05.673925+0000","last_fullsized":"2023-10-06T22:14:05.673925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:14:53.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679653+0000","last_change":"2023-10-06T22:12:43.700058+0000","last_active":"2023-10-06T22:14:05.679653+0000","last_peered":"2023-10-06T22:14:05.679653+0000","last_clean":"2023-10-06T22:14:05.679653+0000","last_became_active":"2023-10-06T22:12:43.697893+0000","last_became_peered":"2023-10-06T22:12:43.697893+0000","last_unstale":"2023-10-06T22:14:05.679653+0000","last_undegraded":"2023-10-06T22:14:05.679653+0000","last_fullsized":"2023-10-06T22:14:05.679653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:01:37.518096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664805+0000","last_change":"2023-10-06T22:13:06.729307+0000","last_active":"2023-10-06T22:14:05.664805+0000","last_peered":"2023-10-06T22:14:05.664805+0000","last_clean":"2023-10-06T22:14:05.664805+0000","last_became_active":"2023-10-06T22:13:06.728877+0000","last_became_peered":"2023-10-06T22:13:06.728877+0000","last_unstale":"2023-10-06T22:14:05.664805+0000","last_undegraded":"2023-10-06T22:14:05.664805+0000","last_fullsized":"2023-10-06T22:14:05.664805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:17.629046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666149+0000","last_change":"2023-10-06T22:12:43.700668+0000","last_active":"2023-10-06T22:14:05.666149+0000","last_peered":"2023-10-06T22:14:05.666149+0000","last_clean":"2023-10-06T22:14:05.666149+0000","last_became_active":"2023-10-06T22:12:43.700387+0000","last_became_peered":"2023-10-06T22:12:43.700387+0000","last_unstale":"2023-10-06T22:14:05.666149+0000","last_undegraded":"2023-10-06T22:14:05.666149+0000","last_fullsized":"2023-10-06T22:14:05.666149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:36:00.053496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.662653+0000","last_change":"2023-10-06T22:12:43.702658+0000","last_active":"2023-10-06T22:14:05.662653+0000","last_peered":"2023-10-06T22:14:05.662653+0000","last_clean":"2023-10-06T22:14:05.662653+0000","last_became_active":"2023-10-06T22:12:43.702264+0000","last_became_peered":"2023-10-06T22:12:43.702264+0000","last_unstale":"2023-10-06T22:14:05.662653+0000","last_undegraded":"2023-10-06T22:14:05.662653+0000","last_fullsized":"2023-10-06T22:14:05.662653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:56.472029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671729+0000","last_change":"2023-10-06T22:12:43.705718+0000","last_active":"2023-10-06T22:14:05.671729+0000","last_peered":"2023-10-06T22:14:05.671729+0000","last_clean":"2023-10-06T22:14:05.671729+0000","last_became_active":"2023-10-06T22:12:43.705479+0000","last_became_peered":"2023-10-06T22:12:43.705479+0000","last_unstale":"2023-10-06T22:14:05.671729+0000","last_undegraded":"2023-10-06T22:14:05.671729+0000","last_fullsized":"2023-10-06T22:14:05.671729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:55:36.393577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672014+0000","last_change":"2023-10-06T22:12:43.701891+0000","last_active":"2023-10-06T22:14:05.672014+0000","last_peered":"2023-10-06T22:14:05.672014+0000","last_clean":"2023-10-06T22:14:05.672014+0000","last_became_active":"2023-10-06T22:12:43.701535+0000","last_became_peered":"2023-10-06T22:12:43.701535+0000","last_unstale":"2023-10-06T22:14:05.672014+0000","last_undegraded":"2023-10-06T22:14:05.672014+0000","last_fullsized":"2023-10-06T22:14:05.672014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:17:40.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667438+0000","last_change":"2023-10-06T22:12:43.707075+0000","last_active":"2023-10-06T22:14:05.667438+0000","last_peered":"2023-10-06T22:14:05.667438+0000","last_clean":"2023-10-06T22:14:05.667438+0000","last_became_active":"2023-10-06T22:12:43.706723+0000","last_became_peered":"2023-10-06T22:12:43.706723+0000","last_unstale":"2023-10-06T22:14:05.667438+0000","last_undegraded":"2023-10-06T22:14:05.667438+0000","last_fullsized":"2023-10-06T22:14:05.667438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:18.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679749+0000","last_change":"2023-10-06T22:12:43.704656+0000","last_active":"2023-10-06T22:14:05.679749+0000","last_peered":"2023-10-06T22:14:05.679749+0000","last_clean":"2023-10-06T22:14:05.679749+0000","last_became_active":"2023-10-06T22:12:43.704312+0000","last_became_peered":"2023-10-06T22:12:43.704312+0000","last_unstale":"2023-10-06T22:14:05.679749+0000","last_undegraded":"2023-10-06T22:14:05.679749+0000","last_fullsized":"2023-10-06T22:14:05.679749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:21:37.951180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663935+0000","last_change":"2023-10-06T22:12:43.702239+0000","last_active":"2023-10-06T22:14:05.663935+0000","last_peered":"2023-10-06T22:14:05.663935+0000","last_clean":"2023-10-06T22:14:05.663935+0000","last_became_active":"2023-10-06T22:12:43.701851+0000","last_became_peered":"2023-10-06T22:12:43.701851+0000","last_unstale":"2023-10-06T22:14:05.663935+0000","last_undegraded":"2023-10-06T22:14:05.663935+0000","last_fullsized":"2023-10-06T22:14:05.663935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:40:53.661587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66404,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1014,"num_read_kb":575,"num_write":245,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449122,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":444759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72099999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.38100000000000001}]},{"osd":1,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.371,"15min":0.371},"max":{"1min":0.85299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.34200000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61899999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.48799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69799999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54500000000000004,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.68400000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.69299999999999995},"min":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.74199999999999999,"5min":1.044,"15min":1.044},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.56599999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54100000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86099999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.61199999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77100000000000002}]},{"osd":9,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.62,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":1.472,"15min":1.472},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.63500000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0109999999999999,"5min":1.151,"15min":1.151},"last":0.83399999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.68300000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82499999999999996,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.64800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85299999999999998,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.751}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1093852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59699999999999998,"5min":0.748,"15min":0.748},"last":0.44}]},{"osd":2,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":1.026,"15min":1.026},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96099999999999997,"5min":1.093,"15min":1.093},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.626},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78000000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81699999999999995,"5min":1.079,"15min":1.079},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.266,"15min":1.266},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81200000000000006},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.376},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.505}]},{"osd":8,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90200000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.751},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95199999999999996,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.81699999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.88800000000000001,"15min":0.88400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.72699999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97099999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.94899999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86099999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":444158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89400000000000002,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.747,"15min":0.77000000000000002},"min":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.036},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.83599999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96599999999999997,"5min":1.486,"15min":1.486},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91500000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.038,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72299999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.73899999999999999,"15min":0.76700000000000002},"min":{"1min":0.436,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93799999999999994,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.82799999999999996,"15min":0.871},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0629999999999999,"5min":1.548,"15min":1.548},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.871,"15min":0.91700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94699999999999995,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.73799999999999999,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89900000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.90200000000000002,"15min":0.94199999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90700000000000003,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.96099999999999997,"15min":1.0189999999999999},"min":{"1min":0.63500000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1259999999999999,"5min":1.585,"15min":1.585},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83899999999999997,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.95499999999999996}]},{"osd":8,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.96799999999999997,"15min":1.006},"min":{"1min":0.71199999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.167,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.86899999999999999,"15min":0.90100000000000002},"min":{"1min":0.63600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.02,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.194,"5min":1.268,"15min":1.268},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.997,"15min":1.018},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.878}]},{"osd":11,"last update":"Fri Oct 6 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.90900000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78800000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45600000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.46899999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89200000000000002,"5min":1.018,"15min":1.018},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.53500000000000003},"min":{"1min":0.49199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.434,"15min":0.41199999999999998},"min":{"1min":0.33100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.378},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46999999999999997,"15min":0.45800000000000002},"min":{"1min":0.39300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.52900000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.63900000000000001,"15min":0.61099999999999999},"min":{"1min":0.60199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.42699999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.64400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.57899999999999996,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.873,"5min":1.073,"15min":1.073},"last":0.745},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.65800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92500000000000004,"5min":1.502,"15min":1.502},"last":0.624}]},{"osd":9,"last update":"Fri Oct 6 22:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.68400000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.78300000000000003,"15min":0.76500000000000001},"min":{"1min":0.77100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.63800000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.754,"15min":0.72999999999999998},"min":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.72999999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.505}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.33600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.374},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44,"15min":0.436},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.627,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65700000000000003,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.875,"15min":0.875},"last":0.35299999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.51500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.438}]},{"osd":8,"last update":"Fri Oct 6 22:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.745},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0169999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.76400000000000001},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.089,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.753}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86899999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72899999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.77800000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81399999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68600000000000005,"15min":0.66300000000000003},"min":{"1min":0.53200000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92700000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93000000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86299999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60499999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.75}]},{"osd":4,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73799999999999999,"15min":0.748},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.754},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.004,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.02,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79300000000000004,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99399999999999999,"5min":1.173,"15min":1.173},"last":0.875},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83899999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97699999999999998,"5min":1.145,"15min":1.145},"last":0.88700000000000001}]},{"osd":8,"last update":"Fri Oct 6 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.59399999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.1080000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85299999999999998,"15min":0.86199999999999999},"min":{"1min":0.64900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.137,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.91800000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.90400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.92300000000000004,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1599999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93700000000000006,"15min":0.93999999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.97799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.441,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.435,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55900000000000005,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56799999999999995},"min":{"1min":0.46200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84699999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63700000000000001,"15min":0.58199999999999996},"min":{"1min":0.47899999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.754}]},{"osd":4,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.443,"15min":0.39700000000000002},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48899999999999999,"15min":0.433},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.433}]},{"osd":5,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.58699999999999997,"15min":0.53700000000000003},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.60699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.59799999999999998},"min":{"1min":0.53800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95099999999999996,"5min":1.45,"15min":1.45},"last":0.625},{"interface":"front","average":{"1min":0.753,"5min":0.65100000000000002,"15min":0.60099999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55700000000000005,"15min":0.53400000000000003},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Oct 6 22:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.56000000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.93200000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.81699999999999995,"15min":0.79000000000000004},"min":{"1min":0.66000000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.63700000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94799999999999995,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.87}]},{"osd":10,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.74199999999999999,"15min":0.72799999999999998},"min":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Oct 6 22:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.84099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":445141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60299999999999998,"15min":0.52500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.56799999999999995},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.62}]},{"osd":3,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.67200000000000004,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.59199999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.73399999999999999,"15min":0.68899999999999995},"min":{"1min":0.59299999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0669999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.97399999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.86899999999999999},"min":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.121,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.64900000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.745},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79000000000000004}]},{"osd":9,"last update":"Fri Oct 6 22:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.012,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.872,"5min":0.88600000000000001,"15min":0.89900000000000002},"min":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.048,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.877,"15min":0.89300000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93300000000000005,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.054,"5min":1.103,"15min":1.103},"last":0.82799999999999996}]},{"osd":11,"last update":"Fri Oct 6 22:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.86199999999999999,"15min":0.85699999999999998},"min":{"1min":0.67300000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0669999999999999,"5min":1.538,"15min":1.538},"last":0.876}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":503812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.125},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57199999999999995,"15min":0.51300000000000001},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.873,"15min":0.873},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.52300000000000002},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 22:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71599999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44900000000000001,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.71199999999999997,"15min":0.66900000000000004},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64500000000000002,"15min":0.61899999999999999},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78900000000000003,"5min":1.071,"15min":1.071},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":1.002,"15min":1.002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.629},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85299999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.58099999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92200000000000004,"5min":1.633,"15min":1.633},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.878,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72199999999999998}]},{"osd":10,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.747},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.83599999999999997}]},{"osd":11,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.75700000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":443439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.40100000000000002},"min":{"1min":0.34999999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.41699999999999998},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.60999999999999999},"min":{"1min":0.48999999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92500000000000004,"5min":1.069,"15min":1.069},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.61799999999999999},"min":{"1min":0.52800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.60799999999999998},"min":{"1min":0.48899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.74299999999999999,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.0109999999999999,"5min":1.635,"15min":1.635},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71199999999999997,"15min":0.629},"min":{"1min":0.501,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68200000000000005,"15min":0.64800000000000002},"min":{"1min":0.53400000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.71099999999999997},"min":{"1min":0.57999999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Oct 6 22:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4019999999999999,"5min":1.794,"15min":1.794},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.163,"5min":1.224,"15min":1.224},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76000000000000001,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0509999999999999,"15min":1.133},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.778,"5min":7.0529999999999999,"15min":7.0529999999999999},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Oct 6 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88400000000000001,"15min":0.91200000000000003},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86599999999999999,"15min":0.88400000000000001},"min":{"1min":0.49299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.448,"5min":1.748,"15min":1.748},"last":0.85599999999999998}]},{"osd":10,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.94899999999999995,"15min":0.95399999999999996},"min":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.0229999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.37},"min":{"1min":0.28999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.436},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.39000000000000001},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.61399999999999999,"15min":0.56499999999999995},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.56299999999999994},"min":{"1min":0.53400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.57999999999999996},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86599999999999999,"5min":1.365,"15min":1.365},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.48199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.879,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59299999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82199999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89300000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 22:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66900000000000004,"15min":0.626},"min":{"1min":0.45200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.68600000000000005,"15min":0.64100000000000001},"min":{"1min":0.55000000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.41799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.018,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51400000000000001}]},{"osd":8,"last update":"Fri Oct 6 22:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.89200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.877},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81299999999999994,"5min":0.877,"15min":0.877},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79400000000000004,"15min":0.78600000000000003},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.89700000000000002,"5min":1.054,"15min":1.054},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1036878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.752,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73299999999999998,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83899999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78000000000000003,"5min":2.802,"15min":2.802},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.308,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.67700000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.42199999999999999},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64600000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.499},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80900000000000005,"5min":1.214,"15min":1.214},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79700000000000004,"5min":2.661,"15min":2.661},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.80200000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.629},"min":{"1min":0.50900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82399999999999995,"5min":1.274,"15min":1.274},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.60299999999999998},"min":{"1min":0.59299999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96599999999999997,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.78800000000000003}]},{"osd":8,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85699999999999998,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.87,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63500000000000001}]},{"osd":9,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80000000000000004,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77700000000000002,"15min":0.77800000000000002},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.89500000000000002,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.65400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":2.774,"15min":2.774},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72499999999999998,"15min":0.71099999999999997},"min":{"1min":0.63800000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94199999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T22:14:31.740 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/2152486320 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88108de0 msgr2=0x7f2b8810b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/2152486320 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88108de0 0x7f2b8810b250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000b0a0 tx=0x7f2b8000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/2152486320 shutdown_connections 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/2152486320 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b88108de0 0x7f2b8810b250 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/2152486320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b88074e90 0x7f2b880732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/2152486320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880738b0 0x7f2b88073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/2152486320 >> 172.21.15.57:0/2152486320 conn(0x7f2b880fc460 msgr2=0x7f2b880fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/2152486320 shutdown_connections 2023-10-06T22:14:31.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.975+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/2152486320 wait complete. 2023-10-06T22:14:31.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.976+0000 7f2b8c9ff700 1 Processor -- start 2023-10-06T22:14:31.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.976+0000 7f2b8c9ff700 1 -- start start 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.976+0000 7f2b8c9ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b880738b0 0x7f2b88198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8c9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b88074e90 0x7f2b881993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8c9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b88108de0 0x7f2b881a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8c9ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b881a09e0 con 0x7f2b880738b0 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8c9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b881a0b20 con 0x7f2b88108de0 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8c9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b881a0cb0 con 0x7f2b88074e90 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b880738b0 0x7f2b88198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b880738b0 0x7f2b88198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33672/0 (socket says 172.21.15.57:33672) 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 -- 172.21.15.57:0/1740255175 learned_addr learned my addr 172.21.15.57:0/1740255175 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b86d9d700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b88108de0 0x7f2b881a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b85d9b700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b88074e90 0x7f2b881993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 -- 172.21.15.57:0/1740255175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b88074e90 msgr2=0x7f2b881993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b88074e90 0x7f2b881993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 -- 172.21.15.57:0/1740255175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b88108de0 msgr2=0x7f2b881a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b88108de0 0x7f2b881a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b8659c700 1 -- 172.21.15.57:0/1740255175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b80009d00 con 0x7f2b880738b0 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b86d9d700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b88108de0 0x7f2b881a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.977+0000 7f2b85d9b700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b88074e90 0x7f2b881993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.978+0000 7f2b8659c700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b880738b0 0x7f2b88198e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e9a0 tx=0x7f2b7800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.978+0000 7f2b777fe700 1 -- 172.21.15.57:0/1740255175 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78009790 con 0x7f2b880738b0 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.978+0000 7f2b777fe700 1 -- 172.21.15.57:0/1740255175 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b7800cbd0 con 0x7f2b880738b0 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.978+0000 7f2b777fe700 1 -- 172.21.15.57:0/1740255175 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78018680 con 0x7f2b880738b0 2023-10-06T22:14:31.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.978+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b881a0f90 con 0x7f2b880738b0 2023-10-06T22:14:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.978+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b881a13b0 con 0x7f2b880738b0 2023-10-06T22:14:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.979+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b881074e0 con 0x7f2b880738b0 2023-10-06T22:14:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.981+0000 7f2b777fe700 1 -- 172.21.15.57:0/1740255175 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2b7800cd40 con 0x7f2b880738b0 2023-10-06T22:14:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.982+0000 7f2b777fe700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2b700a1550 0x7f2b700a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.983+0000 7f2b777fe700 1 -- 172.21.15.57:0/1740255175 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f2b78014070 con 0x7f2b880738b0 2023-10-06T22:14:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.983+0000 7f2b85d9b700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2b700a1550 0x7f2b700a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:31.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.983+0000 7f2b85d9b700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2b700a1550 0x7f2b700a3a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009cb0 tx=0x7f2b7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:31.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:31.987+0000 7f2b777fe700 1 -- 172.21.15.57:0/1740255175 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2b78052100 con 0x7f2b880738b0 2023-10-06T22:14:32.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:31 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/838916634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:32.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:31 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2824817212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:31 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/838916634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:32.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:31 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2824817212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:32.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:31 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/838916634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:32.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:31 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2824817212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:32.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.197+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b8802d350 con 0x7f2b700a1550 2023-10-06T22:14:32.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.208+0000 7f2b777fe700 1 -- 172.21.15.57:0/1740255175 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345234 (secure 0 0 0) 0x7f2b8802d350 con 0x7f2b700a1550 2023-10-06T22:14:32.215 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:32.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2b700a1550 msgr2=0x7f2b700a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2b700a1550 0x7f2b700a3a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009cb0 tx=0x7f2b7c009260 comp rx=0 tx=0).stop 2023-10-06T22:14:32.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b880738b0 msgr2=0x7f2b88198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b880738b0 0x7f2b88198e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e9a0 tx=0x7f2b7800ed60 comp rx=0 tx=0).stop 2023-10-06T22:14:32.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 shutdown_connections 2023-10-06T22:14:32.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f2b700a1550 0x7f2b700a3a10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b88108de0 0x7f2b881a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b880738b0 0x7f2b88198e60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 --2- 172.21.15.57:0/1740255175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b88074e90 0x7f2b881993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.212+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 >> 172.21.15.57:0/1740255175 conn(0x7f2b880fc460 msgr2=0x7f2b8810aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:32.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.213+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 shutdown_connections 2023-10-06T22:14:32.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.213+0000 7f2b8c9ff700 1 -- 172.21.15.57:0/1740255175 wait complete. 2023-10-06T22:14:32.219 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-06T22:14:32.234 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-10-06T22:14:31.647726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657018,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1150,"num_read_kb":694,"num_write":428,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337180,"kb_used_data":12988,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604644,"statfs":{"total":1151940427776,"available":1151595155456,"internally_reserved":0,"allocated":13299712,"data_stored":7285070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":52,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002915"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676898+0000","last_change":"2023-10-06T22:12:43.704484+0000","last_active":"2023-10-06T22:14:05.676898+0000","last_peered":"2023-10-06T22:14:05.676898+0000","last_clean":"2023-10-06T22:14:05.676898+0000","last_became_active":"2023-10-06T22:12:43.703740+0000","last_became_peered":"2023-10-06T22:12:43.703740+0000","last_unstale":"2023-10-06T22:14:05.676898+0000","last_undegraded":"2023-10-06T22:14:05.676898+0000","last_fullsized":"2023-10-06T22:14:05.676898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:25.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672032+0000","last_change":"2023-10-06T22:12:43.696968+0000","last_active":"2023-10-06T22:14:05.672032+0000","last_peered":"2023-10-06T22:14:05.672032+0000","last_clean":"2023-10-06T22:14:05.672032+0000","last_became_active":"2023-10-06T22:12:43.696532+0000","last_became_peered":"2023-10-06T22:12:43.696532+0000","last_unstale":"2023-10-06T22:14:05.672032+0000","last_undegraded":"2023-10-06T22:14:05.672032+0000","last_fullsized":"2023-10-06T22:14:05.672032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:59:50.754606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675300+0000","last_change":"2023-10-06T22:12:43.700845+0000","last_active":"2023-10-06T22:14:05.675300+0000","last_peered":"2023-10-06T22:14:05.675300+0000","last_clean":"2023-10-06T22:14:05.675300+0000","last_became_active":"2023-10-06T22:12:43.700531+0000","last_became_peered":"2023-10-06T22:12:43.700531+0000","last_unstale":"2023-10-06T22:14:05.675300+0000","last_undegraded":"2023-10-06T22:14:05.675300+0000","last_fullsized":"2023-10-06T22:14:05.675300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:18.518772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667765+0000","last_change":"2023-10-06T22:12:43.700806+0000","last_active":"2023-10-06T22:14:05.667765+0000","last_peered":"2023-10-06T22:14:05.667765+0000","last_clean":"2023-10-06T22:14:05.667765+0000","last_became_active":"2023-10-06T22:12:43.699901+0000","last_became_peered":"2023-10-06T22:12:43.699901+0000","last_unstale":"2023-10-06T22:14:05.667765+0000","last_undegraded":"2023-10-06T22:14:05.667765+0000","last_fullsized":"2023-10-06T22:14:05.667765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:41:35.906415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.674088+0000","last_change":"2023-10-06T22:12:43.694776+0000","last_active":"2023-10-06T22:14:05.674088+0000","last_peered":"2023-10-06T22:14:05.674088+0000","last_clean":"2023-10-06T22:14:05.674088+0000","last_became_active":"2023-10-06T22:12:43.694397+0000","last_became_peered":"2023-10-06T22:12:43.694397+0000","last_unstale":"2023-10-06T22:14:05.674088+0000","last_undegraded":"2023-10-06T22:14:05.674088+0000","last_fullsized":"2023-10-06T22:14:05.674088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:47:43.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665097+0000","last_change":"2023-10-06T22:12:43.703515+0000","last_active":"2023-10-06T22:14:05.665097+0000","last_peered":"2023-10-06T22:14:05.665097+0000","last_clean":"2023-10-06T22:14:05.665097+0000","last_became_active":"2023-10-06T22:12:43.703094+0000","last_became_peered":"2023-10-06T22:12:43.703094+0000","last_unstale":"2023-10-06T22:14:05.665097+0000","last_undegraded":"2023-10-06T22:14:05.665097+0000","last_fullsized":"2023-10-06T22:14:05.665097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:54:32.130131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667950+0000","last_change":"2023-10-06T22:12:43.697894+0000","last_active":"2023-10-06T22:14:05.667950+0000","last_peered":"2023-10-06T22:14:05.667950+0000","last_clean":"2023-10-06T22:14:05.667950+0000","last_became_active":"2023-10-06T22:12:43.697591+0000","last_became_peered":"2023-10-06T22:12:43.697591+0000","last_unstale":"2023-10-06T22:14:05.667950+0000","last_undegraded":"2023-10-06T22:14:05.667950+0000","last_fullsized":"2023-10-06T22:14:05.667950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:30.313447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668433+0000","last_change":"2023-10-06T22:13:06.927879+0000","last_active":"2023-10-06T22:14:05.668433+0000","last_peered":"2023-10-06T22:14:05.668433+0000","last_clean":"2023-10-06T22:14:05.668433+0000","last_became_active":"2023-10-06T22:13:06.927559+0000","last_became_peered":"2023-10-06T22:13:06.927559+0000","last_unstale":"2023-10-06T22:14:05.668433+0000","last_undegraded":"2023-10-06T22:14:05.668433+0000","last_fullsized":"2023-10-06T22:14:05.668433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:15.874159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664732+0000","last_change":"2023-10-06T22:12:43.696995+0000","last_active":"2023-10-06T22:14:05.664732+0000","last_peered":"2023-10-06T22:14:05.664732+0000","last_clean":"2023-10-06T22:14:05.664732+0000","last_became_active":"2023-10-06T22:12:43.696731+0000","last_became_peered":"2023-10-06T22:12:43.696731+0000","last_unstale":"2023-10-06T22:14:05.664732+0000","last_undegraded":"2023-10-06T22:14:05.664732+0000","last_fullsized":"2023-10-06T22:14:05.664732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:43:31.261922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663417+0000","last_change":"2023-10-06T22:12:43.704391+0000","last_active":"2023-10-06T22:14:05.663417+0000","last_peered":"2023-10-06T22:14:05.663417+0000","last_clean":"2023-10-06T22:14:05.663417+0000","last_became_active":"2023-10-06T22:12:43.702954+0000","last_became_peered":"2023-10-06T22:12:43.702954+0000","last_unstale":"2023-10-06T22:14:05.663417+0000","last_undegraded":"2023-10-06T22:14:05.663417+0000","last_fullsized":"2023-10-06T22:14:05.663417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:38.973781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666760+0000","last_change":"2023-10-06T22:12:43.696265+0000","last_active":"2023-10-06T22:14:05.666760+0000","last_peered":"2023-10-06T22:14:05.666760+0000","last_clean":"2023-10-06T22:14:05.666760+0000","last_became_active":"2023-10-06T22:12:43.695939+0000","last_became_peered":"2023-10-06T22:12:43.695939+0000","last_unstale":"2023-10-06T22:14:05.666760+0000","last_undegraded":"2023-10-06T22:14:05.666760+0000","last_fullsized":"2023-10-06T22:14:05.666760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:32.882083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668888+0000","last_change":"2023-10-06T22:13:06.812700+0000","last_active":"2023-10-06T22:14:05.668888+0000","last_peered":"2023-10-06T22:14:05.668888+0000","last_clean":"2023-10-06T22:14:05.668888+0000","last_became_active":"2023-10-06T22:13:06.812313+0000","last_became_peered":"2023-10-06T22:13:06.812313+0000","last_unstale":"2023-10-06T22:14:05.668888+0000","last_undegraded":"2023-10-06T22:14:05.668888+0000","last_fullsized":"2023-10-06T22:14:05.668888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:25.010233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676269+0000","last_change":"2023-10-06T22:12:43.694717+0000","last_active":"2023-10-06T22:14:05.676269+0000","last_peered":"2023-10-06T22:14:05.676269+0000","last_clean":"2023-10-06T22:14:05.676269+0000","last_became_active":"2023-10-06T22:12:43.694321+0000","last_became_peered":"2023-10-06T22:12:43.694321+0000","last_unstale":"2023-10-06T22:14:05.676269+0000","last_undegraded":"2023-10-06T22:14:05.676269+0000","last_fullsized":"2023-10-06T22:14:05.676269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:06.334726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669546+0000","last_change":"2023-10-06T22:12:43.703770+0000","last_active":"2023-10-06T22:14:05.669546+0000","last_peered":"2023-10-06T22:14:05.669546+0000","last_clean":"2023-10-06T22:14:05.669546+0000","last_became_active":"2023-10-06T22:12:43.703434+0000","last_became_peered":"2023-10-06T22:12:43.703434+0000","last_unstale":"2023-10-06T22:14:05.669546+0000","last_undegraded":"2023-10-06T22:14:05.669546+0000","last_fullsized":"2023-10-06T22:14:05.669546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:25.894280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676822+0000","last_change":"2023-10-06T22:12:43.700994+0000","last_active":"2023-10-06T22:14:05.676822+0000","last_peered":"2023-10-06T22:14:05.676822+0000","last_clean":"2023-10-06T22:14:05.676822+0000","last_became_active":"2023-10-06T22:12:43.700652+0000","last_became_peered":"2023-10-06T22:12:43.700652+0000","last_unstale":"2023-10-06T22:14:05.676822+0000","last_undegraded":"2023-10-06T22:14:05.676822+0000","last_fullsized":"2023-10-06T22:14:05.676822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:54.855597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676182+0000","last_change":"2023-10-06T22:12:43.702099+0000","last_active":"2023-10-06T22:14:05.676182+0000","last_peered":"2023-10-06T22:14:05.676182+0000","last_clean":"2023-10-06T22:14:05.676182+0000","last_became_active":"2023-10-06T22:12:43.701802+0000","last_became_peered":"2023-10-06T22:12:43.701802+0000","last_unstale":"2023-10-06T22:14:05.676182+0000","last_undegraded":"2023-10-06T22:14:05.676182+0000","last_fullsized":"2023-10-06T22:14:05.676182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:00.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677168+0000","last_change":"2023-10-06T22:12:43.704679+0000","last_active":"2023-10-06T22:14:05.677168+0000","last_peered":"2023-10-06T22:14:05.677168+0000","last_clean":"2023-10-06T22:14:05.677168+0000","last_became_active":"2023-10-06T22:12:43.704186+0000","last_became_peered":"2023-10-06T22:12:43.704186+0000","last_unstale":"2023-10-06T22:14:05.677168+0000","last_undegraded":"2023-10-06T22:14:05.677168+0000","last_fullsized":"2023-10-06T22:14:05.677168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:28.665217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668666+0000","last_change":"2023-10-06T22:12:43.695699+0000","last_active":"2023-10-06T22:14:05.668666+0000","last_peered":"2023-10-06T22:14:05.668666+0000","last_clean":"2023-10-06T22:14:05.668666+0000","last_became_active":"2023-10-06T22:12:43.695359+0000","last_became_peered":"2023-10-06T22:12:43.695359+0000","last_unstale":"2023-10-06T22:14:05.668666+0000","last_undegraded":"2023-10-06T22:14:05.668666+0000","last_fullsized":"2023-10-06T22:14:05.668666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:46.296768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672411+0000","last_change":"2023-10-06T22:12:43.707170+0000","last_active":"2023-10-06T22:14:05.672411+0000","last_peered":"2023-10-06T22:14:05.672411+0000","last_clean":"2023-10-06T22:14:05.672411+0000","last_became_active":"2023-10-06T22:12:43.706875+0000","last_became_peered":"2023-10-06T22:12:43.706875+0000","last_unstale":"2023-10-06T22:14:05.672411+0000","last_undegraded":"2023-10-06T22:14:05.672411+0000","last_fullsized":"2023-10-06T22:14:05.672411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:46.548458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":147,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:24.833047+0000","last_change":"2023-10-06T22:13:06.736382+0000","last_active":"2023-10-06T22:14:24.833047+0000","last_peered":"2023-10-06T22:14:24.833047+0000","last_clean":"2023-10-06T22:14:24.833047+0000","last_became_active":"2023-10-06T22:13:06.725703+0000","last_became_peered":"2023-10-06T22:13:06.725703+0000","last_unstale":"2023-10-06T22:14:24.833047+0000","last_undegraded":"2023-10-06T22:14:24.833047+0000","last_fullsized":"2023-10-06T22:14:24.833047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:10.056384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671657+0000","last_change":"2023-10-06T22:12:43.702574+0000","last_active":"2023-10-06T22:14:05.671657+0000","last_peered":"2023-10-06T22:14:05.671657+0000","last_clean":"2023-10-06T22:14:05.671657+0000","last_became_active":"2023-10-06T22:12:43.702238+0000","last_became_peered":"2023-10-06T22:12:43.702238+0000","last_unstale":"2023-10-06T22:14:05.671657+0000","last_undegraded":"2023-10-06T22:14:05.671657+0000","last_fullsized":"2023-10-06T22:14:05.671657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:34:09.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752842+0000","last_change":"2023-10-06T22:12:42.697256+0000","last_active":"2023-10-06T22:13:05.752842+0000","last_peered":"2023-10-06T22:13:05.752842+0000","last_clean":"2023-10-06T22:13:05.752842+0000","last_became_active":"2023-10-06T22:12:42.696724+0000","last_became_peered":"2023-10-06T22:12:42.696724+0000","last_unstale":"2023-10-06T22:13:05.752842+0000","last_undegraded":"2023-10-06T22:13:05.752842+0000","last_fullsized":"2023-10-06T22:13:05.752842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:50:01.378292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679610+0000","last_change":"2023-10-06T22:12:43.702050+0000","last_active":"2023-10-06T22:14:05.679610+0000","last_peered":"2023-10-06T22:14:05.679610+0000","last_clean":"2023-10-06T22:14:05.679610+0000","last_became_active":"2023-10-06T22:12:43.701432+0000","last_became_peered":"2023-10-06T22:12:43.701432+0000","last_unstale":"2023-10-06T22:14:05.679610+0000","last_undegraded":"2023-10-06T22:14:05.679610+0000","last_fullsized":"2023-10-06T22:14:05.679610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:30:37.909775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.948160+0000","last_change":"2023-10-06T22:13:06.948160+0000","last_active":"2023-10-06T22:13:06.948160+0000","last_peered":"2023-10-06T22:13:06.948160+0000","last_clean":"2023-10-06T22:13:06.948160+0000","last_became_active":"2023-10-06T22:13:06.927995+0000","last_became_peered":"2023-10-06T22:13:06.927995+0000","last_unstale":"2023-10-06T22:13:06.948160+0000","last_undegraded":"2023-10-06T22:13:06.948160+0000","last_fullsized":"2023-10-06T22:13:06.948160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:33:27.974776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.655518+0000","last_change":"2023-10-06T22:12:43.696511+0000","last_active":"2023-10-06T22:14:05.655518+0000","last_peered":"2023-10-06T22:14:05.655518+0000","last_clean":"2023-10-06T22:14:05.655518+0000","last_became_active":"2023-10-06T22:12:43.696087+0000","last_became_peered":"2023-10-06T22:12:43.696087+0000","last_unstale":"2023-10-06T22:14:05.655518+0000","last_undegraded":"2023-10-06T22:14:05.655518+0000","last_fullsized":"2023-10-06T22:14:05.655518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:12:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724948+0000","last_change":"2023-10-06T22:12:42.694963+0000","last_active":"2023-10-06T22:13:05.724948+0000","last_peered":"2023-10-06T22:13:05.724948+0000","last_clean":"2023-10-06T22:13:05.724948+0000","last_became_active":"2023-10-06T22:12:42.694517+0000","last_became_peered":"2023-10-06T22:12:42.694517+0000","last_unstale":"2023-10-06T22:13:05.724948+0000","last_undegraded":"2023-10-06T22:13:05.724948+0000","last_fullsized":"2023-10-06T22:13:05.724948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:07:33.404328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663561+0000","last_change":"2023-10-06T22:12:43.698077+0000","last_active":"2023-10-06T22:14:05.663561+0000","last_peered":"2023-10-06T22:14:05.663561+0000","last_clean":"2023-10-06T22:14:05.663561+0000","last_became_active":"2023-10-06T22:12:43.697676+0000","last_became_peered":"2023-10-06T22:12:43.697676+0000","last_unstale":"2023-10-06T22:14:05.663561+0000","last_undegraded":"2023-10-06T22:14:05.663561+0000","last_fullsized":"2023-10-06T22:14:05.663561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:59:56.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808989+0000","last_change":"2023-10-06T22:12:42.698067+0000","last_active":"2023-10-06T22:14:27.808989+0000","last_peered":"2023-10-06T22:14:27.808989+0000","last_clean":"2023-10-06T22:14:27.808989+0000","last_became_active":"2023-10-06T22:12:42.697671+0000","last_became_peered":"2023-10-06T22:12:42.697671+0000","last_unstale":"2023-10-06T22:14:27.808989+0000","last_undegraded":"2023-10-06T22:14:27.808989+0000","last_fullsized":"2023-10-06T22:14:27.808989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:33.607513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.661778+0000","last_change":"2023-10-06T22:12:43.696213+0000","last_active":"2023-10-06T22:14:05.661778+0000","last_peered":"2023-10-06T22:14:05.661778+0000","last_clean":"2023-10-06T22:14:05.661778+0000","last_became_active":"2023-10-06T22:12:43.695925+0000","last_became_peered":"2023-10-06T22:12:43.695925+0000","last_unstale":"2023-10-06T22:14:05.661778+0000","last_undegraded":"2023-10-06T22:14:05.661778+0000","last_fullsized":"2023-10-06T22:14:05.661778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:24.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.719695+0000","last_change":"2023-10-06T22:12:42.687188+0000","last_active":"2023-10-06T22:13:05.719695+0000","last_peered":"2023-10-06T22:13:05.719695+0000","last_clean":"2023-10-06T22:13:05.719695+0000","last_became_active":"2023-10-06T22:12:42.686747+0000","last_became_peered":"2023-10-06T22:12:42.686747+0000","last_unstale":"2023-10-06T22:13:05.719695+0000","last_undegraded":"2023-10-06T22:13:05.719695+0000","last_fullsized":"2023-10-06T22:13:05.719695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:21.326911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664388+0000","last_change":"2023-10-06T22:12:43.697988+0000","last_active":"2023-10-06T22:14:05.664388+0000","last_peered":"2023-10-06T22:14:05.664388+0000","last_clean":"2023-10-06T22:14:05.664388+0000","last_became_active":"2023-10-06T22:12:43.697641+0000","last_became_peered":"2023-10-06T22:12:43.697641+0000","last_unstale":"2023-10-06T22:14:05.664388+0000","last_undegraded":"2023-10-06T22:14:05.664388+0000","last_fullsized":"2023-10-06T22:14:05.664388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:33.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.423261+0000","last_change":"2023-10-06T22:12:42.688882+0000","last_active":"2023-10-06T22:13:06.423261+0000","last_peered":"2023-10-06T22:13:06.423261+0000","last_clean":"2023-10-06T22:13:06.423261+0000","last_became_active":"2023-10-06T22:12:42.687653+0000","last_became_peered":"2023-10-06T22:12:42.687653+0000","last_unstale":"2023-10-06T22:13:06.423261+0000","last_undegraded":"2023-10-06T22:13:06.423261+0000","last_fullsized":"2023-10-06T22:13:06.423261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:18.146081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667479+0000","last_change":"2023-10-06T22:12:43.696559+0000","last_active":"2023-10-06T22:14:05.667479+0000","last_peered":"2023-10-06T22:14:05.667479+0000","last_clean":"2023-10-06T22:14:05.667479+0000","last_became_active":"2023-10-06T22:12:43.696252+0000","last_became_peered":"2023-10-06T22:12:43.696252+0000","last_unstale":"2023-10-06T22:14:05.667479+0000","last_undegraded":"2023-10-06T22:14:05.667479+0000","last_fullsized":"2023-10-06T22:14:05.667479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:31:36.097000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722785+0000","last_change":"2023-10-06T22:12:42.694299+0000","last_active":"2023-10-06T22:13:05.722785+0000","last_peered":"2023-10-06T22:13:05.722785+0000","last_clean":"2023-10-06T22:13:05.722785+0000","last_became_active":"2023-10-06T22:12:42.693820+0000","last_became_peered":"2023-10-06T22:12:42.693820+0000","last_unstale":"2023-10-06T22:13:05.722785+0000","last_undegraded":"2023-10-06T22:13:05.722785+0000","last_fullsized":"2023-10-06T22:13:05.722785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:13.837379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672816+0000","last_change":"2023-10-06T22:12:43.695234+0000","last_active":"2023-10-06T22:14:05.672816+0000","last_peered":"2023-10-06T22:14:05.672816+0000","last_clean":"2023-10-06T22:14:05.672816+0000","last_became_active":"2023-10-06T22:12:43.694889+0000","last_became_peered":"2023-10-06T22:12:43.694889+0000","last_unstale":"2023-10-06T22:14:05.672816+0000","last_undegraded":"2023-10-06T22:14:05.672816+0000","last_fullsized":"2023-10-06T22:14:05.672816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:51.039264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808525+0000","last_change":"2023-10-06T22:12:42.695438+0000","last_active":"2023-10-06T22:14:27.808525+0000","last_peered":"2023-10-06T22:14:27.808525+0000","last_clean":"2023-10-06T22:14:27.808525+0000","last_became_active":"2023-10-06T22:12:42.695090+0000","last_became_peered":"2023-10-06T22:12:42.695090+0000","last_unstale":"2023-10-06T22:14:27.808525+0000","last_undegraded":"2023-10-06T22:14:27.808525+0000","last_fullsized":"2023-10-06T22:14:27.808525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:25:34.385961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663101+0000","last_change":"2023-10-06T22:12:43.697459+0000","last_active":"2023-10-06T22:14:05.663101+0000","last_peered":"2023-10-06T22:14:05.663101+0000","last_clean":"2023-10-06T22:14:05.663101+0000","last_became_active":"2023-10-06T22:12:43.696405+0000","last_became_peered":"2023-10-06T22:12:43.696405+0000","last_unstale":"2023-10-06T22:14:05.663101+0000","last_undegraded":"2023-10-06T22:14:05.663101+0000","last_fullsized":"2023-10-06T22:14:05.663101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:30.928083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727798+0000","last_change":"2023-10-06T22:12:42.692151+0000","last_active":"2023-10-06T22:13:05.727798+0000","last_peered":"2023-10-06T22:13:05.727798+0000","last_clean":"2023-10-06T22:13:05.727798+0000","last_became_active":"2023-10-06T22:12:42.691780+0000","last_became_peered":"2023-10-06T22:12:42.691780+0000","last_unstale":"2023-10-06T22:13:05.727798+0000","last_undegraded":"2023-10-06T22:13:05.727798+0000","last_fullsized":"2023-10-06T22:13:05.727798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:49:35.697733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679194+0000","last_change":"2023-10-06T22:12:43.701754+0000","last_active":"2023-10-06T22:14:05.679194+0000","last_peered":"2023-10-06T22:14:05.679194+0000","last_clean":"2023-10-06T22:14:05.679194+0000","last_became_active":"2023-10-06T22:12:43.700881+0000","last_became_peered":"2023-10-06T22:12:43.700881+0000","last_unstale":"2023-10-06T22:14:05.679194+0000","last_undegraded":"2023-10-06T22:14:05.679194+0000","last_fullsized":"2023-10-06T22:14:05.679194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:35:21.988307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723824+0000","last_change":"2023-10-06T22:12:42.699751+0000","last_active":"2023-10-06T22:13:05.723824+0000","last_peered":"2023-10-06T22:13:05.723824+0000","last_clean":"2023-10-06T22:13:05.723824+0000","last_became_active":"2023-10-06T22:12:42.699321+0000","last_became_peered":"2023-10-06T22:12:42.699321+0000","last_unstale":"2023-10-06T22:13:05.723824+0000","last_undegraded":"2023-10-06T22:13:05.723824+0000","last_fullsized":"2023-10-06T22:13:05.723824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:35:25.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.660837+0000","last_change":"2023-10-06T22:12:43.702426+0000","last_active":"2023-10-06T22:14:05.660837+0000","last_peered":"2023-10-06T22:14:05.660837+0000","last_clean":"2023-10-06T22:14:05.660837+0000","last_became_active":"2023-10-06T22:12:43.702101+0000","last_became_peered":"2023-10-06T22:12:43.702101+0000","last_unstale":"2023-10-06T22:14:05.660837+0000","last_undegraded":"2023-10-06T22:14:05.660837+0000","last_fullsized":"2023-10-06T22:14:05.660837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:31:09.133941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":252,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808520+0000","last_change":"2023-10-06T22:12:42.688815+0000","last_active":"2023-10-06T22:14:27.808520+0000","last_peered":"2023-10-06T22:14:27.808520+0000","last_clean":"2023-10-06T22:14:27.808520+0000","last_became_active":"2023-10-06T22:12:42.688447+0000","last_became_peered":"2023-10-06T22:12:42.688447+0000","last_unstale":"2023-10-06T22:14:27.808520+0000","last_undegraded":"2023-10-06T22:14:27.808520+0000","last_fullsized":"2023-10-06T22:14:27.808520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:26:25.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58073,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":204,"num_read_kb":159,"num_write":35,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666683+0000","last_change":"2023-10-06T22:12:43.698406+0000","last_active":"2023-10-06T22:14:05.666683+0000","last_peered":"2023-10-06T22:14:05.666683+0000","last_clean":"2023-10-06T22:14:05.666683+0000","last_became_active":"2023-10-06T22:12:43.697204+0000","last_became_peered":"2023-10-06T22:12:43.697204+0000","last_unstale":"2023-10-06T22:14:05.666683+0000","last_undegraded":"2023-10-06T22:14:05.666683+0000","last_fullsized":"2023-10-06T22:14:05.666683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:59.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727727+0000","last_change":"2023-10-06T22:12:42.691007+0000","last_active":"2023-10-06T22:13:05.727727+0000","last_peered":"2023-10-06T22:13:05.727727+0000","last_clean":"2023-10-06T22:13:05.727727+0000","last_became_active":"2023-10-06T22:12:42.690693+0000","last_became_peered":"2023-10-06T22:12:42.690693+0000","last_unstale":"2023-10-06T22:13:05.727727+0000","last_undegraded":"2023-10-06T22:13:05.727727+0000","last_fullsized":"2023-10-06T22:13:05.727727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:47.670091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671518+0000","last_change":"2023-10-06T22:12:43.705131+0000","last_active":"2023-10-06T22:14:05.671518+0000","last_peered":"2023-10-06T22:14:05.671518+0000","last_clean":"2023-10-06T22:14:05.671518+0000","last_became_active":"2023-10-06T22:12:43.704469+0000","last_became_peered":"2023-10-06T22:12:43.704469+0000","last_unstale":"2023-10-06T22:14:05.671518+0000","last_undegraded":"2023-10-06T22:14:05.671518+0000","last_fullsized":"2023-10-06T22:14:05.671518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:36:39.392274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723093+0000","last_change":"2023-10-06T22:12:42.698392+0000","last_active":"2023-10-06T22:13:05.723093+0000","last_peered":"2023-10-06T22:13:05.723093+0000","last_clean":"2023-10-06T22:13:05.723093+0000","last_became_active":"2023-10-06T22:12:42.698019+0000","last_became_peered":"2023-10-06T22:12:42.698019+0000","last_unstale":"2023-10-06T22:13:05.723093+0000","last_undegraded":"2023-10-06T22:13:05.723093+0000","last_fullsized":"2023-10-06T22:13:05.723093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:00.345300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677485+0000","last_change":"2023-10-06T22:12:43.693238+0000","last_active":"2023-10-06T22:14:05.677485+0000","last_peered":"2023-10-06T22:14:05.677485+0000","last_clean":"2023-10-06T22:14:05.677485+0000","last_became_active":"2023-10-06T22:12:43.692948+0000","last_became_peered":"2023-10-06T22:12:43.692948+0000","last_unstale":"2023-10-06T22:14:05.677485+0000","last_undegraded":"2023-10-06T22:14:05.677485+0000","last_fullsized":"2023-10-06T22:14:05.677485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:24:30.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.816392+0000","last_change":"2023-10-06T22:12:42.693213+0000","last_active":"2023-10-06T22:14:26.816392+0000","last_peered":"2023-10-06T22:14:26.816392+0000","last_clean":"2023-10-06T22:14:26.816392+0000","last_became_active":"2023-10-06T22:12:42.692894+0000","last_became_peered":"2023-10-06T22:12:42.692894+0000","last_unstale":"2023-10-06T22:14:26.816392+0000","last_undegraded":"2023-10-06T22:14:26.816392+0000","last_fullsized":"2023-10-06T22:14:26.816392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:40:21.159102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669294+0000","last_change":"2023-10-06T22:12:43.695527+0000","last_active":"2023-10-06T22:14:05.669294+0000","last_peered":"2023-10-06T22:14:05.669294+0000","last_clean":"2023-10-06T22:14:05.669294+0000","last_became_active":"2023-10-06T22:12:43.694364+0000","last_became_peered":"2023-10-06T22:12:43.694364+0000","last_unstale":"2023-10-06T22:14:05.669294+0000","last_undegraded":"2023-10-06T22:14:05.669294+0000","last_fullsized":"2023-10-06T22:14:05.669294+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723938+0000","last_change":"2023-10-06T22:12:42.697854+0000","last_active":"2023-10-06T22:13:05.723938+0000","last_peered":"2023-10-06T22:13:05.723938+0000","last_clean":"2023-10-06T22:13:05.723938+0000","last_became_active":"2023-10-06T22:12:42.697542+0000","last_became_peered":"2023-10-06T22:12:42.697542+0000","last_unstale":"2023-10-06T22:13:05.723938+0000","last_undegraded":"2023-10-06T22:13:05.723938+0000","last_fullsized":"2023-10-06T22:13:05.723938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:12:11.181398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665099+0000","last_change":"2023-10-06T22:12:43.698070+0000","last_active":"2023-10-06T22:14:05.665099+0000","last_peered":"2023-10-06T22:14:05.665099+0000","last_clean":"2023-10-06T22:14:05.665099+0000","last_became_active":"2023-10-06T22:12:43.697682+0000","last_became_peered":"2023-10-06T22:12:43.697682+0000","last_unstale":"2023-10-06T22:14:05.665099+0000","last_undegraded":"2023-10-06T22:14:05.665099+0000","last_fullsized":"2023-10-06T22:14:05.665099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:43.808866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724573+0000","last_change":"2023-10-06T22:12:42.691691+0000","last_active":"2023-10-06T22:13:05.724573+0000","last_peered":"2023-10-06T22:13:05.724573+0000","last_clean":"2023-10-06T22:13:05.724573+0000","last_became_active":"2023-10-06T22:12:42.691001+0000","last_became_peered":"2023-10-06T22:12:42.691001+0000","last_unstale":"2023-10-06T22:13:05.724573+0000","last_undegraded":"2023-10-06T22:13:05.724573+0000","last_fullsized":"2023-10-06T22:13:05.724573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:11:18.559570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.660998+0000","last_change":"2023-10-06T22:13:06.927620+0000","last_active":"2023-10-06T22:14:05.660998+0000","last_peered":"2023-10-06T22:14:05.660998+0000","last_clean":"2023-10-06T22:14:05.660998+0000","last_became_active":"2023-10-06T22:13:06.927206+0000","last_became_peered":"2023-10-06T22:13:06.927206+0000","last_unstale":"2023-10-06T22:14:05.660998+0000","last_undegraded":"2023-10-06T22:14:05.660998+0000","last_fullsized":"2023-10-06T22:14:05.660998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:50:30.780174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.726783+0000","last_change":"2023-10-06T22:13:06.726783+0000","last_active":"2023-10-06T22:13:06.726783+0000","last_peered":"2023-10-06T22:13:06.726783+0000","last_clean":"2023-10-06T22:13:06.726783+0000","last_became_active":"2023-10-06T22:13:06.726342+0000","last_became_peered":"2023-10-06T22:13:06.726342+0000","last_unstale":"2023-10-06T22:13:06.726783+0000","last_undegraded":"2023-10-06T22:13:06.726783+0000","last_fullsized":"2023-10-06T22:13:06.726783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:42.359819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672141+0000","last_change":"2023-10-06T22:12:43.702870+0000","last_active":"2023-10-06T22:14:05.672141+0000","last_peered":"2023-10-06T22:14:05.672141+0000","last_clean":"2023-10-06T22:14:05.672141+0000","last_became_active":"2023-10-06T22:12:43.702564+0000","last_became_peered":"2023-10-06T22:12:43.702564+0000","last_unstale":"2023-10-06T22:14:05.672141+0000","last_undegraded":"2023-10-06T22:14:05.672141+0000","last_fullsized":"2023-10-06T22:14:05.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:23:32.238268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.836399+0000","last_change":"2023-10-06T22:12:42.696603+0000","last_active":"2023-10-06T22:14:26.836399+0000","last_peered":"2023-10-06T22:14:26.836399+0000","last_clean":"2023-10-06T22:14:26.836399+0000","last_became_active":"2023-10-06T22:12:42.696313+0000","last_became_peered":"2023-10-06T22:12:42.696313+0000","last_unstale":"2023-10-06T22:14:26.836399+0000","last_undegraded":"2023-10-06T22:14:26.836399+0000","last_fullsized":"2023-10-06T22:14:26.836399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:15.719749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675740+0000","last_change":"2023-10-06T22:12:43.703356+0000","last_active":"2023-10-06T22:14:05.675740+0000","last_peered":"2023-10-06T22:14:05.675740+0000","last_clean":"2023-10-06T22:14:05.675740+0000","last_became_active":"2023-10-06T22:12:43.702670+0000","last_became_peered":"2023-10-06T22:12:43.702670+0000","last_unstale":"2023-10-06T22:14:05.675740+0000","last_undegraded":"2023-10-06T22:14:05.675740+0000","last_fullsized":"2023-10-06T22:14:05.675740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:15:05.681491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665212+0000","last_change":"2023-10-06T22:12:43.699083+0000","last_active":"2023-10-06T22:14:05.665212+0000","last_peered":"2023-10-06T22:14:05.665212+0000","last_clean":"2023-10-06T22:14:05.665212+0000","last_became_active":"2023-10-06T22:12:43.698826+0000","last_became_peered":"2023-10-06T22:12:43.698826+0000","last_unstale":"2023-10-06T22:14:05.665212+0000","last_undegraded":"2023-10-06T22:14:05.665212+0000","last_fullsized":"2023-10-06T22:14:05.665212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:04.225060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'7","reported_seq":120,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.515215+0000","last_change":"2023-10-06T22:12:42.699873+0000","last_active":"2023-10-06T22:14:26.515215+0000","last_peered":"2023-10-06T22:14:26.515215+0000","last_clean":"2023-10-06T22:14:26.515215+0000","last_became_active":"2023-10-06T22:12:42.699476+0000","last_became_peered":"2023-10-06T22:12:42.699476+0000","last_unstale":"2023-10-06T22:14:26.515215+0000","last_undegraded":"2023-10-06T22:14:26.515215+0000","last_fullsized":"2023-10-06T22:14:26.515215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:13:27.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.658075+0000","last_change":"2023-10-06T22:12:43.697720+0000","last_active":"2023-10-06T22:14:05.658075+0000","last_peered":"2023-10-06T22:14:05.658075+0000","last_clean":"2023-10-06T22:14:05.658075+0000","last_became_active":"2023-10-06T22:12:43.697430+0000","last_became_peered":"2023-10-06T22:12:43.697430+0000","last_unstale":"2023-10-06T22:14:05.658075+0000","last_undegraded":"2023-10-06T22:14:05.658075+0000","last_fullsized":"2023-10-06T22:14:05.658075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:29.633012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722760+0000","last_change":"2023-10-06T22:12:42.690781+0000","last_active":"2023-10-06T22:13:05.722760+0000","last_peered":"2023-10-06T22:13:05.722760+0000","last_clean":"2023-10-06T22:13:05.722760+0000","last_became_active":"2023-10-06T22:12:42.690274+0000","last_became_peered":"2023-10-06T22:12:42.690274+0000","last_unstale":"2023-10-06T22:13:05.722760+0000","last_undegraded":"2023-10-06T22:13:05.722760+0000","last_fullsized":"2023-10-06T22:13:05.722760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:26.653182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.724529+0000","last_change":"2023-10-06T22:11:36.409669+0000","last_active":"2023-10-06T22:13:06.724529+0000","last_peered":"2023-10-06T22:13:06.724529+0000","last_clean":"2023-10-06T22:13:06.724529+0000","last_became_active":"2023-10-06T22:11:36.409197+0000","last_became_peered":"2023-10-06T22:11:36.409197+0000","last_unstale":"2023-10-06T22:13:06.724529+0000","last_undegraded":"2023-10-06T22:13:06.724529+0000","last_fullsized":"2023-10-06T22:13:06.724529+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_clean_scrub_stamp":"2023-10-06T22:08:24.598743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:50:55.982571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672700+0000","last_change":"2023-10-06T22:12:43.698617+0000","last_active":"2023-10-06T22:14:05.672700+0000","last_peered":"2023-10-06T22:14:05.672700+0000","last_clean":"2023-10-06T22:14:05.672700+0000","last_became_active":"2023-10-06T22:12:43.697942+0000","last_became_peered":"2023-10-06T22:12:43.697942+0000","last_unstale":"2023-10-06T22:14:05.672700+0000","last_undegraded":"2023-10-06T22:14:05.672700+0000","last_fullsized":"2023-10-06T22:14:05.672700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:38.075730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727545+0000","last_change":"2023-10-06T22:12:42.695553+0000","last_active":"2023-10-06T22:13:05.727545+0000","last_peered":"2023-10-06T22:13:05.727545+0000","last_clean":"2023-10-06T22:13:05.727545+0000","last_became_active":"2023-10-06T22:12:42.695209+0000","last_became_peered":"2023-10-06T22:12:42.695209+0000","last_unstale":"2023-10-06T22:13:05.727545+0000","last_undegraded":"2023-10-06T22:13:05.727545+0000","last_fullsized":"2023-10-06T22:13:05.727545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:10.387710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724270+0000","last_change":"2023-10-06T22:12:42.699606+0000","last_active":"2023-10-06T22:13:05.724270+0000","last_peered":"2023-10-06T22:13:05.724270+0000","last_clean":"2023-10-06T22:13:05.724270+0000","last_became_active":"2023-10-06T22:12:42.699222+0000","last_became_peered":"2023-10-06T22:12:42.699222+0000","last_unstale":"2023-10-06T22:13:05.724270+0000","last_undegraded":"2023-10-06T22:13:05.724270+0000","last_fullsized":"2023-10-06T22:13:05.724270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:52.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675984+0000","last_change":"2023-10-06T22:12:43.697783+0000","last_active":"2023-10-06T22:14:05.675984+0000","last_peered":"2023-10-06T22:14:05.675984+0000","last_clean":"2023-10-06T22:14:05.675984+0000","last_became_active":"2023-10-06T22:12:43.697388+0000","last_became_peered":"2023-10-06T22:12:43.697388+0000","last_unstale":"2023-10-06T22:14:05.675984+0000","last_undegraded":"2023-10-06T22:14:05.675984+0000","last_fullsized":"2023-10-06T22:14:05.675984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:50.113458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.721888+0000","last_change":"2023-10-06T22:12:42.694180+0000","last_active":"2023-10-06T22:13:06.721888+0000","last_peered":"2023-10-06T22:13:06.721888+0000","last_clean":"2023-10-06T22:13:06.721888+0000","last_became_active":"2023-10-06T22:12:42.693861+0000","last_became_peered":"2023-10-06T22:12:42.693861+0000","last_unstale":"2023-10-06T22:13:06.721888+0000","last_undegraded":"2023-10-06T22:13:06.721888+0000","last_fullsized":"2023-10-06T22:13:06.721888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:28:02.511117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664374+0000","last_change":"2023-10-06T22:12:43.699118+0000","last_active":"2023-10-06T22:14:05.664374+0000","last_peered":"2023-10-06T22:14:05.664374+0000","last_clean":"2023-10-06T22:14:05.664374+0000","last_became_active":"2023-10-06T22:12:43.698540+0000","last_became_peered":"2023-10-06T22:12:43.698540+0000","last_unstale":"2023-10-06T22:14:05.664374+0000","last_undegraded":"2023-10-06T22:14:05.664374+0000","last_fullsized":"2023-10-06T22:14:05.664374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:31.192762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722961+0000","last_change":"2023-10-06T22:12:42.690617+0000","last_active":"2023-10-06T22:13:05.722961+0000","last_peered":"2023-10-06T22:13:05.722961+0000","last_clean":"2023-10-06T22:13:05.722961+0000","last_became_active":"2023-10-06T22:12:42.690238+0000","last_became_peered":"2023-10-06T22:12:42.690238+0000","last_unstale":"2023-10-06T22:13:05.722961+0000","last_undegraded":"2023-10-06T22:13:05.722961+0000","last_fullsized":"2023-10-06T22:13:05.722961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:09.819570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676115+0000","last_change":"2023-10-06T22:12:43.699064+0000","last_active":"2023-10-06T22:14:05.676115+0000","last_peered":"2023-10-06T22:14:05.676115+0000","last_clean":"2023-10-06T22:14:05.676115+0000","last_became_active":"2023-10-06T22:12:43.698727+0000","last_became_peered":"2023-10-06T22:12:43.698727+0000","last_unstale":"2023-10-06T22:14:05.676115+0000","last_undegraded":"2023-10-06T22:14:05.676115+0000","last_fullsized":"2023-10-06T22:14:05.676115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:48:14.493376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738711+0000","last_change":"2023-10-06T22:12:42.693779+0000","last_active":"2023-10-06T22:13:05.738711+0000","last_peered":"2023-10-06T22:13:05.738711+0000","last_clean":"2023-10-06T22:13:05.738711+0000","last_became_active":"2023-10-06T22:12:42.693257+0000","last_became_peered":"2023-10-06T22:12:42.693257+0000","last_unstale":"2023-10-06T22:13:05.738711+0000","last_undegraded":"2023-10-06T22:13:05.738711+0000","last_fullsized":"2023-10-06T22:13:05.738711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:56:48.945078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.631287+0000","last_change":"2023-10-06T22:12:43.700620+0000","last_active":"2023-10-06T22:14:05.631287+0000","last_peered":"2023-10-06T22:14:05.631287+0000","last_clean":"2023-10-06T22:14:05.631287+0000","last_became_active":"2023-10-06T22:12:43.700093+0000","last_became_peered":"2023-10-06T22:12:43.700093+0000","last_unstale":"2023-10-06T22:14:05.631287+0000","last_undegraded":"2023-10-06T22:14:05.631287+0000","last_fullsized":"2023-10-06T22:14:05.631287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:29.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751425+0000","last_change":"2023-10-06T22:12:42.689640+0000","last_active":"2023-10-06T22:13:05.751425+0000","last_peered":"2023-10-06T22:13:05.751425+0000","last_clean":"2023-10-06T22:13:05.751425+0000","last_became_active":"2023-10-06T22:12:42.688584+0000","last_became_peered":"2023-10-06T22:12:42.688584+0000","last_unstale":"2023-10-06T22:13:05.751425+0000","last_undegraded":"2023-10-06T22:13:05.751425+0000","last_fullsized":"2023-10-06T22:13:05.751425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:43.066396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676684+0000","last_change":"2023-10-06T22:12:43.700084+0000","last_active":"2023-10-06T22:14:05.676684+0000","last_peered":"2023-10-06T22:14:05.676684+0000","last_clean":"2023-10-06T22:14:05.676684+0000","last_became_active":"2023-10-06T22:12:43.699787+0000","last_became_peered":"2023-10-06T22:12:43.699787+0000","last_unstale":"2023-10-06T22:14:05.676684+0000","last_undegraded":"2023-10-06T22:14:05.676684+0000","last_fullsized":"2023-10-06T22:14:05.676684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:07.219800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727627+0000","last_change":"2023-10-06T22:12:42.688750+0000","last_active":"2023-10-06T22:13:05.727627+0000","last_peered":"2023-10-06T22:13:05.727627+0000","last_clean":"2023-10-06T22:13:05.727627+0000","last_became_active":"2023-10-06T22:12:42.688387+0000","last_became_peered":"2023-10-06T22:12:42.688387+0000","last_unstale":"2023-10-06T22:13:05.727627+0000","last_undegraded":"2023-10-06T22:13:05.727627+0000","last_fullsized":"2023-10-06T22:13:05.727627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:32.980631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676460+0000","last_change":"2023-10-06T22:12:43.706741+0000","last_active":"2023-10-06T22:14:05.676460+0000","last_peered":"2023-10-06T22:14:05.676460+0000","last_clean":"2023-10-06T22:14:05.676460+0000","last_became_active":"2023-10-06T22:12:43.706416+0000","last_became_peered":"2023-10-06T22:12:43.706416+0000","last_unstale":"2023-10-06T22:14:05.676460+0000","last_undegraded":"2023-10-06T22:14:05.676460+0000","last_fullsized":"2023-10-06T22:14:05.676460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:55:16.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724994+0000","last_change":"2023-10-06T22:12:42.692850+0000","last_active":"2023-10-06T22:13:05.724994+0000","last_peered":"2023-10-06T22:13:05.724994+0000","last_clean":"2023-10-06T22:13:05.724994+0000","last_became_active":"2023-10-06T22:12:42.692287+0000","last_became_peered":"2023-10-06T22:12:42.692287+0000","last_unstale":"2023-10-06T22:13:05.724994+0000","last_undegraded":"2023-10-06T22:13:05.724994+0000","last_fullsized":"2023-10-06T22:13:05.724994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:37:59.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667276+0000","last_change":"2023-10-06T22:12:43.700667+0000","last_active":"2023-10-06T22:14:05.667276+0000","last_peered":"2023-10-06T22:14:05.667276+0000","last_clean":"2023-10-06T22:14:05.667276+0000","last_became_active":"2023-10-06T22:12:43.698758+0000","last_became_peered":"2023-10-06T22:12:43.698758+0000","last_unstale":"2023-10-06T22:14:05.667276+0000","last_undegraded":"2023-10-06T22:14:05.667276+0000","last_fullsized":"2023-10-06T22:14:05.667276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:49:53.091578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.809273+0000","last_change":"2023-10-06T22:12:42.695964+0000","last_active":"2023-10-06T22:14:27.809273+0000","last_peered":"2023-10-06T22:14:27.809273+0000","last_clean":"2023-10-06T22:14:27.809273+0000","last_became_active":"2023-10-06T22:12:42.695431+0000","last_became_peered":"2023-10-06T22:12:42.695431+0000","last_unstale":"2023-10-06T22:14:27.809273+0000","last_undegraded":"2023-10-06T22:14:27.809273+0000","last_fullsized":"2023-10-06T22:14:27.809273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:21:42.761421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667361+0000","last_change":"2023-10-06T22:12:43.698088+0000","last_active":"2023-10-06T22:14:05.667361+0000","last_peered":"2023-10-06T22:14:05.667361+0000","last_clean":"2023-10-06T22:14:05.667361+0000","last_became_active":"2023-10-06T22:12:43.697765+0000","last_became_peered":"2023-10-06T22:12:43.697765+0000","last_unstale":"2023-10-06T22:14:05.667361+0000","last_undegraded":"2023-10-06T22:14:05.667361+0000","last_fullsized":"2023-10-06T22:14:05.667361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:26:22.958348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723709+0000","last_change":"2023-10-06T22:12:42.698586+0000","last_active":"2023-10-06T22:13:05.723709+0000","last_peered":"2023-10-06T22:13:05.723709+0000","last_clean":"2023-10-06T22:13:05.723709+0000","last_became_active":"2023-10-06T22:12:42.698198+0000","last_became_peered":"2023-10-06T22:12:42.698198+0000","last_unstale":"2023-10-06T22:13:05.723709+0000","last_undegraded":"2023-10-06T22:13:05.723709+0000","last_fullsized":"2023-10-06T22:13:05.723709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:03:43.898987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671926+0000","last_change":"2023-10-06T22:12:43.698615+0000","last_active":"2023-10-06T22:14:05.671926+0000","last_peered":"2023-10-06T22:14:05.671926+0000","last_clean":"2023-10-06T22:14:05.671926+0000","last_became_active":"2023-10-06T22:12:43.698302+0000","last_became_peered":"2023-10-06T22:12:43.698302+0000","last_unstale":"2023-10-06T22:14:05.671926+0000","last_undegraded":"2023-10-06T22:14:05.671926+0000","last_fullsized":"2023-10-06T22:14:05.671926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:35.973119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422877+0000","last_change":"2023-10-06T22:12:42.700325+0000","last_active":"2023-10-06T22:13:06.422877+0000","last_peered":"2023-10-06T22:13:06.422877+0000","last_clean":"2023-10-06T22:13:06.422877+0000","last_became_active":"2023-10-06T22:12:42.699426+0000","last_became_peered":"2023-10-06T22:12:42.699426+0000","last_unstale":"2023-10-06T22:13:06.422877+0000","last_undegraded":"2023-10-06T22:13:06.422877+0000","last_fullsized":"2023-10-06T22:13:06.422877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:09.649737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667113+0000","last_change":"2023-10-06T22:12:43.698620+0000","last_active":"2023-10-06T22:14:05.667113+0000","last_peered":"2023-10-06T22:14:05.667113+0000","last_clean":"2023-10-06T22:14:05.667113+0000","last_became_active":"2023-10-06T22:12:43.698335+0000","last_became_peered":"2023-10-06T22:12:43.698335+0000","last_unstale":"2023-10-06T22:14:05.667113+0000","last_undegraded":"2023-10-06T22:14:05.667113+0000","last_fullsized":"2023-10-06T22:14:05.667113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:49.222848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669528+0000","last_change":"2023-10-06T22:12:43.700477+0000","last_active":"2023-10-06T22:14:05.669528+0000","last_peered":"2023-10-06T22:14:05.669528+0000","last_clean":"2023-10-06T22:14:05.669528+0000","last_became_active":"2023-10-06T22:12:43.700177+0000","last_became_peered":"2023-10-06T22:12:43.700177+0000","last_unstale":"2023-10-06T22:14:05.669528+0000","last_undegraded":"2023-10-06T22:14:05.669528+0000","last_fullsized":"2023-10-06T22:14:05.669528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:11.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664348+0000","last_change":"2023-10-06T22:12:43.701792+0000","last_active":"2023-10-06T22:14:05.664348+0000","last_peered":"2023-10-06T22:14:05.664348+0000","last_clean":"2023-10-06T22:14:05.664348+0000","last_became_active":"2023-10-06T22:12:43.701316+0000","last_became_peered":"2023-10-06T22:12:43.701316+0000","last_unstale":"2023-10-06T22:14:05.664348+0000","last_undegraded":"2023-10-06T22:14:05.664348+0000","last_fullsized":"2023-10-06T22:14:05.664348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:03.085219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677454+0000","last_change":"2023-10-06T22:12:43.707871+0000","last_active":"2023-10-06T22:14:05.677454+0000","last_peered":"2023-10-06T22:14:05.677454+0000","last_clean":"2023-10-06T22:14:05.677454+0000","last_became_active":"2023-10-06T22:12:43.707659+0000","last_became_peered":"2023-10-06T22:12:43.707659+0000","last_unstale":"2023-10-06T22:14:05.677454+0000","last_undegraded":"2023-10-06T22:14:05.677454+0000","last_fullsized":"2023-10-06T22:14:05.677454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:43.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.673925+0000","last_change":"2023-10-06T22:12:43.697701+0000","last_active":"2023-10-06T22:14:05.673925+0000","last_peered":"2023-10-06T22:14:05.673925+0000","last_clean":"2023-10-06T22:14:05.673925+0000","last_became_active":"2023-10-06T22:12:43.697361+0000","last_became_peered":"2023-10-06T22:12:43.697361+0000","last_unstale":"2023-10-06T22:14:05.673925+0000","last_undegraded":"2023-10-06T22:14:05.673925+0000","last_fullsized":"2023-10-06T22:14:05.673925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:14:53.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679653+0000","last_change":"2023-10-06T22:12:43.700058+0000","last_active":"2023-10-06T22:14:05.679653+0000","last_peered":"2023-10-06T22:14:05.679653+0000","last_clean":"2023-10-06T22:14:05.679653+0000","last_became_active":"2023-10-06T22:12:43.697893+0000","last_became_peered":"2023-10-06T22:12:43.697893+0000","last_unstale":"2023-10-06T22:14:05.679653+0000","last_undegraded":"2023-10-06T22:14:05.679653+0000","last_fullsized":"2023-10-06T22:14:05.679653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:01:37.518096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664805+0000","last_change":"2023-10-06T22:13:06.729307+0000","last_active":"2023-10-06T22:14:05.664805+0000","last_peered":"2023-10-06T22:14:05.664805+0000","last_clean":"2023-10-06T22:14:05.664805+0000","last_became_active":"2023-10-06T22:13:06.728877+0000","last_became_peered":"2023-10-06T22:13:06.728877+0000","last_unstale":"2023-10-06T22:14:05.664805+0000","last_undegraded":"2023-10-06T22:14:05.664805+0000","last_fullsized":"2023-10-06T22:14:05.664805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:17.629046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666149+0000","last_change":"2023-10-06T22:12:43.700668+0000","last_active":"2023-10-06T22:14:05.666149+0000","last_peered":"2023-10-06T22:14:05.666149+0000","last_clean":"2023-10-06T22:14:05.666149+0000","last_became_active":"2023-10-06T22:12:43.700387+0000","last_became_peered":"2023-10-06T22:12:43.700387+0000","last_unstale":"2023-10-06T22:14:05.666149+0000","last_undegraded":"2023-10-06T22:14:05.666149+0000","last_fullsized":"2023-10-06T22:14:05.666149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:36:00.053496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.662653+0000","last_change":"2023-10-06T22:12:43.702658+0000","last_active":"2023-10-06T22:14:05.662653+0000","last_peered":"2023-10-06T22:14:05.662653+0000","last_clean":"2023-10-06T22:14:05.662653+0000","last_became_active":"2023-10-06T22:12:43.702264+0000","last_became_peered":"2023-10-06T22:12:43.702264+0000","last_unstale":"2023-10-06T22:14:05.662653+0000","last_undegraded":"2023-10-06T22:14:05.662653+0000","last_fullsized":"2023-10-06T22:14:05.662653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:56.472029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671729+0000","last_change":"2023-10-06T22:12:43.705718+0000","last_active":"2023-10-06T22:14:05.671729+0000","last_peered":"2023-10-06T22:14:05.671729+0000","last_clean":"2023-10-06T22:14:05.671729+0000","last_became_active":"2023-10-06T22:12:43.705479+0000","last_became_peered":"2023-10-06T22:12:43.705479+0000","last_unstale":"2023-10-06T22:14:05.671729+0000","last_undegraded":"2023-10-06T22:14:05.671729+0000","last_fullsized":"2023-10-06T22:14:05.671729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:55:36.393577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672014+0000","last_change":"2023-10-06T22:12:43.701891+0000","last_active":"2023-10-06T22:14:05.672014+0000","last_peered":"2023-10-06T22:14:05.672014+0000","last_clean":"2023-10-06T22:14:05.672014+0000","last_became_active":"2023-10-06T22:12:43.701535+0000","last_became_peered":"2023-10-06T22:12:43.701535+0000","last_unstale":"2023-10-06T22:14:05.672014+0000","last_undegraded":"2023-10-06T22:14:05.672014+0000","last_fullsized":"2023-10-06T22:14:05.672014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:17:40.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667438+0000","last_change":"2023-10-06T22:12:43.707075+0000","last_active":"2023-10-06T22:14:05.667438+0000","last_peered":"2023-10-06T22:14:05.667438+0000","last_clean":"2023-10-06T22:14:05.667438+0000","last_became_active":"2023-10-06T22:12:43.706723+0000","last_became_peered":"2023-10-06T22:12:43.706723+0000","last_unstale":"2023-10-06T22:14:05.667438+0000","last_undegraded":"2023-10-06T22:14:05.667438+0000","last_fullsized":"2023-10-06T22:14:05.667438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:18.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679749+0000","last_change":"2023-10-06T22:12:43.704656+0000","last_active":"2023-10-06T22:14:05.679749+0000","last_peered":"2023-10-06T22:14:05.679749+0000","last_clean":"2023-10-06T22:14:05.679749+0000","last_became_active":"2023-10-06T22:12:43.704312+0000","last_became_peered":"2023-10-06T22:12:43.704312+0000","last_unstale":"2023-10-06T22:14:05.679749+0000","last_undegraded":"2023-10-06T22:14:05.679749+0000","last_fullsized":"2023-10-06T22:14:05.679749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:21:37.951180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663935+0000","last_change":"2023-10-06T22:12:43.702239+0000","last_active":"2023-10-06T22:14:05.663935+0000","last_peered":"2023-10-06T22:14:05.663935+0000","last_clean":"2023-10-06T22:14:05.663935+0000","last_became_active":"2023-10-06T22:12:43.701851+0000","last_became_peered":"2023-10-06T22:12:43.701851+0000","last_unstale":"2023-10-06T22:14:05.663935+0000","last_undegraded":"2023-10-06T22:14:05.663935+0000","last_fullsized":"2023-10-06T22:14:05.663935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:40:53.661587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66404,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1014,"num_read_kb":575,"num_write":245,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449122,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":444759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72099999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.38100000000000001}]},{"osd":1,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.371,"15min":0.371},"max":{"1min":0.85299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.34200000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61899999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.48799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69799999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54500000000000004,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.68400000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.69299999999999995},"min":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.74199999999999999,"5min":1.044,"15min":1.044},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.56599999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54100000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86099999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.61199999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77100000000000002}]},{"osd":9,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.62,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":1.472,"15min":1.472},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.63500000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0109999999999999,"5min":1.151,"15min":1.151},"last":0.83399999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.68300000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82499999999999996,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.64800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85299999999999998,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.751}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1093852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59699999999999998,"5min":0.748,"15min":0.748},"last":0.44}]},{"osd":2,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":1.026,"15min":1.026},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96099999999999997,"5min":1.093,"15min":1.093},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.626},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78000000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81699999999999995,"5min":1.079,"15min":1.079},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.266,"15min":1.266},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81200000000000006},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.376},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.505}]},{"osd":8,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90200000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.751},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95199999999999996,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.81699999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.88800000000000001,"15min":0.88400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.72699999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97099999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.94899999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86099999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":444158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89400000000000002,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.747,"15min":0.77000000000000002},"min":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.036},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.83599999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96599999999999997,"5min":1.486,"15min":1.486},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91500000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.038,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72299999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.73899999999999999,"15min":0.76700000000000002},"min":{"1min":0.436,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93799999999999994,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.82799999999999996,"15min":0.871},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0629999999999999,"5min":1.548,"15min":1.548},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.871,"15min":0.91700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94699999999999995,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.73799999999999999,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89900000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.90200000000000002,"15min":0.94199999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90700000000000003,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.96099999999999997,"15min":1.0189999999999999},"min":{"1min":0.63500000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1259999999999999,"5min":1.585,"15min":1.585},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83899999999999997,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.95499999999999996}]},{"osd":8,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.96799999999999997,"15min":1.006},"min":{"1min":0.71199999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.167,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.86899999999999999,"15min":0.90100000000000002},"min":{"1min":0.63600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.02,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.194,"5min":1.268,"15min":1.268},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.997,"15min":1.018},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.878}]},{"osd":11,"last update":"Fri Oct 6 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.90900000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78800000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45600000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.46899999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89200000000000002,"5min":1.018,"15min":1.018},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.53500000000000003},"min":{"1min":0.49199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.434,"15min":0.41199999999999998},"min":{"1min":0.33100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.378},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46999999999999997,"15min":0.45800000000000002},"min":{"1min":0.39300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.52900000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.63900000000000001,"15min":0.61099999999999999},"min":{"1min":0.60199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.42699999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.64400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.57899999999999996,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.873,"5min":1.073,"15min":1.073},"last":0.745},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.65800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92500000000000004,"5min":1.502,"15min":1.502},"last":0.624}]},{"osd":9,"last update":"Fri Oct 6 22:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.68400000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.78300000000000003,"15min":0.76500000000000001},"min":{"1min":0.77100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.63800000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.754,"15min":0.72999999999999998},"min":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.72999999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.505}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.33600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.374},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44,"15min":0.436},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.627,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65700000000000003,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.875,"15min":0.875},"last":0.35299999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.51500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.438}]},{"osd":8,"last update":"Fri Oct 6 22:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.745},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0169999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.76400000000000001},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.089,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.753}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86899999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72899999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.77800000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81399999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68600000000000005,"15min":0.66300000000000003},"min":{"1min":0.53200000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92700000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93000000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86299999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60499999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.75}]},{"osd":4,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73799999999999999,"15min":0.748},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.754},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.004,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.02,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79300000000000004,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99399999999999999,"5min":1.173,"15min":1.173},"last":0.875},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83899999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97699999999999998,"5min":1.145,"15min":1.145},"last":0.88700000000000001}]},{"osd":8,"last update":"Fri Oct 6 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.59399999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.1080000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85299999999999998,"15min":0.86199999999999999},"min":{"1min":0.64900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.137,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.91800000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.90400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.92300000000000004,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1599999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93700000000000006,"15min":0.93999999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.97799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.441,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.435,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55900000000000005,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56799999999999995},"min":{"1min":0.46200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84699999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63700000000000001,"15min":0.58199999999999996},"min":{"1min":0.47899999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.754}]},{"osd":4,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.443,"15min":0.39700000000000002},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48899999999999999,"15min":0.433},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.433}]},{"osd":5,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.58699999999999997,"15min":0.53700000000000003},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.60699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.59799999999999998},"min":{"1min":0.53800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95099999999999996,"5min":1.45,"15min":1.45},"last":0.625},{"interface":"front","average":{"1min":0.753,"5min":0.65100000000000002,"15min":0.60099999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55700000000000005,"15min":0.53400000000000003},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Oct 6 22:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.56000000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.93200000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.81699999999999995,"15min":0.79000000000000004},"min":{"1min":0.66000000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.63700000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94799999999999995,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.87}]},{"osd":10,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.74199999999999999,"15min":0.72799999999999998},"min":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Oct 6 22:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.84099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":445141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60299999999999998,"15min":0.52500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.56799999999999995},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.62}]},{"osd":3,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.67200000000000004,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.59199999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.73399999999999999,"15min":0.68899999999999995},"min":{"1min":0.59299999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0669999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.97399999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.86899999999999999},"min":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.121,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.64900000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.745},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79000000000000004}]},{"osd":9,"last update":"Fri Oct 6 22:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.012,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.872,"5min":0.88600000000000001,"15min":0.89900000000000002},"min":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.048,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.877,"15min":0.89300000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93300000000000005,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.054,"5min":1.103,"15min":1.103},"last":0.82799999999999996}]},{"osd":11,"last update":"Fri Oct 6 22:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.86199999999999999,"15min":0.85699999999999998},"min":{"1min":0.67300000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0669999999999999,"5min":1.538,"15min":1.538},"last":0.876}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":503812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.125},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57199999999999995,"15min":0.51300000000000001},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.873,"15min":0.873},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.52300000000000002},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 22:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71599999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44900000000000001,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.71199999999999997,"15min":0.66900000000000004},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64500000000000002,"15min":0.61899999999999999},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78900000000000003,"5min":1.071,"15min":1.071},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":1.002,"15min":1.002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.629},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85299999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.58099999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92200000000000004,"5min":1.633,"15min":1.633},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.878,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72199999999999998}]},{"osd":10,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.747},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.83599999999999997}]},{"osd":11,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.75700000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":443439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.40100000000000002},"min":{"1min":0.34999999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.41699999999999998},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.60999999999999999},"min":{"1min":0.48999999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92500000000000004,"5min":1.069,"15min":1.069},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.61799999999999999},"min":{"1min":0.52800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.60799999999999998},"min":{"1min":0.48899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.74299999999999999,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.0109999999999999,"5min":1.635,"15min":1.635},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71199999999999997,"15min":0.629},"min":{"1min":0.501,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68200000000000005,"15min":0.64800000000000002},"min":{"1min":0.53400000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.71099999999999997},"min":{"1min":0.57999999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Oct 6 22:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4019999999999999,"5min":1.794,"15min":1.794},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.163,"5min":1.224,"15min":1.224},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76000000000000001,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0509999999999999,"15min":1.133},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.778,"5min":7.0529999999999999,"15min":7.0529999999999999},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Oct 6 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88400000000000001,"15min":0.91200000000000003},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86599999999999999,"15min":0.88400000000000001},"min":{"1min":0.49299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.448,"5min":1.748,"15min":1.748},"last":0.85599999999999998}]},{"osd":10,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.94899999999999995,"15min":0.95399999999999996},"min":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.0229999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.37},"min":{"1min":0.28999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.436},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.39000000000000001},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.61399999999999999,"15min":0.56499999999999995},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.56299999999999994},"min":{"1min":0.53400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.57999999999999996},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86599999999999999,"5min":1.365,"15min":1.365},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.48199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.879,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59299999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82199999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89300000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 22:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66900000000000004,"15min":0.626},"min":{"1min":0.45200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.68600000000000005,"15min":0.64100000000000001},"min":{"1min":0.55000000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.41799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.018,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51400000000000001}]},{"osd":8,"last update":"Fri Oct 6 22:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.89200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.877},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81299999999999994,"5min":0.877,"15min":0.877},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79400000000000004,"15min":0.78600000000000003},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.89700000000000002,"5min":1.054,"15min":1.054},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1036878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.752,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73299999999999998,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83899999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78000000000000003,"5min":2.802,"15min":2.802},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.308,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.67700000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.42199999999999999},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64600000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.499},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80900000000000005,"5min":1.214,"15min":1.214},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79700000000000004,"5min":2.661,"15min":2.661},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.80200000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.629},"min":{"1min":0.50900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82399999999999995,"5min":1.274,"15min":1.274},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.60299999999999998},"min":{"1min":0.59299999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96599999999999997,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.78800000000000003}]},{"osd":8,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85699999999999998,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.87,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63500000000000001}]},{"osd":9,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80000000000000004,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77700000000000002,"15min":0.77800000000000002},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.89500000000000002,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.65400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":2.774,"15min":2.774},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72499999999999998,"15min":0.71099999999999997},"min":{"1min":0.63800000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94199999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T22:14:32.243 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-06T22:14:32.244 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T22:14:32.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 -- 172.21.15.57:0/3180799946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102080 msgr2=0x7efdd8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 --2- 172.21.15.57:0/3180799946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102080 0x7efdd8102500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009990 tx=0x7efdc4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 -- 172.21.15.57:0/3180799946 shutdown_connections 2023-10-06T22:14:32.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 --2- 172.21.15.57:0/3180799946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd8102a40 0x7efdd8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 --2- 172.21.15.57:0/3180799946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8100ea0 0x7efdd81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 --2- 172.21.15.57:0/3180799946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102080 0x7efdd8102500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 -- 172.21.15.57:0/3180799946 >> 172.21.15.57:0/3180799946 conn(0x7efdd80fc420 msgr2=0x7efdd80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:32.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 -- 172.21.15.57:0/3180799946 shutdown_connections 2023-10-06T22:14:32.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.494+0000 7efddefff700 1 -- 172.21.15.57:0/3180799946 wait complete. 2023-10-06T22:14:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 Processor -- start 2023-10-06T22:14:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 -- start start 2023-10-06T22:14:32.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd8100ea0 0x7efdd8198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8102080 0x7efdd8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102a40 0x7efdd81a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd81a0900 con 0x7efdd8102080 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd81a0a70 con 0x7efdd8102a40 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.495+0000 7efddefff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd81a0c30 con 0x7efdd8100ea0 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8102080 0x7efdd8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8102080 0x7efdd8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33690/0 (socket says 172.21.15.57:33690) 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 -- 172.21.15.57:0/843736287 learned_addr learned my addr 172.21.15.57:0/843736287 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efddcd9b700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd8100ea0 0x7efdd8198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efddd59c700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102a40 0x7efdd81a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 -- 172.21.15.57:0/843736287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd8100ea0 msgr2=0x7efdd8198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd8100ea0 0x7efdd8198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 -- 172.21.15.57:0/843736287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102a40 msgr2=0x7efdd81a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102a40 0x7efdd81a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 -- 172.21.15.57:0/843736287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc40095f0 con 0x7efdd8102080 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.496+0000 7efdd7fff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8102080 0x7efdd8199300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efdc4000c00 tx=0x7efdc4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.497+0000 7efdd5ffb700 1 -- 172.21.15.57:0/843736287 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc401d070 con 0x7efdd8102080 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.497+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd81a0eb0 con 0x7efdd8102080 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.497+0000 7efdd5ffb700 1 -- 172.21.15.57:0/843736287 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc4022470 con 0x7efdd8102080 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.497+0000 7efdd5ffb700 1 -- 172.21.15.57:0/843736287 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4017650 con 0x7efdd8102080 2023-10-06T22:14:32.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.497+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd81a1220 con 0x7efdd8102080 2023-10-06T22:14:32.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.498+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd8193500 con 0x7efdd8102080 2023-10-06T22:14:32.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.500+0000 7efdd5ffb700 1 -- 172.21.15.57:0/843736287 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efdc40225e0 con 0x7efdd8102080 2023-10-06T22:14:32.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.501+0000 7efdd5ffb700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdc80a14d0 0x7efdc80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.501+0000 7efdd5ffb700 1 -- 172.21.15.57:0/843736287 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7efdc40cdfb0 con 0x7efdd8102080 2023-10-06T22:14:32.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.501+0000 7efddcd9b700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdc80a14d0 0x7efdc80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.502+0000 7efddcd9b700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdc80a14d0 0x7efdc80a3990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efdd819a290 tx=0x7efdcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:32.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.505+0000 7efdd5ffb700 1 -- 172.21.15.57:0/843736287 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efdc405d100 con 0x7efdd8102080 2023-10-06T22:14:32.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.710+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdd802d250 con 0x7efdc80a14d0 2023-10-06T22:14:32.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.721+0000 7efdd5ffb700 1 -- 172.21.15.57:0/843736287 <== mgr.14150 v2:172.21.15.57:6800/409950778 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345234 (secure 0 0 0) 0x7efdd802d250 con 0x7efdc80a14d0 2023-10-06T22:14:32.728 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdc80a14d0 msgr2=0x7efdc80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdc80a14d0 0x7efdc80a3990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efdd819a290 tx=0x7efdcc009260 comp rx=0 tx=0).stop 2023-10-06T22:14:32.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8102080 msgr2=0x7efdd8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8102080 0x7efdd8199300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efdc4000c00 tx=0x7efdc4005720 comp rx=0 tx=0).stop 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 shutdown_connections 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7efdc80a14d0 0x7efdc80a3990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdd8102a40 0x7efdd81a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd8100ea0 0x7efdd8198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 --2- 172.21.15.57:0/843736287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd8102080 0x7efdd8199300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 >> 172.21.15.57:0/843736287 conn(0x7efdd80fc420 msgr2=0x7efdd8107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 shutdown_connections 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.726+0000 7efddefff700 1 -- 172.21.15.57:0/843736287 wait complete. 2023-10-06T22:14:32.732 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-06T22:14:32.748 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-10-06T22:14:31.647726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657018,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1150,"num_read_kb":694,"num_write":428,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337180,"kb_used_data":12988,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604644,"statfs":{"total":1151940427776,"available":1151595155456,"internally_reserved":0,"allocated":13299712,"data_stored":7285070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":52,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002915"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676898+0000","last_change":"2023-10-06T22:12:43.704484+0000","last_active":"2023-10-06T22:14:05.676898+0000","last_peered":"2023-10-06T22:14:05.676898+0000","last_clean":"2023-10-06T22:14:05.676898+0000","last_became_active":"2023-10-06T22:12:43.703740+0000","last_became_peered":"2023-10-06T22:12:43.703740+0000","last_unstale":"2023-10-06T22:14:05.676898+0000","last_undegraded":"2023-10-06T22:14:05.676898+0000","last_fullsized":"2023-10-06T22:14:05.676898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:25.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672032+0000","last_change":"2023-10-06T22:12:43.696968+0000","last_active":"2023-10-06T22:14:05.672032+0000","last_peered":"2023-10-06T22:14:05.672032+0000","last_clean":"2023-10-06T22:14:05.672032+0000","last_became_active":"2023-10-06T22:12:43.696532+0000","last_became_peered":"2023-10-06T22:12:43.696532+0000","last_unstale":"2023-10-06T22:14:05.672032+0000","last_undegraded":"2023-10-06T22:14:05.672032+0000","last_fullsized":"2023-10-06T22:14:05.672032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:59:50.754606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675300+0000","last_change":"2023-10-06T22:12:43.700845+0000","last_active":"2023-10-06T22:14:05.675300+0000","last_peered":"2023-10-06T22:14:05.675300+0000","last_clean":"2023-10-06T22:14:05.675300+0000","last_became_active":"2023-10-06T22:12:43.700531+0000","last_became_peered":"2023-10-06T22:12:43.700531+0000","last_unstale":"2023-10-06T22:14:05.675300+0000","last_undegraded":"2023-10-06T22:14:05.675300+0000","last_fullsized":"2023-10-06T22:14:05.675300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:18.518772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667765+0000","last_change":"2023-10-06T22:12:43.700806+0000","last_active":"2023-10-06T22:14:05.667765+0000","last_peered":"2023-10-06T22:14:05.667765+0000","last_clean":"2023-10-06T22:14:05.667765+0000","last_became_active":"2023-10-06T22:12:43.699901+0000","last_became_peered":"2023-10-06T22:12:43.699901+0000","last_unstale":"2023-10-06T22:14:05.667765+0000","last_undegraded":"2023-10-06T22:14:05.667765+0000","last_fullsized":"2023-10-06T22:14:05.667765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:41:35.906415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.674088+0000","last_change":"2023-10-06T22:12:43.694776+0000","last_active":"2023-10-06T22:14:05.674088+0000","last_peered":"2023-10-06T22:14:05.674088+0000","last_clean":"2023-10-06T22:14:05.674088+0000","last_became_active":"2023-10-06T22:12:43.694397+0000","last_became_peered":"2023-10-06T22:12:43.694397+0000","last_unstale":"2023-10-06T22:14:05.674088+0000","last_undegraded":"2023-10-06T22:14:05.674088+0000","last_fullsized":"2023-10-06T22:14:05.674088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:47:43.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665097+0000","last_change":"2023-10-06T22:12:43.703515+0000","last_active":"2023-10-06T22:14:05.665097+0000","last_peered":"2023-10-06T22:14:05.665097+0000","last_clean":"2023-10-06T22:14:05.665097+0000","last_became_active":"2023-10-06T22:12:43.703094+0000","last_became_peered":"2023-10-06T22:12:43.703094+0000","last_unstale":"2023-10-06T22:14:05.665097+0000","last_undegraded":"2023-10-06T22:14:05.665097+0000","last_fullsized":"2023-10-06T22:14:05.665097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:54:32.130131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667950+0000","last_change":"2023-10-06T22:12:43.697894+0000","last_active":"2023-10-06T22:14:05.667950+0000","last_peered":"2023-10-06T22:14:05.667950+0000","last_clean":"2023-10-06T22:14:05.667950+0000","last_became_active":"2023-10-06T22:12:43.697591+0000","last_became_peered":"2023-10-06T22:12:43.697591+0000","last_unstale":"2023-10-06T22:14:05.667950+0000","last_undegraded":"2023-10-06T22:14:05.667950+0000","last_fullsized":"2023-10-06T22:14:05.667950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:30.313447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668433+0000","last_change":"2023-10-06T22:13:06.927879+0000","last_active":"2023-10-06T22:14:05.668433+0000","last_peered":"2023-10-06T22:14:05.668433+0000","last_clean":"2023-10-06T22:14:05.668433+0000","last_became_active":"2023-10-06T22:13:06.927559+0000","last_became_peered":"2023-10-06T22:13:06.927559+0000","last_unstale":"2023-10-06T22:14:05.668433+0000","last_undegraded":"2023-10-06T22:14:05.668433+0000","last_fullsized":"2023-10-06T22:14:05.668433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:51:15.874159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664732+0000","last_change":"2023-10-06T22:12:43.696995+0000","last_active":"2023-10-06T22:14:05.664732+0000","last_peered":"2023-10-06T22:14:05.664732+0000","last_clean":"2023-10-06T22:14:05.664732+0000","last_became_active":"2023-10-06T22:12:43.696731+0000","last_became_peered":"2023-10-06T22:12:43.696731+0000","last_unstale":"2023-10-06T22:14:05.664732+0000","last_undegraded":"2023-10-06T22:14:05.664732+0000","last_fullsized":"2023-10-06T22:14:05.664732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:43:31.261922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663417+0000","last_change":"2023-10-06T22:12:43.704391+0000","last_active":"2023-10-06T22:14:05.663417+0000","last_peered":"2023-10-06T22:14:05.663417+0000","last_clean":"2023-10-06T22:14:05.663417+0000","last_became_active":"2023-10-06T22:12:43.702954+0000","last_became_peered":"2023-10-06T22:12:43.702954+0000","last_unstale":"2023-10-06T22:14:05.663417+0000","last_undegraded":"2023-10-06T22:14:05.663417+0000","last_fullsized":"2023-10-06T22:14:05.663417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:34:38.973781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666760+0000","last_change":"2023-10-06T22:12:43.696265+0000","last_active":"2023-10-06T22:14:05.666760+0000","last_peered":"2023-10-06T22:14:05.666760+0000","last_clean":"2023-10-06T22:14:05.666760+0000","last_became_active":"2023-10-06T22:12:43.695939+0000","last_became_peered":"2023-10-06T22:12:43.695939+0000","last_unstale":"2023-10-06T22:14:05.666760+0000","last_undegraded":"2023-10-06T22:14:05.666760+0000","last_fullsized":"2023-10-06T22:14:05.666760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:22:32.882083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668888+0000","last_change":"2023-10-06T22:13:06.812700+0000","last_active":"2023-10-06T22:14:05.668888+0000","last_peered":"2023-10-06T22:14:05.668888+0000","last_clean":"2023-10-06T22:14:05.668888+0000","last_became_active":"2023-10-06T22:13:06.812313+0000","last_became_peered":"2023-10-06T22:13:06.812313+0000","last_unstale":"2023-10-06T22:14:05.668888+0000","last_undegraded":"2023-10-06T22:14:05.668888+0000","last_fullsized":"2023-10-06T22:14:05.668888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:25.010233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676269+0000","last_change":"2023-10-06T22:12:43.694717+0000","last_active":"2023-10-06T22:14:05.676269+0000","last_peered":"2023-10-06T22:14:05.676269+0000","last_clean":"2023-10-06T22:14:05.676269+0000","last_became_active":"2023-10-06T22:12:43.694321+0000","last_became_peered":"2023-10-06T22:12:43.694321+0000","last_unstale":"2023-10-06T22:14:05.676269+0000","last_undegraded":"2023-10-06T22:14:05.676269+0000","last_fullsized":"2023-10-06T22:14:05.676269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:06.334726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669546+0000","last_change":"2023-10-06T22:12:43.703770+0000","last_active":"2023-10-06T22:14:05.669546+0000","last_peered":"2023-10-06T22:14:05.669546+0000","last_clean":"2023-10-06T22:14:05.669546+0000","last_became_active":"2023-10-06T22:12:43.703434+0000","last_became_peered":"2023-10-06T22:12:43.703434+0000","last_unstale":"2023-10-06T22:14:05.669546+0000","last_undegraded":"2023-10-06T22:14:05.669546+0000","last_fullsized":"2023-10-06T22:14:05.669546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:37:25.894280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676822+0000","last_change":"2023-10-06T22:12:43.700994+0000","last_active":"2023-10-06T22:14:05.676822+0000","last_peered":"2023-10-06T22:14:05.676822+0000","last_clean":"2023-10-06T22:14:05.676822+0000","last_became_active":"2023-10-06T22:12:43.700652+0000","last_became_peered":"2023-10-06T22:12:43.700652+0000","last_unstale":"2023-10-06T22:14:05.676822+0000","last_undegraded":"2023-10-06T22:14:05.676822+0000","last_fullsized":"2023-10-06T22:14:05.676822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:54.855597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676182+0000","last_change":"2023-10-06T22:12:43.702099+0000","last_active":"2023-10-06T22:14:05.676182+0000","last_peered":"2023-10-06T22:14:05.676182+0000","last_clean":"2023-10-06T22:14:05.676182+0000","last_became_active":"2023-10-06T22:12:43.701802+0000","last_became_peered":"2023-10-06T22:12:43.701802+0000","last_unstale":"2023-10-06T22:14:05.676182+0000","last_undegraded":"2023-10-06T22:14:05.676182+0000","last_fullsized":"2023-10-06T22:14:05.676182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:00.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677168+0000","last_change":"2023-10-06T22:12:43.704679+0000","last_active":"2023-10-06T22:14:05.677168+0000","last_peered":"2023-10-06T22:14:05.677168+0000","last_clean":"2023-10-06T22:14:05.677168+0000","last_became_active":"2023-10-06T22:12:43.704186+0000","last_became_peered":"2023-10-06T22:12:43.704186+0000","last_unstale":"2023-10-06T22:14:05.677168+0000","last_undegraded":"2023-10-06T22:14:05.677168+0000","last_fullsized":"2023-10-06T22:14:05.677168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:28.665217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.668666+0000","last_change":"2023-10-06T22:12:43.695699+0000","last_active":"2023-10-06T22:14:05.668666+0000","last_peered":"2023-10-06T22:14:05.668666+0000","last_clean":"2023-10-06T22:14:05.668666+0000","last_became_active":"2023-10-06T22:12:43.695359+0000","last_became_peered":"2023-10-06T22:12:43.695359+0000","last_unstale":"2023-10-06T22:14:05.668666+0000","last_undegraded":"2023-10-06T22:14:05.668666+0000","last_fullsized":"2023-10-06T22:14:05.668666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:46.296768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672411+0000","last_change":"2023-10-06T22:12:43.707170+0000","last_active":"2023-10-06T22:14:05.672411+0000","last_peered":"2023-10-06T22:14:05.672411+0000","last_clean":"2023-10-06T22:14:05.672411+0000","last_became_active":"2023-10-06T22:12:43.706875+0000","last_became_peered":"2023-10-06T22:12:43.706875+0000","last_unstale":"2023-10-06T22:14:05.672411+0000","last_undegraded":"2023-10-06T22:14:05.672411+0000","last_fullsized":"2023-10-06T22:14:05.672411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:46.548458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":147,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:24.833047+0000","last_change":"2023-10-06T22:13:06.736382+0000","last_active":"2023-10-06T22:14:24.833047+0000","last_peered":"2023-10-06T22:14:24.833047+0000","last_clean":"2023-10-06T22:14:24.833047+0000","last_became_active":"2023-10-06T22:13:06.725703+0000","last_became_peered":"2023-10-06T22:13:06.725703+0000","last_unstale":"2023-10-06T22:14:24.833047+0000","last_undegraded":"2023-10-06T22:14:24.833047+0000","last_fullsized":"2023-10-06T22:14:24.833047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:26:10.056384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671657+0000","last_change":"2023-10-06T22:12:43.702574+0000","last_active":"2023-10-06T22:14:05.671657+0000","last_peered":"2023-10-06T22:14:05.671657+0000","last_clean":"2023-10-06T22:14:05.671657+0000","last_became_active":"2023-10-06T22:12:43.702238+0000","last_became_peered":"2023-10-06T22:12:43.702238+0000","last_unstale":"2023-10-06T22:14:05.671657+0000","last_undegraded":"2023-10-06T22:14:05.671657+0000","last_fullsized":"2023-10-06T22:14:05.671657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:34:09.975301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.752842+0000","last_change":"2023-10-06T22:12:42.697256+0000","last_active":"2023-10-06T22:13:05.752842+0000","last_peered":"2023-10-06T22:13:05.752842+0000","last_clean":"2023-10-06T22:13:05.752842+0000","last_became_active":"2023-10-06T22:12:42.696724+0000","last_became_peered":"2023-10-06T22:12:42.696724+0000","last_unstale":"2023-10-06T22:13:05.752842+0000","last_undegraded":"2023-10-06T22:13:05.752842+0000","last_fullsized":"2023-10-06T22:13:05.752842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:50:01.378292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679610+0000","last_change":"2023-10-06T22:12:43.702050+0000","last_active":"2023-10-06T22:14:05.679610+0000","last_peered":"2023-10-06T22:14:05.679610+0000","last_clean":"2023-10-06T22:14:05.679610+0000","last_became_active":"2023-10-06T22:12:43.701432+0000","last_became_peered":"2023-10-06T22:12:43.701432+0000","last_unstale":"2023-10-06T22:14:05.679610+0000","last_undegraded":"2023-10-06T22:14:05.679610+0000","last_fullsized":"2023-10-06T22:14:05.679610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:30:37.909775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.948160+0000","last_change":"2023-10-06T22:13:06.948160+0000","last_active":"2023-10-06T22:13:06.948160+0000","last_peered":"2023-10-06T22:13:06.948160+0000","last_clean":"2023-10-06T22:13:06.948160+0000","last_became_active":"2023-10-06T22:13:06.927995+0000","last_became_peered":"2023-10-06T22:13:06.927995+0000","last_unstale":"2023-10-06T22:13:06.948160+0000","last_undegraded":"2023-10-06T22:13:06.948160+0000","last_fullsized":"2023-10-06T22:13:06.948160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:33:27.974776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.655518+0000","last_change":"2023-10-06T22:12:43.696511+0000","last_active":"2023-10-06T22:14:05.655518+0000","last_peered":"2023-10-06T22:14:05.655518+0000","last_clean":"2023-10-06T22:14:05.655518+0000","last_became_active":"2023-10-06T22:12:43.696087+0000","last_became_peered":"2023-10-06T22:12:43.696087+0000","last_unstale":"2023-10-06T22:14:05.655518+0000","last_undegraded":"2023-10-06T22:14:05.655518+0000","last_fullsized":"2023-10-06T22:14:05.655518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:12:44.056664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724948+0000","last_change":"2023-10-06T22:12:42.694963+0000","last_active":"2023-10-06T22:13:05.724948+0000","last_peered":"2023-10-06T22:13:05.724948+0000","last_clean":"2023-10-06T22:13:05.724948+0000","last_became_active":"2023-10-06T22:12:42.694517+0000","last_became_peered":"2023-10-06T22:12:42.694517+0000","last_unstale":"2023-10-06T22:13:05.724948+0000","last_undegraded":"2023-10-06T22:13:05.724948+0000","last_fullsized":"2023-10-06T22:13:05.724948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:07:33.404328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663561+0000","last_change":"2023-10-06T22:12:43.698077+0000","last_active":"2023-10-06T22:14:05.663561+0000","last_peered":"2023-10-06T22:14:05.663561+0000","last_clean":"2023-10-06T22:14:05.663561+0000","last_became_active":"2023-10-06T22:12:43.697676+0000","last_became_peered":"2023-10-06T22:12:43.697676+0000","last_unstale":"2023-10-06T22:14:05.663561+0000","last_undegraded":"2023-10-06T22:14:05.663561+0000","last_fullsized":"2023-10-06T22:14:05.663561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:59:56.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808989+0000","last_change":"2023-10-06T22:12:42.698067+0000","last_active":"2023-10-06T22:14:27.808989+0000","last_peered":"2023-10-06T22:14:27.808989+0000","last_clean":"2023-10-06T22:14:27.808989+0000","last_became_active":"2023-10-06T22:12:42.697671+0000","last_became_peered":"2023-10-06T22:12:42.697671+0000","last_unstale":"2023-10-06T22:14:27.808989+0000","last_undegraded":"2023-10-06T22:14:27.808989+0000","last_fullsized":"2023-10-06T22:14:27.808989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:54:33.607513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.661778+0000","last_change":"2023-10-06T22:12:43.696213+0000","last_active":"2023-10-06T22:14:05.661778+0000","last_peered":"2023-10-06T22:14:05.661778+0000","last_clean":"2023-10-06T22:14:05.661778+0000","last_became_active":"2023-10-06T22:12:43.695925+0000","last_became_peered":"2023-10-06T22:12:43.695925+0000","last_unstale":"2023-10-06T22:14:05.661778+0000","last_undegraded":"2023-10-06T22:14:05.661778+0000","last_fullsized":"2023-10-06T22:14:05.661778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:24.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.719695+0000","last_change":"2023-10-06T22:12:42.687188+0000","last_active":"2023-10-06T22:13:05.719695+0000","last_peered":"2023-10-06T22:13:05.719695+0000","last_clean":"2023-10-06T22:13:05.719695+0000","last_became_active":"2023-10-06T22:12:42.686747+0000","last_became_peered":"2023-10-06T22:12:42.686747+0000","last_unstale":"2023-10-06T22:13:05.719695+0000","last_undegraded":"2023-10-06T22:13:05.719695+0000","last_fullsized":"2023-10-06T22:13:05.719695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:21.326911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664388+0000","last_change":"2023-10-06T22:12:43.697988+0000","last_active":"2023-10-06T22:14:05.664388+0000","last_peered":"2023-10-06T22:14:05.664388+0000","last_clean":"2023-10-06T22:14:05.664388+0000","last_became_active":"2023-10-06T22:12:43.697641+0000","last_became_peered":"2023-10-06T22:12:43.697641+0000","last_unstale":"2023-10-06T22:14:05.664388+0000","last_undegraded":"2023-10-06T22:14:05.664388+0000","last_fullsized":"2023-10-06T22:14:05.664388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:33.449914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.423261+0000","last_change":"2023-10-06T22:12:42.688882+0000","last_active":"2023-10-06T22:13:06.423261+0000","last_peered":"2023-10-06T22:13:06.423261+0000","last_clean":"2023-10-06T22:13:06.423261+0000","last_became_active":"2023-10-06T22:12:42.687653+0000","last_became_peered":"2023-10-06T22:12:42.687653+0000","last_unstale":"2023-10-06T22:13:06.423261+0000","last_undegraded":"2023-10-06T22:13:06.423261+0000","last_fullsized":"2023-10-06T22:13:06.423261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:06:18.146081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667479+0000","last_change":"2023-10-06T22:12:43.696559+0000","last_active":"2023-10-06T22:14:05.667479+0000","last_peered":"2023-10-06T22:14:05.667479+0000","last_clean":"2023-10-06T22:14:05.667479+0000","last_became_active":"2023-10-06T22:12:43.696252+0000","last_became_peered":"2023-10-06T22:12:43.696252+0000","last_unstale":"2023-10-06T22:14:05.667479+0000","last_undegraded":"2023-10-06T22:14:05.667479+0000","last_fullsized":"2023-10-06T22:14:05.667479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:31:36.097000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722785+0000","last_change":"2023-10-06T22:12:42.694299+0000","last_active":"2023-10-06T22:13:05.722785+0000","last_peered":"2023-10-06T22:13:05.722785+0000","last_clean":"2023-10-06T22:13:05.722785+0000","last_became_active":"2023-10-06T22:12:42.693820+0000","last_became_peered":"2023-10-06T22:12:42.693820+0000","last_unstale":"2023-10-06T22:13:05.722785+0000","last_undegraded":"2023-10-06T22:13:05.722785+0000","last_fullsized":"2023-10-06T22:13:05.722785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:13.837379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672816+0000","last_change":"2023-10-06T22:12:43.695234+0000","last_active":"2023-10-06T22:14:05.672816+0000","last_peered":"2023-10-06T22:14:05.672816+0000","last_clean":"2023-10-06T22:14:05.672816+0000","last_became_active":"2023-10-06T22:12:43.694889+0000","last_became_peered":"2023-10-06T22:12:43.694889+0000","last_unstale":"2023-10-06T22:14:05.672816+0000","last_undegraded":"2023-10-06T22:14:05.672816+0000","last_fullsized":"2023-10-06T22:14:05.672816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:51.039264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808525+0000","last_change":"2023-10-06T22:12:42.695438+0000","last_active":"2023-10-06T22:14:27.808525+0000","last_peered":"2023-10-06T22:14:27.808525+0000","last_clean":"2023-10-06T22:14:27.808525+0000","last_became_active":"2023-10-06T22:12:42.695090+0000","last_became_peered":"2023-10-06T22:12:42.695090+0000","last_unstale":"2023-10-06T22:14:27.808525+0000","last_undegraded":"2023-10-06T22:14:27.808525+0000","last_fullsized":"2023-10-06T22:14:27.808525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:25:34.385961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663101+0000","last_change":"2023-10-06T22:12:43.697459+0000","last_active":"2023-10-06T22:14:05.663101+0000","last_peered":"2023-10-06T22:14:05.663101+0000","last_clean":"2023-10-06T22:14:05.663101+0000","last_became_active":"2023-10-06T22:12:43.696405+0000","last_became_peered":"2023-10-06T22:12:43.696405+0000","last_unstale":"2023-10-06T22:14:05.663101+0000","last_undegraded":"2023-10-06T22:14:05.663101+0000","last_fullsized":"2023-10-06T22:14:05.663101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:16:30.928083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727798+0000","last_change":"2023-10-06T22:12:42.692151+0000","last_active":"2023-10-06T22:13:05.727798+0000","last_peered":"2023-10-06T22:13:05.727798+0000","last_clean":"2023-10-06T22:13:05.727798+0000","last_became_active":"2023-10-06T22:12:42.691780+0000","last_became_peered":"2023-10-06T22:12:42.691780+0000","last_unstale":"2023-10-06T22:13:05.727798+0000","last_undegraded":"2023-10-06T22:13:05.727798+0000","last_fullsized":"2023-10-06T22:13:05.727798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:49:35.697733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679194+0000","last_change":"2023-10-06T22:12:43.701754+0000","last_active":"2023-10-06T22:14:05.679194+0000","last_peered":"2023-10-06T22:14:05.679194+0000","last_clean":"2023-10-06T22:14:05.679194+0000","last_became_active":"2023-10-06T22:12:43.700881+0000","last_became_peered":"2023-10-06T22:12:43.700881+0000","last_unstale":"2023-10-06T22:14:05.679194+0000","last_undegraded":"2023-10-06T22:14:05.679194+0000","last_fullsized":"2023-10-06T22:14:05.679194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:35:21.988307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723824+0000","last_change":"2023-10-06T22:12:42.699751+0000","last_active":"2023-10-06T22:13:05.723824+0000","last_peered":"2023-10-06T22:13:05.723824+0000","last_clean":"2023-10-06T22:13:05.723824+0000","last_became_active":"2023-10-06T22:12:42.699321+0000","last_became_peered":"2023-10-06T22:12:42.699321+0000","last_unstale":"2023-10-06T22:13:05.723824+0000","last_undegraded":"2023-10-06T22:13:05.723824+0000","last_fullsized":"2023-10-06T22:13:05.723824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:35:25.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.660837+0000","last_change":"2023-10-06T22:12:43.702426+0000","last_active":"2023-10-06T22:14:05.660837+0000","last_peered":"2023-10-06T22:14:05.660837+0000","last_clean":"2023-10-06T22:14:05.660837+0000","last_became_active":"2023-10-06T22:12:43.702101+0000","last_became_peered":"2023-10-06T22:12:43.702101+0000","last_unstale":"2023-10-06T22:14:05.660837+0000","last_undegraded":"2023-10-06T22:14:05.660837+0000","last_fullsized":"2023-10-06T22:14:05.660837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:31:09.133941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":252,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.808520+0000","last_change":"2023-10-06T22:12:42.688815+0000","last_active":"2023-10-06T22:14:27.808520+0000","last_peered":"2023-10-06T22:14:27.808520+0000","last_clean":"2023-10-06T22:14:27.808520+0000","last_became_active":"2023-10-06T22:12:42.688447+0000","last_became_peered":"2023-10-06T22:12:42.688447+0000","last_unstale":"2023-10-06T22:14:27.808520+0000","last_undegraded":"2023-10-06T22:14:27.808520+0000","last_fullsized":"2023-10-06T22:14:27.808520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:26:25.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58073,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":204,"num_read_kb":159,"num_write":35,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666683+0000","last_change":"2023-10-06T22:12:43.698406+0000","last_active":"2023-10-06T22:14:05.666683+0000","last_peered":"2023-10-06T22:14:05.666683+0000","last_clean":"2023-10-06T22:14:05.666683+0000","last_became_active":"2023-10-06T22:12:43.697204+0000","last_became_peered":"2023-10-06T22:12:43.697204+0000","last_unstale":"2023-10-06T22:14:05.666683+0000","last_undegraded":"2023-10-06T22:14:05.666683+0000","last_fullsized":"2023-10-06T22:14:05.666683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:59.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727727+0000","last_change":"2023-10-06T22:12:42.691007+0000","last_active":"2023-10-06T22:13:05.727727+0000","last_peered":"2023-10-06T22:13:05.727727+0000","last_clean":"2023-10-06T22:13:05.727727+0000","last_became_active":"2023-10-06T22:12:42.690693+0000","last_became_peered":"2023-10-06T22:12:42.690693+0000","last_unstale":"2023-10-06T22:13:05.727727+0000","last_undegraded":"2023-10-06T22:13:05.727727+0000","last_fullsized":"2023-10-06T22:13:05.727727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:09:47.670091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671518+0000","last_change":"2023-10-06T22:12:43.705131+0000","last_active":"2023-10-06T22:14:05.671518+0000","last_peered":"2023-10-06T22:14:05.671518+0000","last_clean":"2023-10-06T22:14:05.671518+0000","last_became_active":"2023-10-06T22:12:43.704469+0000","last_became_peered":"2023-10-06T22:12:43.704469+0000","last_unstale":"2023-10-06T22:14:05.671518+0000","last_undegraded":"2023-10-06T22:14:05.671518+0000","last_fullsized":"2023-10-06T22:14:05.671518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:36:39.392274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723093+0000","last_change":"2023-10-06T22:12:42.698392+0000","last_active":"2023-10-06T22:13:05.723093+0000","last_peered":"2023-10-06T22:13:05.723093+0000","last_clean":"2023-10-06T22:13:05.723093+0000","last_became_active":"2023-10-06T22:12:42.698019+0000","last_became_peered":"2023-10-06T22:12:42.698019+0000","last_unstale":"2023-10-06T22:13:05.723093+0000","last_undegraded":"2023-10-06T22:13:05.723093+0000","last_fullsized":"2023-10-06T22:13:05.723093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:00.345300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677485+0000","last_change":"2023-10-06T22:12:43.693238+0000","last_active":"2023-10-06T22:14:05.677485+0000","last_peered":"2023-10-06T22:14:05.677485+0000","last_clean":"2023-10-06T22:14:05.677485+0000","last_became_active":"2023-10-06T22:12:43.692948+0000","last_became_peered":"2023-10-06T22:12:43.692948+0000","last_unstale":"2023-10-06T22:14:05.677485+0000","last_undegraded":"2023-10-06T22:14:05.677485+0000","last_fullsized":"2023-10-06T22:14:05.677485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:24:30.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.816392+0000","last_change":"2023-10-06T22:12:42.693213+0000","last_active":"2023-10-06T22:14:26.816392+0000","last_peered":"2023-10-06T22:14:26.816392+0000","last_clean":"2023-10-06T22:14:26.816392+0000","last_became_active":"2023-10-06T22:12:42.692894+0000","last_became_peered":"2023-10-06T22:12:42.692894+0000","last_unstale":"2023-10-06T22:14:26.816392+0000","last_undegraded":"2023-10-06T22:14:26.816392+0000","last_fullsized":"2023-10-06T22:14:26.816392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:40:21.159102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669294+0000","last_change":"2023-10-06T22:12:43.695527+0000","last_active":"2023-10-06T22:14:05.669294+0000","last_peered":"2023-10-06T22:14:05.669294+0000","last_clean":"2023-10-06T22:14:05.669294+0000","last_became_active":"2023-10-06T22:12:43.694364+0000","last_became_peered":"2023-10-06T22:12:43.694364+0000","last_unstale":"2023-10-06T22:14:05.669294+0000","last_undegraded":"2023-10-06T22:14:05.669294+0000","last_fullsized":"2023-10-06T22:14:05.669294+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723938+0000","last_change":"2023-10-06T22:12:42.697854+0000","last_active":"2023-10-06T22:13:05.723938+0000","last_peered":"2023-10-06T22:13:05.723938+0000","last_clean":"2023-10-06T22:13:05.723938+0000","last_became_active":"2023-10-06T22:12:42.697542+0000","last_became_peered":"2023-10-06T22:12:42.697542+0000","last_unstale":"2023-10-06T22:13:05.723938+0000","last_undegraded":"2023-10-06T22:13:05.723938+0000","last_fullsized":"2023-10-06T22:13:05.723938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:12:11.181398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665099+0000","last_change":"2023-10-06T22:12:43.698070+0000","last_active":"2023-10-06T22:14:05.665099+0000","last_peered":"2023-10-06T22:14:05.665099+0000","last_clean":"2023-10-06T22:14:05.665099+0000","last_became_active":"2023-10-06T22:12:43.697682+0000","last_became_peered":"2023-10-06T22:12:43.697682+0000","last_unstale":"2023-10-06T22:14:05.665099+0000","last_undegraded":"2023-10-06T22:14:05.665099+0000","last_fullsized":"2023-10-06T22:14:05.665099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:43.808866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724573+0000","last_change":"2023-10-06T22:12:42.691691+0000","last_active":"2023-10-06T22:13:05.724573+0000","last_peered":"2023-10-06T22:13:05.724573+0000","last_clean":"2023-10-06T22:13:05.724573+0000","last_became_active":"2023-10-06T22:12:42.691001+0000","last_became_peered":"2023-10-06T22:12:42.691001+0000","last_unstale":"2023-10-06T22:13:05.724573+0000","last_undegraded":"2023-10-06T22:13:05.724573+0000","last_fullsized":"2023-10-06T22:13:05.724573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:11:18.559570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.660998+0000","last_change":"2023-10-06T22:13:06.927620+0000","last_active":"2023-10-06T22:14:05.660998+0000","last_peered":"2023-10-06T22:14:05.660998+0000","last_clean":"2023-10-06T22:14:05.660998+0000","last_became_active":"2023-10-06T22:13:06.927206+0000","last_became_peered":"2023-10-06T22:13:06.927206+0000","last_unstale":"2023-10-06T22:14:05.660998+0000","last_undegraded":"2023-10-06T22:14:05.660998+0000","last_fullsized":"2023-10-06T22:14:05.660998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:50:30.780174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.726783+0000","last_change":"2023-10-06T22:13:06.726783+0000","last_active":"2023-10-06T22:13:06.726783+0000","last_peered":"2023-10-06T22:13:06.726783+0000","last_clean":"2023-10-06T22:13:06.726783+0000","last_became_active":"2023-10-06T22:13:06.726342+0000","last_became_peered":"2023-10-06T22:13:06.726342+0000","last_unstale":"2023-10-06T22:13:06.726783+0000","last_undegraded":"2023-10-06T22:13:06.726783+0000","last_fullsized":"2023-10-06T22:13:06.726783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:42.359819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672141+0000","last_change":"2023-10-06T22:12:43.702870+0000","last_active":"2023-10-06T22:14:05.672141+0000","last_peered":"2023-10-06T22:14:05.672141+0000","last_clean":"2023-10-06T22:14:05.672141+0000","last_became_active":"2023-10-06T22:12:43.702564+0000","last_became_peered":"2023-10-06T22:12:43.702564+0000","last_unstale":"2023-10-06T22:14:05.672141+0000","last_undegraded":"2023-10-06T22:14:05.672141+0000","last_fullsized":"2023-10-06T22:14:05.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:23:32.238268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.836399+0000","last_change":"2023-10-06T22:12:42.696603+0000","last_active":"2023-10-06T22:14:26.836399+0000","last_peered":"2023-10-06T22:14:26.836399+0000","last_clean":"2023-10-06T22:14:26.836399+0000","last_became_active":"2023-10-06T22:12:42.696313+0000","last_became_peered":"2023-10-06T22:12:42.696313+0000","last_unstale":"2023-10-06T22:14:26.836399+0000","last_undegraded":"2023-10-06T22:14:26.836399+0000","last_fullsized":"2023-10-06T22:14:26.836399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:15.719749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675740+0000","last_change":"2023-10-06T22:12:43.703356+0000","last_active":"2023-10-06T22:14:05.675740+0000","last_peered":"2023-10-06T22:14:05.675740+0000","last_clean":"2023-10-06T22:14:05.675740+0000","last_became_active":"2023-10-06T22:12:43.702670+0000","last_became_peered":"2023-10-06T22:12:43.702670+0000","last_unstale":"2023-10-06T22:14:05.675740+0000","last_undegraded":"2023-10-06T22:14:05.675740+0000","last_fullsized":"2023-10-06T22:14:05.675740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:15:05.681491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.665212+0000","last_change":"2023-10-06T22:12:43.699083+0000","last_active":"2023-10-06T22:14:05.665212+0000","last_peered":"2023-10-06T22:14:05.665212+0000","last_clean":"2023-10-06T22:14:05.665212+0000","last_became_active":"2023-10-06T22:12:43.698826+0000","last_became_peered":"2023-10-06T22:12:43.698826+0000","last_unstale":"2023-10-06T22:14:05.665212+0000","last_undegraded":"2023-10-06T22:14:05.665212+0000","last_fullsized":"2023-10-06T22:14:05.665212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:23:04.225060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'7","reported_seq":120,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:26.515215+0000","last_change":"2023-10-06T22:12:42.699873+0000","last_active":"2023-10-06T22:14:26.515215+0000","last_peered":"2023-10-06T22:14:26.515215+0000","last_clean":"2023-10-06T22:14:26.515215+0000","last_became_active":"2023-10-06T22:12:42.699476+0000","last_became_peered":"2023-10-06T22:12:42.699476+0000","last_unstale":"2023-10-06T22:14:26.515215+0000","last_undegraded":"2023-10-06T22:14:26.515215+0000","last_fullsized":"2023-10-06T22:14:26.515215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:13:27.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.658075+0000","last_change":"2023-10-06T22:12:43.697720+0000","last_active":"2023-10-06T22:14:05.658075+0000","last_peered":"2023-10-06T22:14:05.658075+0000","last_clean":"2023-10-06T22:14:05.658075+0000","last_became_active":"2023-10-06T22:12:43.697430+0000","last_became_peered":"2023-10-06T22:12:43.697430+0000","last_unstale":"2023-10-06T22:14:05.658075+0000","last_undegraded":"2023-10-06T22:14:05.658075+0000","last_fullsized":"2023-10-06T22:14:05.658075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:29.633012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722760+0000","last_change":"2023-10-06T22:12:42.690781+0000","last_active":"2023-10-06T22:13:05.722760+0000","last_peered":"2023-10-06T22:13:05.722760+0000","last_clean":"2023-10-06T22:13:05.722760+0000","last_became_active":"2023-10-06T22:12:42.690274+0000","last_became_peered":"2023-10-06T22:12:42.690274+0000","last_unstale":"2023-10-06T22:13:05.722760+0000","last_undegraded":"2023-10-06T22:13:05.722760+0000","last_fullsized":"2023-10-06T22:13:05.722760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:26.653182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.724529+0000","last_change":"2023-10-06T22:11:36.409669+0000","last_active":"2023-10-06T22:13:06.724529+0000","last_peered":"2023-10-06T22:13:06.724529+0000","last_clean":"2023-10-06T22:13:06.724529+0000","last_became_active":"2023-10-06T22:11:36.409197+0000","last_became_peered":"2023-10-06T22:11:36.409197+0000","last_unstale":"2023-10-06T22:13:06.724529+0000","last_undegraded":"2023-10-06T22:13:06.724529+0000","last_fullsized":"2023-10-06T22:13:06.724529+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:08:24.598743+0000","last_clean_scrub_stamp":"2023-10-06T22:08:24.598743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:50:55.982571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672700+0000","last_change":"2023-10-06T22:12:43.698617+0000","last_active":"2023-10-06T22:14:05.672700+0000","last_peered":"2023-10-06T22:14:05.672700+0000","last_clean":"2023-10-06T22:14:05.672700+0000","last_became_active":"2023-10-06T22:12:43.697942+0000","last_became_peered":"2023-10-06T22:12:43.697942+0000","last_unstale":"2023-10-06T22:14:05.672700+0000","last_undegraded":"2023-10-06T22:14:05.672700+0000","last_fullsized":"2023-10-06T22:14:05.672700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:38.075730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727545+0000","last_change":"2023-10-06T22:12:42.695553+0000","last_active":"2023-10-06T22:13:05.727545+0000","last_peered":"2023-10-06T22:13:05.727545+0000","last_clean":"2023-10-06T22:13:05.727545+0000","last_became_active":"2023-10-06T22:12:42.695209+0000","last_became_peered":"2023-10-06T22:12:42.695209+0000","last_unstale":"2023-10-06T22:13:05.727545+0000","last_undegraded":"2023-10-06T22:13:05.727545+0000","last_fullsized":"2023-10-06T22:13:05.727545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:10.387710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724270+0000","last_change":"2023-10-06T22:12:42.699606+0000","last_active":"2023-10-06T22:13:05.724270+0000","last_peered":"2023-10-06T22:13:05.724270+0000","last_clean":"2023-10-06T22:13:05.724270+0000","last_became_active":"2023-10-06T22:12:42.699222+0000","last_became_peered":"2023-10-06T22:12:42.699222+0000","last_unstale":"2023-10-06T22:13:05.724270+0000","last_undegraded":"2023-10-06T22:13:05.724270+0000","last_fullsized":"2023-10-06T22:13:05.724270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:52.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.675984+0000","last_change":"2023-10-06T22:12:43.697783+0000","last_active":"2023-10-06T22:14:05.675984+0000","last_peered":"2023-10-06T22:14:05.675984+0000","last_clean":"2023-10-06T22:14:05.675984+0000","last_became_active":"2023-10-06T22:12:43.697388+0000","last_became_peered":"2023-10-06T22:12:43.697388+0000","last_unstale":"2023-10-06T22:14:05.675984+0000","last_undegraded":"2023-10-06T22:14:05.675984+0000","last_fullsized":"2023-10-06T22:14:05.675984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:50.113458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.721888+0000","last_change":"2023-10-06T22:12:42.694180+0000","last_active":"2023-10-06T22:13:06.721888+0000","last_peered":"2023-10-06T22:13:06.721888+0000","last_clean":"2023-10-06T22:13:06.721888+0000","last_became_active":"2023-10-06T22:12:42.693861+0000","last_became_peered":"2023-10-06T22:12:42.693861+0000","last_unstale":"2023-10-06T22:13:06.721888+0000","last_undegraded":"2023-10-06T22:13:06.721888+0000","last_fullsized":"2023-10-06T22:13:06.721888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:28:02.511117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664374+0000","last_change":"2023-10-06T22:12:43.699118+0000","last_active":"2023-10-06T22:14:05.664374+0000","last_peered":"2023-10-06T22:14:05.664374+0000","last_clean":"2023-10-06T22:14:05.664374+0000","last_became_active":"2023-10-06T22:12:43.698540+0000","last_became_peered":"2023-10-06T22:12:43.698540+0000","last_unstale":"2023-10-06T22:14:05.664374+0000","last_undegraded":"2023-10-06T22:14:05.664374+0000","last_fullsized":"2023-10-06T22:14:05.664374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:31.192762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.722961+0000","last_change":"2023-10-06T22:12:42.690617+0000","last_active":"2023-10-06T22:13:05.722961+0000","last_peered":"2023-10-06T22:13:05.722961+0000","last_clean":"2023-10-06T22:13:05.722961+0000","last_became_active":"2023-10-06T22:12:42.690238+0000","last_became_peered":"2023-10-06T22:12:42.690238+0000","last_unstale":"2023-10-06T22:13:05.722961+0000","last_undegraded":"2023-10-06T22:13:05.722961+0000","last_fullsized":"2023-10-06T22:13:05.722961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:09.819570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676115+0000","last_change":"2023-10-06T22:12:43.699064+0000","last_active":"2023-10-06T22:14:05.676115+0000","last_peered":"2023-10-06T22:14:05.676115+0000","last_clean":"2023-10-06T22:14:05.676115+0000","last_became_active":"2023-10-06T22:12:43.698727+0000","last_became_peered":"2023-10-06T22:12:43.698727+0000","last_unstale":"2023-10-06T22:14:05.676115+0000","last_undegraded":"2023-10-06T22:14:05.676115+0000","last_fullsized":"2023-10-06T22:14:05.676115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:48:14.493376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.738711+0000","last_change":"2023-10-06T22:12:42.693779+0000","last_active":"2023-10-06T22:13:05.738711+0000","last_peered":"2023-10-06T22:13:05.738711+0000","last_clean":"2023-10-06T22:13:05.738711+0000","last_became_active":"2023-10-06T22:12:42.693257+0000","last_became_peered":"2023-10-06T22:12:42.693257+0000","last_unstale":"2023-10-06T22:13:05.738711+0000","last_undegraded":"2023-10-06T22:13:05.738711+0000","last_fullsized":"2023-10-06T22:13:05.738711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:56:48.945078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.631287+0000","last_change":"2023-10-06T22:12:43.700620+0000","last_active":"2023-10-06T22:14:05.631287+0000","last_peered":"2023-10-06T22:14:05.631287+0000","last_clean":"2023-10-06T22:14:05.631287+0000","last_became_active":"2023-10-06T22:12:43.700093+0000","last_became_peered":"2023-10-06T22:12:43.700093+0000","last_unstale":"2023-10-06T22:14:05.631287+0000","last_undegraded":"2023-10-06T22:14:05.631287+0000","last_fullsized":"2023-10-06T22:14:05.631287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:29.499778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.751425+0000","last_change":"2023-10-06T22:12:42.689640+0000","last_active":"2023-10-06T22:13:05.751425+0000","last_peered":"2023-10-06T22:13:05.751425+0000","last_clean":"2023-10-06T22:13:05.751425+0000","last_became_active":"2023-10-06T22:12:42.688584+0000","last_became_peered":"2023-10-06T22:12:42.688584+0000","last_unstale":"2023-10-06T22:13:05.751425+0000","last_undegraded":"2023-10-06T22:13:05.751425+0000","last_fullsized":"2023-10-06T22:13:05.751425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:43.066396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676684+0000","last_change":"2023-10-06T22:12:43.700084+0000","last_active":"2023-10-06T22:14:05.676684+0000","last_peered":"2023-10-06T22:14:05.676684+0000","last_clean":"2023-10-06T22:14:05.676684+0000","last_became_active":"2023-10-06T22:12:43.699787+0000","last_became_peered":"2023-10-06T22:12:43.699787+0000","last_unstale":"2023-10-06T22:14:05.676684+0000","last_undegraded":"2023-10-06T22:14:05.676684+0000","last_fullsized":"2023-10-06T22:14:05.676684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:07.219800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.727627+0000","last_change":"2023-10-06T22:12:42.688750+0000","last_active":"2023-10-06T22:13:05.727627+0000","last_peered":"2023-10-06T22:13:05.727627+0000","last_clean":"2023-10-06T22:13:05.727627+0000","last_became_active":"2023-10-06T22:12:42.688387+0000","last_became_peered":"2023-10-06T22:12:42.688387+0000","last_unstale":"2023-10-06T22:13:05.727627+0000","last_undegraded":"2023-10-06T22:13:05.727627+0000","last_fullsized":"2023-10-06T22:13:05.727627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:32.980631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.676460+0000","last_change":"2023-10-06T22:12:43.706741+0000","last_active":"2023-10-06T22:14:05.676460+0000","last_peered":"2023-10-06T22:14:05.676460+0000","last_clean":"2023-10-06T22:14:05.676460+0000","last_became_active":"2023-10-06T22:12:43.706416+0000","last_became_peered":"2023-10-06T22:12:43.706416+0000","last_unstale":"2023-10-06T22:14:05.676460+0000","last_undegraded":"2023-10-06T22:14:05.676460+0000","last_fullsized":"2023-10-06T22:14:05.676460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:55:16.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.724994+0000","last_change":"2023-10-06T22:12:42.692850+0000","last_active":"2023-10-06T22:13:05.724994+0000","last_peered":"2023-10-06T22:13:05.724994+0000","last_clean":"2023-10-06T22:13:05.724994+0000","last_became_active":"2023-10-06T22:12:42.692287+0000","last_became_peered":"2023-10-06T22:12:42.692287+0000","last_unstale":"2023-10-06T22:13:05.724994+0000","last_undegraded":"2023-10-06T22:13:05.724994+0000","last_fullsized":"2023-10-06T22:13:05.724994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:37:59.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667276+0000","last_change":"2023-10-06T22:12:43.700667+0000","last_active":"2023-10-06T22:14:05.667276+0000","last_peered":"2023-10-06T22:14:05.667276+0000","last_clean":"2023-10-06T22:14:05.667276+0000","last_became_active":"2023-10-06T22:12:43.698758+0000","last_became_peered":"2023-10-06T22:12:43.698758+0000","last_unstale":"2023-10-06T22:14:05.667276+0000","last_undegraded":"2023-10-06T22:14:05.667276+0000","last_fullsized":"2023-10-06T22:14:05.667276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:49:53.091578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'7","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:27.809273+0000","last_change":"2023-10-06T22:12:42.695964+0000","last_active":"2023-10-06T22:14:27.809273+0000","last_peered":"2023-10-06T22:14:27.809273+0000","last_clean":"2023-10-06T22:14:27.809273+0000","last_became_active":"2023-10-06T22:12:42.695431+0000","last_became_peered":"2023-10-06T22:12:42.695431+0000","last_unstale":"2023-10-06T22:14:27.809273+0000","last_undegraded":"2023-10-06T22:14:27.809273+0000","last_fullsized":"2023-10-06T22:14:27.809273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:21:42.761421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667361+0000","last_change":"2023-10-06T22:12:43.698088+0000","last_active":"2023-10-06T22:14:05.667361+0000","last_peered":"2023-10-06T22:14:05.667361+0000","last_clean":"2023-10-06T22:14:05.667361+0000","last_became_active":"2023-10-06T22:12:43.697765+0000","last_became_peered":"2023-10-06T22:12:43.697765+0000","last_unstale":"2023-10-06T22:14:05.667361+0000","last_undegraded":"2023-10-06T22:14:05.667361+0000","last_fullsized":"2023-10-06T22:14:05.667361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:26:22.958348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:05.723709+0000","last_change":"2023-10-06T22:12:42.698586+0000","last_active":"2023-10-06T22:13:05.723709+0000","last_peered":"2023-10-06T22:13:05.723709+0000","last_clean":"2023-10-06T22:13:05.723709+0000","last_became_active":"2023-10-06T22:12:42.698198+0000","last_became_peered":"2023-10-06T22:12:42.698198+0000","last_unstale":"2023-10-06T22:13:05.723709+0000","last_undegraded":"2023-10-06T22:13:05.723709+0000","last_fullsized":"2023-10-06T22:13:05.723709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:03:43.898987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671926+0000","last_change":"2023-10-06T22:12:43.698615+0000","last_active":"2023-10-06T22:14:05.671926+0000","last_peered":"2023-10-06T22:14:05.671926+0000","last_clean":"2023-10-06T22:14:05.671926+0000","last_became_active":"2023-10-06T22:12:43.698302+0000","last_became_peered":"2023-10-06T22:12:43.698302+0000","last_unstale":"2023-10-06T22:14:05.671926+0000","last_undegraded":"2023-10-06T22:14:05.671926+0000","last_fullsized":"2023-10-06T22:14:05.671926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:35.973119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T22:13:06.422877+0000","last_change":"2023-10-06T22:12:42.700325+0000","last_active":"2023-10-06T22:13:06.422877+0000","last_peered":"2023-10-06T22:13:06.422877+0000","last_clean":"2023-10-06T22:13:06.422877+0000","last_became_active":"2023-10-06T22:12:42.699426+0000","last_became_peered":"2023-10-06T22:12:42.699426+0000","last_unstale":"2023-10-06T22:13:06.422877+0000","last_undegraded":"2023-10-06T22:13:06.422877+0000","last_fullsized":"2023-10-06T22:13:06.422877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:41.662683+0000","last_clean_scrub_stamp":"2023-10-06T22:12:41.662683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:09.649737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667113+0000","last_change":"2023-10-06T22:12:43.698620+0000","last_active":"2023-10-06T22:14:05.667113+0000","last_peered":"2023-10-06T22:14:05.667113+0000","last_clean":"2023-10-06T22:14:05.667113+0000","last_became_active":"2023-10-06T22:12:43.698335+0000","last_became_peered":"2023-10-06T22:12:43.698335+0000","last_unstale":"2023-10-06T22:14:05.667113+0000","last_undegraded":"2023-10-06T22:14:05.667113+0000","last_fullsized":"2023-10-06T22:14:05.667113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:53:49.222848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.669528+0000","last_change":"2023-10-06T22:12:43.700477+0000","last_active":"2023-10-06T22:14:05.669528+0000","last_peered":"2023-10-06T22:14:05.669528+0000","last_clean":"2023-10-06T22:14:05.669528+0000","last_became_active":"2023-10-06T22:12:43.700177+0000","last_became_peered":"2023-10-06T22:12:43.700177+0000","last_unstale":"2023-10-06T22:14:05.669528+0000","last_undegraded":"2023-10-06T22:14:05.669528+0000","last_fullsized":"2023-10-06T22:14:05.669528+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:11.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664348+0000","last_change":"2023-10-06T22:12:43.701792+0000","last_active":"2023-10-06T22:14:05.664348+0000","last_peered":"2023-10-06T22:14:05.664348+0000","last_clean":"2023-10-06T22:14:05.664348+0000","last_became_active":"2023-10-06T22:12:43.701316+0000","last_became_peered":"2023-10-06T22:12:43.701316+0000","last_unstale":"2023-10-06T22:14:05.664348+0000","last_undegraded":"2023-10-06T22:14:05.664348+0000","last_fullsized":"2023-10-06T22:14:05.664348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:03.085219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.677454+0000","last_change":"2023-10-06T22:12:43.707871+0000","last_active":"2023-10-06T22:14:05.677454+0000","last_peered":"2023-10-06T22:14:05.677454+0000","last_clean":"2023-10-06T22:14:05.677454+0000","last_became_active":"2023-10-06T22:12:43.707659+0000","last_became_peered":"2023-10-06T22:12:43.707659+0000","last_unstale":"2023-10-06T22:14:05.677454+0000","last_undegraded":"2023-10-06T22:14:05.677454+0000","last_fullsized":"2023-10-06T22:14:05.677454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:37:43.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.673925+0000","last_change":"2023-10-06T22:12:43.697701+0000","last_active":"2023-10-06T22:14:05.673925+0000","last_peered":"2023-10-06T22:14:05.673925+0000","last_clean":"2023-10-06T22:14:05.673925+0000","last_became_active":"2023-10-06T22:12:43.697361+0000","last_became_peered":"2023-10-06T22:12:43.697361+0000","last_unstale":"2023-10-06T22:14:05.673925+0000","last_undegraded":"2023-10-06T22:14:05.673925+0000","last_fullsized":"2023-10-06T22:14:05.673925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:14:53.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679653+0000","last_change":"2023-10-06T22:12:43.700058+0000","last_active":"2023-10-06T22:14:05.679653+0000","last_peered":"2023-10-06T22:14:05.679653+0000","last_clean":"2023-10-06T22:14:05.679653+0000","last_became_active":"2023-10-06T22:12:43.697893+0000","last_became_peered":"2023-10-06T22:12:43.697893+0000","last_unstale":"2023-10-06T22:14:05.679653+0000","last_undegraded":"2023-10-06T22:14:05.679653+0000","last_fullsized":"2023-10-06T22:14:05.679653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:01:37.518096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.664805+0000","last_change":"2023-10-06T22:13:06.729307+0000","last_active":"2023-10-06T22:14:05.664805+0000","last_peered":"2023-10-06T22:14:05.664805+0000","last_clean":"2023-10-06T22:14:05.664805+0000","last_became_active":"2023-10-06T22:13:06.728877+0000","last_became_peered":"2023-10-06T22:13:06.728877+0000","last_unstale":"2023-10-06T22:14:05.664805+0000","last_undegraded":"2023-10-06T22:14:05.664805+0000","last_fullsized":"2023-10-06T22:14:05.664805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:17.629046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.666149+0000","last_change":"2023-10-06T22:12:43.700668+0000","last_active":"2023-10-06T22:14:05.666149+0000","last_peered":"2023-10-06T22:14:05.666149+0000","last_clean":"2023-10-06T22:14:05.666149+0000","last_became_active":"2023-10-06T22:12:43.700387+0000","last_became_peered":"2023-10-06T22:12:43.700387+0000","last_unstale":"2023-10-06T22:14:05.666149+0000","last_undegraded":"2023-10-06T22:14:05.666149+0000","last_fullsized":"2023-10-06T22:14:05.666149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:36:00.053496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.662653+0000","last_change":"2023-10-06T22:12:43.702658+0000","last_active":"2023-10-06T22:14:05.662653+0000","last_peered":"2023-10-06T22:14:05.662653+0000","last_clean":"2023-10-06T22:14:05.662653+0000","last_became_active":"2023-10-06T22:12:43.702264+0000","last_became_peered":"2023-10-06T22:12:43.702264+0000","last_unstale":"2023-10-06T22:14:05.662653+0000","last_undegraded":"2023-10-06T22:14:05.662653+0000","last_fullsized":"2023-10-06T22:14:05.662653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:56.472029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.671729+0000","last_change":"2023-10-06T22:12:43.705718+0000","last_active":"2023-10-06T22:14:05.671729+0000","last_peered":"2023-10-06T22:14:05.671729+0000","last_clean":"2023-10-06T22:14:05.671729+0000","last_became_active":"2023-10-06T22:12:43.705479+0000","last_became_peered":"2023-10-06T22:12:43.705479+0000","last_unstale":"2023-10-06T22:14:05.671729+0000","last_undegraded":"2023-10-06T22:14:05.671729+0000","last_fullsized":"2023-10-06T22:14:05.671729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:55:36.393577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.672014+0000","last_change":"2023-10-06T22:12:43.701891+0000","last_active":"2023-10-06T22:14:05.672014+0000","last_peered":"2023-10-06T22:14:05.672014+0000","last_clean":"2023-10-06T22:14:05.672014+0000","last_became_active":"2023-10-06T22:12:43.701535+0000","last_became_peered":"2023-10-06T22:12:43.701535+0000","last_unstale":"2023-10-06T22:14:05.672014+0000","last_undegraded":"2023-10-06T22:14:05.672014+0000","last_fullsized":"2023-10-06T22:14:05.672014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:17:40.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.667438+0000","last_change":"2023-10-06T22:12:43.707075+0000","last_active":"2023-10-06T22:14:05.667438+0000","last_peered":"2023-10-06T22:14:05.667438+0000","last_clean":"2023-10-06T22:14:05.667438+0000","last_became_active":"2023-10-06T22:12:43.706723+0000","last_became_peered":"2023-10-06T22:12:43.706723+0000","last_unstale":"2023-10-06T22:14:05.667438+0000","last_undegraded":"2023-10-06T22:14:05.667438+0000","last_fullsized":"2023-10-06T22:14:05.667438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:18.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.679749+0000","last_change":"2023-10-06T22:12:43.704656+0000","last_active":"2023-10-06T22:14:05.679749+0000","last_peered":"2023-10-06T22:14:05.679749+0000","last_clean":"2023-10-06T22:14:05.679749+0000","last_became_active":"2023-10-06T22:12:43.704312+0000","last_became_peered":"2023-10-06T22:12:43.704312+0000","last_unstale":"2023-10-06T22:14:05.679749+0000","last_undegraded":"2023-10-06T22:14:05.679749+0000","last_fullsized":"2023-10-06T22:14:05.679749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:21:37.951180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T22:14:05.663935+0000","last_change":"2023-10-06T22:12:43.702239+0000","last_active":"2023-10-06T22:14:05.663935+0000","last_peered":"2023-10-06T22:14:05.663935+0000","last_clean":"2023-10-06T22:14:05.663935+0000","last_became_active":"2023-10-06T22:12:43.701851+0000","last_became_peered":"2023-10-06T22:12:43.701851+0000","last_unstale":"2023-10-06T22:14:05.663935+0000","last_undegraded":"2023-10-06T22:14:05.663935+0000","last_fullsized":"2023-10-06T22:14:05.663935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T22:12:42.669440+0000","last_clean_scrub_stamp":"2023-10-06T22:12:42.669440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:40:53.661587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66404,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1014,"num_read_kb":575,"num_write":245,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449122,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":444759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72099999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.38100000000000001}]},{"osd":1,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.371,"15min":0.371},"max":{"1min":0.85299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.34200000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61899999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82599999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.48799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69799999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54500000000000004,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.311,"15min":0.311},"max":{"1min":0.68400000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82399999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.69299999999999995},"min":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.74199999999999999,"5min":1.044,"15min":1.044},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.56599999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54100000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86099999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.61199999999999999}]},{"osd":8,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77100000000000002}]},{"osd":9,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.62,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89100000000000001,"5min":1.472,"15min":1.472},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.63500000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0109999999999999,"5min":1.151,"15min":1.151},"last":0.83399999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.68300000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82499999999999996,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.64800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85299999999999998,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.751}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1093852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.59699999999999998,"5min":0.748,"15min":0.748},"last":0.44}]},{"osd":2,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61399999999999999,"15min":0.61299999999999999},"min":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":1.026,"15min":1.026},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96099999999999997,"5min":1.093,"15min":1.093},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.626},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78000000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81699999999999995,"5min":1.079,"15min":1.079},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.266,"15min":1.266},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81200000000000006},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80500000000000005,"15min":0.79900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.376},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.505}]},{"osd":8,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90200000000000002,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.751},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95199999999999996,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.81699999999999995}]},{"osd":9,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86899999999999999,"15min":0.86499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.88800000000000001,"15min":0.88400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.72699999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97099999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.94899999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86099999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":444158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.34300000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89400000000000002,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.747,"15min":0.77000000000000002},"min":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.83599999999999997,"5min":1.036,"15min":1.036},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79800000000000004,"15min":0.83599999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96599999999999997,"5min":1.486,"15min":1.486},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":0.78800000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91500000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.038,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72299999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.73899999999999999,"15min":0.76700000000000002},"min":{"1min":0.436,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93799999999999994,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.82799999999999996,"15min":0.871},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0629999999999999,"5min":1.548,"15min":1.548},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.871,"15min":0.91700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94699999999999995,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.73799999999999999,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89900000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.90200000000000002,"15min":0.94199999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90700000000000003,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.96099999999999997,"15min":1.0189999999999999},"min":{"1min":0.63500000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1259999999999999,"5min":1.585,"15min":1.585},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83899999999999997,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.95499999999999996}]},{"osd":8,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.96799999999999997,"15min":1.006},"min":{"1min":0.71199999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.167,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.86899999999999999,"15min":0.90100000000000002},"min":{"1min":0.63600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.02,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.0109999999999999,"15min":1.028},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.194,"5min":1.268,"15min":1.268},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.997,"15min":1.018},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.878}]},{"osd":11,"last update":"Fri Oct 6 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.90900000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78800000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45600000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.46899999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89200000000000002,"5min":1.018,"15min":1.018},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.53500000000000003},"min":{"1min":0.49199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.434,"15min":0.41199999999999998},"min":{"1min":0.33100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.378},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46999999999999997,"15min":0.45800000000000002},"min":{"1min":0.39300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.52900000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.63900000000000001,"15min":0.61099999999999999},"min":{"1min":0.60199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.42699999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.64400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Oct 6 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.57899999999999996,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.873,"5min":1.073,"15min":1.073},"last":0.745},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.65800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92500000000000004,"5min":1.502,"15min":1.502},"last":0.624}]},{"osd":9,"last update":"Fri Oct 6 22:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.68400000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.78300000000000003,"15min":0.76500000000000001},"min":{"1min":0.77100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.63800000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.754,"15min":0.72999999999999998},"min":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.72999999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.505}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.33600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.374},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44,"15min":0.436},"min":{"1min":0.35899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.627,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65700000000000003,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.847,"5min":3.847,"15min":3.847},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51400000000000001,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.875,"15min":0.875},"last":0.35299999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.52700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.51500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.438}]},{"osd":8,"last update":"Fri Oct 6 22:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.745},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0169999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.76400000000000001},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.089,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.753}]},{"osd":10,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.52500000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.86899999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72899999999999998}]},{"osd":11,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.77800000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81399999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68600000000000005,"15min":0.66300000000000003},"min":{"1min":0.53200000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92700000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93000000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86299999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60499999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.75}]},{"osd":4,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73799999999999999,"15min":0.748},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Fri Oct 6 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.754},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.004,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.02,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79300000000000004,"15min":0.77600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99399999999999999,"5min":1.173,"15min":1.173},"last":0.875},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83899999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97699999999999998,"5min":1.145,"15min":1.145},"last":0.88700000000000001}]},{"osd":8,"last update":"Fri Oct 6 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.59399999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.1080000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85299999999999998,"15min":0.86199999999999999},"min":{"1min":0.64900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.137,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Oct 6 22:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.91800000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.90400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.92300000000000004,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1599999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.93700000000000006,"15min":0.93999999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.97799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.441,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.78000000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.435,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Oct 6 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55900000000000005,"15min":0.52600000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56799999999999995},"min":{"1min":0.46200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84699999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63700000000000001,"15min":0.58199999999999996},"min":{"1min":0.47899999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.754}]},{"osd":4,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.443,"15min":0.39700000000000002},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48899999999999999,"15min":0.433},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.433}]},{"osd":5,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.58699999999999997,"15min":0.53700000000000003},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.60699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.59799999999999998},"min":{"1min":0.53800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95099999999999996,"5min":1.45,"15min":1.45},"last":0.625},{"interface":"front","average":{"1min":0.753,"5min":0.65100000000000002,"15min":0.60099999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Oct 6 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55700000000000005,"15min":0.53400000000000003},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Oct 6 22:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.51500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.56000000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.93200000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.81699999999999995,"15min":0.79000000000000004},"min":{"1min":0.66000000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.139,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.63700000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94799999999999995,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.87}]},{"osd":10,"last update":"Fri Oct 6 22:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.74199999999999999,"15min":0.72799999999999998},"min":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Oct 6 22:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.84099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":445141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60299999999999998,"15min":0.52500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Oct 6 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61099999999999999,"15min":0.56799999999999995},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.62}]},{"osd":3,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Oct 6 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.67200000000000004,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.59199999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65100000000000002,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.73399999999999999,"15min":0.68899999999999995},"min":{"1min":0.59299999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0669999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68100000000000005,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.97399999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri Oct 6 22:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.86899999999999999},"min":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.121,"5min":4.5819999999999999,"15min":4.5819999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.64900000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.745},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.77000000000000002},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79000000000000004}]},{"osd":9,"last update":"Fri Oct 6 22:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.012,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.872,"5min":0.88600000000000001,"15min":0.89900000000000002},"min":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.048,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.877,"15min":0.89300000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93300000000000005,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.054,"5min":1.103,"15min":1.103},"last":0.82799999999999996}]},{"osd":11,"last update":"Fri Oct 6 22:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.86199999999999999,"15min":0.85699999999999998},"min":{"1min":0.67300000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0669999999999999,"5min":1.538,"15min":1.538},"last":0.876}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":503812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.125},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57199999999999995,"15min":0.51300000000000001},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.873,"15min":0.873},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.52300000000000002},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 22:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71599999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44900000000000001,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.373,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 22:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.71199999999999997,"15min":0.66900000000000004},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64500000000000002,"15min":0.61899999999999999},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78900000000000003,"5min":1.071,"15min":1.071},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":1.002,"15min":1.002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92100000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Oct 6 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.629},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85299999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.58099999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92200000000000004,"5min":1.633,"15min":1.633},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.878,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72199999999999998}]},{"osd":10,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.747},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.83599999999999997}]},{"osd":11,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.75700000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":443439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.40100000000000002},"min":{"1min":0.34999999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.41699999999999998},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.60999999999999999},"min":{"1min":0.48999999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92500000000000004,"5min":1.069,"15min":1.069},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.61799999999999999},"min":{"1min":0.52800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 22:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.60799999999999998},"min":{"1min":0.48899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.74299999999999999,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Oct 6 22:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.0109999999999999,"5min":1.635,"15min":1.635},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71199999999999997,"15min":0.629},"min":{"1min":0.501,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Oct 6 22:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68200000000000005,"15min":0.64800000000000002},"min":{"1min":0.53400000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.71099999999999997},"min":{"1min":0.57999999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Oct 6 22:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4019999999999999,"5min":1.794,"15min":1.794},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.163,"5min":1.224,"15min":1.224},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76000000000000001,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.0509999999999999,"15min":1.133},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.778,"5min":7.0529999999999999,"15min":7.0529999999999999},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Oct 6 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.95399999999999996}]},{"osd":9,"last update":"Fri Oct 6 22:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88400000000000001,"15min":0.91200000000000003},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86599999999999999,"15min":0.88400000000000001},"min":{"1min":0.49299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.448,"5min":1.748,"15min":1.748},"last":0.85599999999999998}]},{"osd":10,"last update":"Fri Oct 6 22:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.94899999999999995,"15min":0.95399999999999996},"min":{"1min":0.752,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":1.0229999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.37},"min":{"1min":0.28999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.436},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.39000000000000001},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.61399999999999999,"15min":0.56499999999999995},"min":{"1min":0.504,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.56299999999999994},"min":{"1min":0.53400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.57999999999999996},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86599999999999999,"5min":1.365,"15min":1.365},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.48199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.879,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59299999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82199999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89300000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 22:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66900000000000004,"15min":0.626},"min":{"1min":0.45200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.68600000000000005,"15min":0.64100000000000001},"min":{"1min":0.55000000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.41799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.018,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51400000000000001}]},{"osd":8,"last update":"Fri Oct 6 22:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.89200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.877},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81299999999999994,"5min":0.877,"15min":0.877},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Oct 6 22:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79400000000000004,"15min":0.78600000000000003},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Fri Oct 6 22:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.89700000000000002,"5min":1.054,"15min":1.054},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Oct 6 22:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77300000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1036878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.752,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73299999999999998,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83899999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78000000000000003,"5min":2.802,"15min":2.802},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40200000000000002,"15min":0.38600000000000001},"min":{"1min":0.308,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.67700000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.42199999999999999},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64600000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.499},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.45500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80900000000000005,"5min":1.214,"15min":1.214},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79700000000000004,"5min":2.661,"15min":2.661},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.80200000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Oct 6 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.56999999999999995},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.629},"min":{"1min":0.50900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82399999999999995,"5min":1.274,"15min":1.274},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 22:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.60299999999999998},"min":{"1min":0.59299999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96599999999999997,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.78800000000000003}]},{"osd":8,"last update":"Fri Oct 6 22:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85699999999999998,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.87,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63500000000000001}]},{"osd":9,"last update":"Fri Oct 6 22:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80000000000000004,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77700000000000002,"15min":0.77800000000000002},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.89500000000000002,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.65400000000000003}]},{"osd":10,"last update":"Fri Oct 6 22:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.60399999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":2.774,"15min":2.774},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72499999999999998,"15min":0.71099999999999997},"min":{"1min":0.63800000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94199999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T22:14:32.756 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:14:32.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.988+0000 7fe769bff700 1 -- 172.21.15.57:0/3304979085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 msgr2=0x7fe764109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.988+0000 7fe769bff700 1 --2- 172.21.15.57:0/3304979085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe764109360 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00b0a0 tx=0x7fe75c00b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.989+0000 7fe769bff700 1 -- 172.21.15.57:0/3304979085 shutdown_connections 2023-10-06T22:14:32.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.989+0000 7fe769bff700 1 --2- 172.21.15.57:0/3304979085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe764109360 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.989+0000 7fe769bff700 1 --2- 172.21.15.57:0/3304979085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe764100ea0 0x7fe7641012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.989+0000 7fe769bff700 1 --2- 172.21.15.57:0/3304979085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe764102080 0x7fe764102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.989+0000 7fe769bff700 1 -- 172.21.15.57:0/3304979085 >> 172.21.15.57:0/3304979085 conn(0x7fe7640fc420 msgr2=0x7fe7640fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:32.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.989+0000 7fe769bff700 1 -- 172.21.15.57:0/3304979085 shutdown_connections 2023-10-06T22:14:32.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.989+0000 7fe769bff700 1 -- 172.21.15.57:0/3304979085 wait complete. 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 Processor -- start 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 -- start start 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe764100ea0 0x7fe764198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe764102080 0x7fe764199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe7641a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641a0a50 con 0x7fe764102a40 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641a0b90 con 0x7fe764102080 2023-10-06T22:14:32.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe769bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641a0d20 con 0x7fe764100ea0 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.990+0000 7fe763fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe7641a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe7641a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33710/0 (socket says 172.21.15.57:33710) 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 -- 172.21.15.57:0/171871393 learned_addr learned my addr 172.21.15.57:0/171871393 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe762ffd700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe764102080 0x7fe764199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe7637fe700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe764100ea0 0x7fe764198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 -- 172.21.15.57:0/171871393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe764100ea0 msgr2=0x7fe764198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe764100ea0 0x7fe764198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 -- 172.21.15.57:0/171871393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe764102080 msgr2=0x7fe764199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe764102080 0x7fe764199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 -- 172.21.15.57:0/171871393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c009d00 con 0x7fe764102a40 2023-10-06T22:14:32.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe763fff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe7641a0480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009fd0 tx=0x7fe75c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe760ff9700 1 -- 172.21.15.57:0/171871393 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00e040 con 0x7fe764102a40 2023-10-06T22:14:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe760ff9700 1 -- 172.21.15.57:0/171871393 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe75c0094a0 con 0x7fe764102a40 2023-10-06T22:14:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7641a0fa0 con 0x7fe764102a40 2023-10-06T22:14:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.991+0000 7fe760ff9700 1 -- 172.21.15.57:0/171871393 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c01b6c0 con 0x7fe764102a40 2023-10-06T22:14:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.992+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7641a13c0 con 0x7fe764102a40 2023-10-06T22:14:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.992+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76410c760 con 0x7fe764102a40 2023-10-06T22:14:32.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.994+0000 7fe760ff9700 1 -- 172.21.15.57:0/171871393 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe75c01d470 con 0x7fe764102a40 2023-10-06T22:14:32.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.997+0000 7fe760ff9700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe74c0a1550 0x7fe74c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:32.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.997+0000 7fe760ff9700 1 -- 172.21.15.57:0/171871393 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fe75c0cdc00 con 0x7fe764102a40 2023-10-06T22:14:32.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.997+0000 7fe7637fe700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe74c0a1550 0x7fe74c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:32.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:32.998+0000 7fe7637fe700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe74c0a1550 0x7fe74c0a3a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe764101bf0 tx=0x7fe754009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:33.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.001+0000 7fe760ff9700 1 -- 172.21.15.57:0/171871393 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe75c0972c0 con 0x7fe764102a40 2023-10-06T22:14:33.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:32 smithi156 ceph-mon[134358]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-10-06T22:14:33.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:32 smithi156 ceph-mon[134358]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:32 smithi190 ceph-mon[134891]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-10-06T22:14:33.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:32 smithi190 ceph-mon[134891]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:32 smithi057 ceph-mon[98492]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-10-06T22:14:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:32 smithi057 ceph-mon[98492]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:33.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.212+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7641a15b0 con 0x7fe764102a40 2023-10-06T22:14:33.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.214+0000 7fe760ff9700 1 -- 172.21.15.57:0/171871393 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7fe75c09a8e0 con 0x7fe764102a40 2023-10-06T22:14:33.217 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:33.217 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:14:33.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe74c0a1550 msgr2=0x7fe74c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe74c0a1550 0x7fe74c0a3a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe764101bf0 tx=0x7fe754009260 comp rx=0 tx=0).stop 2023-10-06T22:14:33.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 msgr2=0x7fe7641a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe7641a0480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009fd0 tx=0x7fe75c00bce0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 shutdown_connections 2023-10-06T22:14:33.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fe74c0a1550 0x7fe74c0a3a10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe764102a40 0x7fe7641a0480 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe764100ea0 0x7fe764198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 --2- 172.21.15.57:0/171871393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe764102080 0x7fe764199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 >> 172.21.15.57:0/171871393 conn(0x7fe7640fc420 msgr2=0x7fe7640fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 shutdown_connections 2023-10-06T22:14:33.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.216+0000 7fe769bff700 1 -- 172.21.15.57:0/171871393 wait complete. 2023-10-06T22:14:33.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:14:33.239 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 -- 172.21.15.57:0/1480650174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290101f50 msgr2=0x7f12901023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 --2- 172.21.15.57:0/1480650174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290101f50 0x7f12901023b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f12840099c0 tx=0x7f1284009cd0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 -- 172.21.15.57:0/1480650174 shutdown_connections 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 --2- 172.21.15.57:0/1480650174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12901028f0 0x7f12901091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 --2- 172.21.15.57:0/1480650174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1290100d60 0x7f1290101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 --2- 172.21.15.57:0/1480650174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290101f50 0x7f12901023b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 -- 172.21.15.57:0/1480650174 >> 172.21.15.57:0/1480650174 conn(0x7f12900fc2b0 msgr2=0x7f12900fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 -- 172.21.15.57:0/1480650174 shutdown_connections 2023-10-06T22:14:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.444+0000 7f1294f9d700 1 -- 172.21.15.57:0/1480650174 wait complete. 2023-10-06T22:14:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.446+0000 7f1294f9d700 1 Processor -- start 2023-10-06T22:14:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.446+0000 7f1294f9d700 1 -- start start 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.446+0000 7f1294f9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290100d60 0x7f1290196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.446+0000 7f1294f9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1290101f50 0x7f1290196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f1294f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12901028f0 0x7f129019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f1294f9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129019e4e0 con 0x7f1290100d60 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f1294f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129019e620 con 0x7f12901028f0 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f1294f9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129019e7b0 con 0x7f1290101f50 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290100d60 0x7f1290196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290100d60 0x7f1290196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33738/0 (socket says 172.21.15.57:33738) 2023-10-06T22:14:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128f7fe700 1 -- 172.21.15.57:0/680937749 learned_addr learned my addr 172.21.15.57:0/680937749 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:33.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128effd700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1290101f50 0x7f1290196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128ffff700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12901028f0 0x7f129019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128effd700 1 -- 172.21.15.57:0/680937749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12901028f0 msgr2=0x7f129019df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128effd700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12901028f0 0x7f129019df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128effd700 1 -- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290100d60 msgr2=0x7f1290196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128effd700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290100d60 0x7f1290196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128effd700 1 -- 172.21.15.57:0/680937749 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12840095f0 con 0x7f1290101f50 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.447+0000 7f128ffff700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12901028f0 0x7f129019df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.448+0000 7f128effd700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1290101f50 0x7f1290196ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12840098e0 tx=0x7f128400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.448+0000 7f128cff9700 1 -- 172.21.15.57:0/680937749 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128401d070 con 0x7f1290101f50 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.448+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129019ea30 con 0x7f1290101f50 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.448+0000 7f128cff9700 1 -- 172.21.15.57:0/680937749 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12840054e0 con 0x7f1290101f50 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.448+0000 7f128cff9700 1 -- 172.21.15.57:0/680937749 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284004a90 con 0x7f1290101f50 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.448+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129019ee50 con 0x7f1290101f50 2023-10-06T22:14:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.449+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1290107ba0 con 0x7f1290101f50 2023-10-06T22:14:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.451+0000 7f128cff9700 1 -- 172.21.15.57:0/680937749 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f128400fb80 con 0x7f1290101f50 2023-10-06T22:14:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.453+0000 7f128cff9700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f12780a15a0 0x7f12780a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.453+0000 7f128cff9700 1 -- 172.21.15.57:0/680937749 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f12840cdc40 con 0x7f1290101f50 2023-10-06T22:14:33.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.453+0000 7f128f7fe700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f12780a15a0 0x7f12780a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.454+0000 7f128f7fe700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f12780a15a0 0x7f12780a3a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1280005950 tx=0x7f1280009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:33.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.455+0000 7f128cff9700 1 -- 172.21.15.57:0/680937749 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1284097300 con 0x7f1290101f50 2023-10-06T22:14:33.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.666+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f129010c750 con 0x7f1290101f50 2023-10-06T22:14:33.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.667+0000 7f128cff9700 1 -- 172.21.15.57:0/680937749 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f128409a920 con 0x7f1290101f50 2023-10-06T22:14:33.669 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:14:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f12780a15a0 msgr2=0x7f12780a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f12780a15a0 0x7f12780a3a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1280005950 tx=0x7f1280009260 comp rx=0 tx=0).stop 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1290101f50 msgr2=0x7f1290196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1290101f50 0x7f1290196ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12840098e0 tx=0x7f128400f690 comp rx=0 tx=0).stop 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 shutdown_connections 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f12780a15a0 0x7f12780a3a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12901028f0 0x7f129019df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1290100d60 0x7f1290196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 --2- 172.21.15.57:0/680937749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1290101f50 0x7f1290196ea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 >> 172.21.15.57:0/680937749 conn(0x7f12900fc2b0 msgr2=0x7f12900fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 shutdown_connections 2023-10-06T22:14:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.669+0000 7f1294f9d700 1 -- 172.21.15.57:0/680937749 wait complete. 2023-10-06T22:14:33.690 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:14:33.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 -- 172.21.15.57:0/29539599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102a40 msgr2=0x7f4a50109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 --2- 172.21.15.57:0/29539599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102a40 0x7f4a50109360 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800b0a0 tx=0x7f4a4800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 -- 172.21.15.57:0/29539599 shutdown_connections 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 --2- 172.21.15.57:0/29539599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102a40 0x7f4a50109360 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 --2- 172.21.15.57:0/29539599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a50100ea0 0x7f4a501012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 --2- 172.21.15.57:0/29539599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a50102080 0x7f4a50102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 -- 172.21.15.57:0/29539599 >> 172.21.15.57:0/29539599 conn(0x7f4a500fc460 msgr2=0x7f4a500fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 -- 172.21.15.57:0/29539599 shutdown_connections 2023-10-06T22:14:33.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.897+0000 7f4a553ff700 1 -- 172.21.15.57:0/29539599 wait complete. 2023-10-06T22:14:33.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.898+0000 7f4a553ff700 1 Processor -- start 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.898+0000 7f4a553ff700 1 -- start start 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a553ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a50100ea0 0x7f4a50198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a553ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102080 0x7f4a50199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a553ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a50102a40 0x7f4a501a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a553ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501a0a90 con 0x7f4a50102080 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a553ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501a0bd0 con 0x7f4a50102a40 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a553ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501a0d60 con 0x7f4a50100ea0 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102080 0x7f4a50199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102080 0x7f4a50199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33756/0 (socket says 172.21.15.57:33756) 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 -- 172.21.15.57:0/997080442 learned_addr learned my addr 172.21.15.57:0/997080442 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:33.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a4f7fe700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a50102a40 0x7f4a501a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a4effd700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a50100ea0 0x7f4a50198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 -- 172.21.15.57:0/997080442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a50100ea0 msgr2=0x7f4a50198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a50100ea0 0x7f4a50198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 -- 172.21.15.57:0/997080442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a50102a40 msgr2=0x7f4a501a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a50102a40 0x7f4a501a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a47fff700 1 -- 172.21.15.57:0/997080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48009d00 con 0x7f4a50102080 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.899+0000 7f4a4f7fe700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a50102a40 0x7f4a501a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.900+0000 7f4a47fff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102080 0x7f4a50199450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800e970 tx=0x7f4a3800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.900+0000 7f4a4cff9700 1 -- 172.21.15.57:0/997080442 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38009790 con 0x7f4a50102080 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.900+0000 7f4a4cff9700 1 -- 172.21.15.57:0/997080442 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3800c7e0 con 0x7f4a50102080 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.900+0000 7f4a4cff9700 1 -- 172.21.15.57:0/997080442 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38003c70 con 0x7f4a50102080 2023-10-06T22:14:33.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.900+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a501a0fe0 con 0x7f4a50102080 2023-10-06T22:14:33.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.900+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a501a1470 con 0x7f4a50102080 2023-10-06T22:14:33.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.901+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5010c760 con 0x7f4a50102080 2023-10-06T22:14:33.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.903+0000 7f4a4cff9700 1 -- 172.21.15.57:0/997080442 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4a3800cc50 con 0x7f4a50102080 2023-10-06T22:14:33.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.904+0000 7f4a4cff9700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4a3c0a1500 0x7f4a3c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:33.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.905+0000 7f4a4cff9700 1 -- 172.21.15.57:0/997080442 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f4a38014070 con 0x7f4a50102080 2023-10-06T22:14:33.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.905+0000 7f4a4effd700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4a3c0a1500 0x7f4a3c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:33.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.905+0000 7f4a4effd700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4a3c0a1500 0x7f4a3c0a39c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a50101bf0 tx=0x7f4a40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:33.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:33.908+0000 7f4a4cff9700 1 -- 172.21.15.57:0/997080442 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4a38096480 con 0x7f4a50102080 2023-10-06T22:14:34.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:33 smithi156 ceph-mon[134358]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:34.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:33 smithi156 ceph-mon[134358]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:34.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:33 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/171871393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:34.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:33 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/680937749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:33 smithi190 ceph-mon[134891]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:33 smithi190 ceph-mon[134891]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:34.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:33 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/171871393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:34.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:33 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/680937749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:34.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.118+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a501a1970 con 0x7f4a50102080 2023-10-06T22:14:34.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.119+0000 7f4a4cff9700 1 -- 172.21.15.57:0/997080442 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a38003dd0 con 0x7f4a50102080 2023-10-06T22:14:34.121 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:14:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4a3c0a1500 msgr2=0x7f4a3c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4a3c0a1500 0x7f4a3c0a39c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a50101bf0 tx=0x7f4a40009260 comp rx=0 tx=0).stop 2023-10-06T22:14:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102080 msgr2=0x7f4a50199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102080 0x7f4a50199450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800e970 tx=0x7f4a3800ec80 comp rx=0 tx=0).stop 2023-10-06T22:14:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 shutdown_connections 2023-10-06T22:14:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f4a3c0a1500 0x7f4a3c0a39c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a50102a40 0x7f4a501a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a50100ea0 0x7f4a50198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 --2- 172.21.15.57:0/997080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a50102080 0x7f4a50199450 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 >> 172.21.15.57:0/997080442 conn(0x7f4a500fc460 msgr2=0x7f4a50107950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 shutdown_connections 2023-10-06T22:14:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.121+0000 7f4a553ff700 1 -- 172.21.15.57:0/997080442 wait complete. 2023-10-06T22:14:34.140 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:33 smithi057 ceph-mon[98492]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:34.140 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:33 smithi057 ceph-mon[98492]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T22:14:34.140 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:33 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/171871393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:34.140 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:33 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/680937749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:14:34.142 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:14:34.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 -- 172.21.15.57:0/1416270976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0100e90 msgr2=0x7f57b01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 --2- 172.21.15.57:0/1416270976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0100e90 0x7f57b01012b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009990 tx=0x7f57ac009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 -- 172.21.15.57:0/1416270976 shutdown_connections 2023-10-06T22:14:34.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 --2- 172.21.15.57:0/1416270976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57b0102a30 0x7f57b0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 --2- 172.21.15.57:0/1416270976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0100e90 0x7f57b01012b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 --2- 172.21.15.57:0/1416270976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0102070 0x7f57b01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 -- 172.21.15.57:0/1416270976 >> 172.21.15.57:0/1416270976 conn(0x7f57b00fc430 msgr2=0x7f57b00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:34.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 -- 172.21.15.57:0/1416270976 shutdown_connections 2023-10-06T22:14:34.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.349+0000 7f57b73ff700 1 -- 172.21.15.57:0/1416270976 wait complete. 2023-10-06T22:14:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 Processor -- start 2023-10-06T22:14:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 -- start start 2023-10-06T22:14:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0100e90 0x7f57b0196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0102070 0x7f57b0197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57b0102a30 0x7f57b019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b019e840 con 0x7f57b0102a30 2023-10-06T22:14:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b019e9b0 con 0x7f57b0102070 2023-10-06T22:14:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.350+0000 7f57b73ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b019eb70 con 0x7f57b0100e90 2023-10-06T22:14:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0100e90 0x7f57b0196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b599c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57b0102a30 0x7f57b019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0100e90 0x7f57b0196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44056/0 (socket says 172.21.15.57:44056) 2023-10-06T22:14:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 -- 172.21.15.57:0/2867977894 learned_addr learned my addr 172.21.15.57:0/2867977894 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b499a700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0102070 0x7f57b0197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 -- 172.21.15.57:0/2867977894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0102070 msgr2=0x7f57b0197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0102070 0x7f57b0197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 -- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57b0102a30 msgr2=0x7f57b019e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57b0102a30 0x7f57b019e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 -- 172.21.15.57:0/2867977894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0095f0 con 0x7f57b0100e90 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b599c700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57b0102a30 0x7f57b019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.351+0000 7f57b519b700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0100e90 0x7f57b0196d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005720 tx=0x7f57ac009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.352+0000 7f57a67fc700 1 -- 172.21.15.57:0/2867977894 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac01d070 con 0x7f57b0100e90 2023-10-06T22:14:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.352+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b019edf0 con 0x7f57b0100e90 2023-10-06T22:14:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.352+0000 7f57a67fc700 1 -- 172.21.15.57:0/2867977894 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57ac021410 con 0x7f57b0100e90 2023-10-06T22:14:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.352+0000 7f57a67fc700 1 -- 172.21.15.57:0/2867977894 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac017650 con 0x7f57b0100e90 2023-10-06T22:14:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.352+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b019f1c0 con 0x7f57b0100e90 2023-10-06T22:14:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.353+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b01085a0 con 0x7f57b0100e90 2023-10-06T22:14:34.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.355+0000 7f57a67fc700 1 -- 172.21.15.57:0/2867977894 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f57ac021580 con 0x7f57b0100e90 2023-10-06T22:14:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.356+0000 7f57a67fc700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f579c0a1480 0x7f579c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.356+0000 7f57a67fc700 1 -- 172.21.15.57:0/2867977894 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f57ac0ce3a0 con 0x7f57b0100e90 2023-10-06T22:14:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.356+0000 7f57b499a700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f579c0a1480 0x7f579c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.357+0000 7f57b499a700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f579c0a1480 0x7f579c0a3940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57b0100bb0 tx=0x7f57a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.358+0000 7f57a67fc700 1 -- 172.21.15.57:0/2867977894 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f57ac097a60 con 0x7f57b0100e90 2023-10-06T22:14:34.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.571+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57b010c9a0 con 0x7f57b0100e90 2023-10-06T22:14:34.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.572+0000 7f57a67fc700 1 -- 172.21.15.57:0/2867977894 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57ac09b080 con 0x7f57b0100e90 2023-10-06T22:14:34.574 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:14:34.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f579c0a1480 msgr2=0x7f579c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f579c0a1480 0x7f579c0a3940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57b0100bb0 tx=0x7f57a0009260 comp rx=0 tx=0).stop 2023-10-06T22:14:34.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0100e90 msgr2=0x7f57b0196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0100e90 0x7f57b0196d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005720 tx=0x7f57ac009580 comp rx=0 tx=0).stop 2023-10-06T22:14:34.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 shutdown_connections 2023-10-06T22:14:34.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f579c0a1480 0x7f579c0a3940 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57b0102a30 0x7f57b019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b0100e90 0x7f57b0196d00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 --2- 172.21.15.57:0/2867977894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57b0102070 0x7f57b0197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 >> 172.21.15.57:0/2867977894 conn(0x7f57b00fc430 msgr2=0x7f57b01056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:34.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 shutdown_connections 2023-10-06T22:14:34.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.575+0000 7f57b73ff700 1 -- 172.21.15.57:0/2867977894 wait complete. 2023-10-06T22:14:34.596 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:14:34.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.800+0000 7f81e0fff700 1 -- 172.21.15.57:0/4178988064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc074ef0 msgr2=0x7f81dc073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.800+0000 7f81e0fff700 1 --2- 172.21.15.57:0/4178988064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc074ef0 0x7f81dc073340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009990 tx=0x7f81cc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.801+0000 7f81e0fff700 1 -- 172.21.15.57:0/4178988064 shutdown_connections 2023-10-06T22:14:34.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.801+0000 7f81e0fff700 1 --2- 172.21.15.57:0/4178988064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81dc108db0 0x7f81dc10b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.801+0000 7f81e0fff700 1 --2- 172.21.15.57:0/4178988064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc074ef0 0x7f81dc073340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.801+0000 7f81e0fff700 1 --2- 172.21.15.57:0/4178988064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81dc073910 0x7f81dc073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.801+0000 7f81e0fff700 1 -- 172.21.15.57:0/4178988064 >> 172.21.15.57:0/4178988064 conn(0x7f81dc0fc420 msgr2=0x7f81dc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:34.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.801+0000 7f81e0fff700 1 -- 172.21.15.57:0/4178988064 shutdown_connections 2023-10-06T22:14:34.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.801+0000 7f81e0fff700 1 -- 172.21.15.57:0/4178988064 wait complete. 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 Processor -- start 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 -- start start 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81dc073910 0x7f81dc198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81dc074ef0 0x7f81dc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc108db0 0x7f81dc1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc1a08f0 con 0x7f81dc108db0 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc1a0a30 con 0x7f81dc073910 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81e0fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc1a0bc0 con 0x7f81dc074ef0 2023-10-06T22:14:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.802+0000 7f81dad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc108db0 0x7f81dc1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc108db0 0x7f81dc1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33782/0 (socket says 172.21.15.57:33782) 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 -- 172.21.15.57:0/1032197199 learned_addr learned my addr 172.21.15.57:0/1032197199 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81d9d9b700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81dc074ef0 0x7f81dc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81da59c700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81dc073910 0x7f81dc198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 -- 172.21.15.57:0/1032197199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81dc074ef0 msgr2=0x7f81dc1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81dc074ef0 0x7f81dc1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 -- 172.21.15.57:0/1032197199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81dc073910 msgr2=0x7f81dc198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81dc073910 0x7f81dc198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 -- 172.21.15.57:0/1032197199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0095f0 con 0x7f81dc108db0 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81dad9d700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc108db0 0x7f81dc1a0320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007d70 tx=0x7f81d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81cb7fe700 1 -- 172.21.15.57:0/1032197199 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d400ac90 con 0x7f81dc108db0 2023-10-06T22:14:34.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.803+0000 7f81cb7fe700 1 -- 172.21.15.57:0/1032197199 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d400adf0 con 0x7f81dc108db0 2023-10-06T22:14:34.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.804+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc1a0ea0 con 0x7f81dc108db0 2023-10-06T22:14:34.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.804+0000 7f81cb7fe700 1 -- 172.21.15.57:0/1032197199 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d40115e0 con 0x7f81dc108db0 2023-10-06T22:14:34.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.804+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc1a12d0 con 0x7f81dc108db0 2023-10-06T22:14:34.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.805+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc107460 con 0x7f81dc108db0 2023-10-06T22:14:34.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.807+0000 7f81cb7fe700 1 -- 172.21.15.57:0/1032197199 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f81d4012420 con 0x7f81dc108db0 2023-10-06T22:14:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.808+0000 7f81cb7fe700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81c40a14d0 0x7f81c40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.808+0000 7f81cb7fe700 1 -- 172.21.15.57:0/1032197199 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f81d40ccef0 con 0x7f81dc108db0 2023-10-06T22:14:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.808+0000 7f81da59c700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81c40a14d0 0x7f81c40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:34.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.809+0000 7f81da59c700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81c40a14d0 0x7f81c40a3990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00b5c0 tx=0x7f81cc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:34.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:34.812+0000 7f81cb7fe700 1 -- 172.21.15.57:0/1032197199 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f81d4096520 con 0x7f81dc108db0 2023-10-06T22:14:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:34 smithi057 ceph-mon[98492]: pgmap v329: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-06T22:14:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:34 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/997080442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:34.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:34 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2867977894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:35.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:34 smithi156 ceph-mon[134358]: pgmap v329: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-06T22:14:35.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:34 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/997080442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:35.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:34 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2867977894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:34 smithi190 ceph-mon[134891]: pgmap v329: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-06T22:14:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:34 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/997080442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:14:35.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:34 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2867977894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:14:35.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.073+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81dc10e930 con 0x7f81dc108db0 2023-10-06T22:14:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.074+0000 7f81cb7fe700 1 -- 172.21.15.57:0/1032197199 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f81d4099b40 con 0x7f81dc108db0 2023-10-06T22:14:35.077 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:35.077 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:14:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81c40a14d0 msgr2=0x7f81c40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81c40a14d0 0x7f81c40a3990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00b5c0 tx=0x7f81cc005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc108db0 msgr2=0x7f81dc1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc108db0 0x7f81dc1a0320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007d70 tx=0x7f81d400a340 comp rx=0 tx=0).stop 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 shutdown_connections 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f81c40a14d0 0x7f81c40a3990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81dc108db0 0x7f81dc1a0320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81dc073910 0x7f81dc198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 --2- 172.21.15.57:0/1032197199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81dc074ef0 0x7f81dc1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 >> 172.21.15.57:0/1032197199 conn(0x7f81dc0fc420 msgr2=0x7f81dc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 shutdown_connections 2023-10-06T22:14:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.077+0000 7f81e0fff700 1 -- 172.21.15.57:0/1032197199 wait complete. 2023-10-06T22:14:35.081 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:14:35.098 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:14:35.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 -- 172.21.15.57:0/3573962706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080100ea0 msgr2=0x7fc0801012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 --2- 172.21.15.57:0/3573962706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080100ea0 0x7fc0801012c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc0740098f0 tx=0x7fc074009c00 comp rx=0 tx=0).stop 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 -- 172.21.15.57:0/3573962706 shutdown_connections 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 --2- 172.21.15.57:0/3573962706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080102a40 0x7fc080109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 --2- 172.21.15.57:0/3573962706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080100ea0 0x7fc0801012c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 --2- 172.21.15.57:0/3573962706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102080 0x7fc080102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 -- 172.21.15.57:0/3573962706 >> 172.21.15.57:0/3573962706 conn(0x7fc0800fc420 msgr2=0x7fc0800fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 -- 172.21.15.57:0/3573962706 shutdown_connections 2023-10-06T22:14:35.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.300+0000 7fc086bff700 1 -- 172.21.15.57:0/3573962706 wait complete. 2023-10-06T22:14:35.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 Processor -- start 2023-10-06T22:14:35.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 -- start start 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080100ea0 0x7fc080196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080102080 0x7fc080197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102a40 0x7fc08019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019e790 con 0x7fc080100ea0 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019e900 con 0x7fc080102080 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.301+0000 7fc086bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019eac0 con 0x7fc080102a40 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080100ea0 0x7fc080196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080100ea0 0x7fc080196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33804/0 (socket says 172.21.15.57:33804) 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 -- 172.21.15.57:0/3631234988 learned_addr learned my addr 172.21.15.57:0/3631234988 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08519c700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102a40 0x7fc08019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc07ffff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080102080 0x7fc080197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 -- 172.21.15.57:0/3631234988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102a40 msgr2=0x7fc08019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102a40 0x7fc08019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 -- 172.21.15.57:0/3631234988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080102080 msgr2=0x7fc080197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080102080 0x7fc080197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 -- 172.21.15.57:0/3631234988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc074009520 con 0x7fc080100ea0 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc07ffff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080102080 0x7fc080197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.302+0000 7fc08499b700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080100ea0 0x7fc080196c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc074009fd0 tx=0x7fc07400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.303+0000 7fc07dffb700 1 -- 172.21.15.57:0/3631234988 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07401c070 con 0x7fc080100ea0 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.303+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08019ed40 con 0x7fc080100ea0 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.303+0000 7fc07dffb700 1 -- 172.21.15.57:0/3631234988 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07400f7d0 con 0x7fc080100ea0 2023-10-06T22:14:35.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.303+0000 7fc07dffb700 1 -- 172.21.15.57:0/3631234988 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074003a40 con 0x7fc080100ea0 2023-10-06T22:14:35.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.303+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08019f1c0 con 0x7fc080100ea0 2023-10-06T22:14:35.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.304+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc080108510 con 0x7fc080100ea0 2023-10-06T22:14:35.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.306+0000 7fc07dffb700 1 -- 172.21.15.57:0/3631234988 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc074020030 con 0x7fc080100ea0 2023-10-06T22:14:35.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.308+0000 7fc07dffb700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc0700a14d0 0x7fc0700a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.308+0000 7fc07dffb700 1 -- 172.21.15.57:0/3631234988 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7fc0740cdca0 con 0x7fc080100ea0 2023-10-06T22:14:35.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.308+0000 7fc07ffff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc0700a14d0 0x7fc0700a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.309+0000 7fc07ffff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc0700a14d0 0x7fc0700a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc080100bc0 tx=0x7fc06c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.312+0000 7fc07dffb700 1 -- 172.21.15.57:0/3631234988 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc074097360 con 0x7fc080100ea0 2023-10-06T22:14:35.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.524+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc08010c760 con 0x7fc080100ea0 2023-10-06T22:14:35.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.526+0000 7fc07dffb700 1 -- 172.21.15.57:0/3631234988 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22514 (secure 0 0 0) 0x7fc07409a980 con 0x7fc080100ea0 2023-10-06T22:14:35.528 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:14:35.528 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":85,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:13:06.715910+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc0700a14d0 msgr2=0x7fc0700a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc0700a14d0 0x7fc0700a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc080100bc0 tx=0x7fc06c009260 comp rx=0 tx=0).stop 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080100ea0 msgr2=0x7fc080196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080100ea0 0x7fc080196c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc074009fd0 tx=0x7fc07400b790 comp rx=0 tx=0).stop 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 shutdown_connections 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7fc0700a14d0 0x7fc0700a3990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc080102a40 0x7fc08019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc080100ea0 0x7fc080196c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 --2- 172.21.15.57:0/3631234988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc080102080 0x7fc080197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 >> 172.21.15.57:0/3631234988 conn(0x7fc0800fc420 msgr2=0x7fc080105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 shutdown_connections 2023-10-06T22:14:35.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.528+0000 7fc086bff700 1 -- 172.21.15.57:0/3631234988 wait complete. 2023-10-06T22:14:35.549 DEBUG:tasks.fwd_scrub:Yielding 2023-10-06T22:14:35.550 INFO:teuthology.run_tasks:Running task exec... 2023-10-06T22:14:35.563 INFO:teuthology.task.exec:Executing custom commands... 2023-10-06T22:14:35.563 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:14:35.563 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-10-06T22:14:35.564 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-06T22:14:35.564 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T22:14:35.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T22:14:35.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 -- 172.21.15.57:0/3061083034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 msgr2=0x7f7800073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 --2- 172.21.15.57:0/3061083034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f7800073d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009990 tx=0x7f77f0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 -- 172.21.15.57:0/3061083034 shutdown_connections 2023-10-06T22:14:35.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 --2- 172.21.15.57:0/3061083034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f780010d4c0 0x7f780010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 --2- 172.21.15.57:0/3061083034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800074ef0 0x7f7800073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 --2- 172.21.15.57:0/3061083034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f7800073d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 -- 172.21.15.57:0/3061083034 >> 172.21.15.57:0/3061083034 conn(0x7f78000fc480 msgr2=0x7f78000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 -- 172.21.15.57:0/3061083034 shutdown_connections 2023-10-06T22:14:35.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.773+0000 7f78087ff700 1 -- 172.21.15.57:0/3061083034 wait complete. 2023-10-06T22:14:35.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.774+0000 7f78087ff700 1 Processor -- start 2023-10-06T22:14:35.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f78087ff700 1 -- start start 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f78087ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f780019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f78087ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800074ef0 0x7f780019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f78087ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f780010d4c0 0x7f780019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f78087ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780019ed20 con 0x7f780010d4c0 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f78087ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780019ee90 con 0x7f7800074ef0 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f78087ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780019f050 con 0x7f7800073910 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.775+0000 7f780659b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f780019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f7806d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f780010d4c0 0x7f780019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f780019d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44132/0 (socket says 172.21.15.57:44132) 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 -- 172.21.15.57:0/3516908029 learned_addr learned my addr 172.21.15.57:0/3516908029 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f7805d9a700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800074ef0 0x7f780019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 -- 172.21.15.57:0/3516908029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800074ef0 msgr2=0x7f780019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800074ef0 0x7f780019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 -- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f780010d4c0 msgr2=0x7f780019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f780010d4c0 0x7f780019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 -- 172.21.15.57:0/3516908029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00095f0 con 0x7f7800073910 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f780659b700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f780019d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9a0 tx=0x7f77fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc009790 con 0x7f7800073910 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77fc010430 con 0x7f7800073910 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc018680 con 0x7f7800073910 2023-10-06T22:14:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.776+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780019f330 con 0x7f7800073910 2023-10-06T22:14:35.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.777+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78001a5af0 con 0x7f7800073910 2023-10-06T22:14:35.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.778+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7800103010 con 0x7f7800073910 2023-10-06T22:14:35.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.780+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f77fc0105a0 con 0x7f7800073910 2023-10-06T22:14:35.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.781+0000 7f77f77fe700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f77ec0a1480 0x7f77ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.781+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f77fc014070 con 0x7f7800073910 2023-10-06T22:14:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.781+0000 7f7805d9a700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f77ec0a1480 0x7f77ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.782+0000 7f7805d9a700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f77ec0a1480 0x7f77ec0a3940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b2e0 tx=0x7f77f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.785+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f77fc052100 con 0x7f7800073910 2023-10-06T22:14:35.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 -- 172.21.15.57:0/1881021120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b00ff790 msgr2=0x7f35b00ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1881021120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b00ff790 0x7f35b00ffbb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35a00098f0 tx=0x7f35a0009c00 comp rx=0 tx=0).stop 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 -- 172.21.15.57:0/1881021120 shutdown_connections 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1881021120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 0x7f35b0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1881021120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b00ff790 0x7f35b00ffbb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1881021120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35b01000f0 0x7f35b0100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 -- 172.21.15.57:0/1881021120 >> 172.21.15.57:0/1881021120 conn(0x7f35b00fb370 msgr2=0x7f35b00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 -- 172.21.15.57:0/1881021120 shutdown_connections 2023-10-06T22:14:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.786+0000 7f35b55ff700 1 -- 172.21.15.57:0/1881021120 wait complete. 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 Processor -- start 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 -- start start 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35b00ff790 0x7f35b0198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b01000f0 0x7f35b01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 0x7f35b01a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01a0a00 con 0x7f35b00ff790 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01a0b40 con 0x7f35b0100ab0 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.787+0000 7f35b55ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01a0cd0 con 0x7f35b01000f0 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35ae7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b01000f0 0x7f35b01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 0x7f35b01a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35ae7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b01000f0 0x7f35b01993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44152/0 (socket says 172.21.15.57:44152) 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 0x7f35b01a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39304/0 (socket says 172.21.15.57:39304) 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35ae7fc700 1 -- 172.21.15.57:0/1980090768 learned_addr learned my addr 172.21.15.57:0/1980090768 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35aeffd700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35b00ff790 0x7f35b0198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 -- 172.21.15.57:0/1980090768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b01000f0 msgr2=0x7f35b01993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b01000f0 0x7f35b01993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 -- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35b00ff790 msgr2=0x7f35b0198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35b00ff790 0x7f35b0198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 -- 172.21.15.57:0/1980090768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a0009520 con 0x7f35b0100ab0 2023-10-06T22:14:35.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f35af7fe700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 0x7f35b01a0430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35a800eb80 tx=0x7f35a800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.788+0000 7f3597fff700 1 -- 172.21.15.57:0/1980090768 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8018070 con 0x7f35b0100ab0 2023-10-06T22:14:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.789+0000 7f3597fff700 1 -- 172.21.15.57:0/1980090768 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a800f040 con 0x7f35b0100ab0 2023-10-06T22:14:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.789+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b01a0f50 con 0x7f35b0100ab0 2023-10-06T22:14:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.789+0000 7f3597fff700 1 -- 172.21.15.57:0/1980090768 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800cc90 con 0x7f35b0100ab0 2023-10-06T22:14:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.789+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b01a1380 con 0x7f35b0100ab0 2023-10-06T22:14:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.792+0000 7f3597fff700 1 -- 172.21.15.57:0/1980090768 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f35a8003a50 con 0x7f35b0100ab0 2023-10-06T22:14:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.793+0000 7f35c8f4cb80 1 Processor -- start 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.793+0000 7f3597fff700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35980a1500 0x7f35980a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.793+0000 7f35c8f4cb80 1 -- start start 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f3597fff700 1 -- 172.21.15.57:0/1980090768 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f35a80cc8a0 con 0x7f35b0100ab0 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d8c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d8d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d8df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075cf44e0 con 0x565075d8c2e0 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35980a1500 0x7f35980a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075ce5ca0 con 0x565075d8cc40 2023-10-06T22:14:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075ce35c0 con 0x565075d8dae0 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d8c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d8c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33852/0 (socket says 172.21.15.57:33852) 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35aeffd700 1 -- 172.21.15.57:0/1710303627 learned_addr learned my addr 172.21.15.57:0/1710303627 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d8d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.794+0000 7f35af7fe700 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d8df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35aeffd700 1 -- 172.21.15.57:0/1710303627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 msgr2=0x565075d8df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35aeffd700 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d8df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35aeffd700 1 -- 172.21.15.57:0/1710303627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 msgr2=0x565075d8d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35aeffd700 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d8d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35aeffd700 1 -- 172.21.15.57:0/1710303627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565075b09e60 con 0x565075d8c2e0 2023-10-06T22:14:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35980a1500 0x7f35980a39c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009d30 tx=0x7f35a4011720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35aeffd700 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d8c700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f35a000fa40 tx=0x7f35a0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0e726374b85885 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35957fa700 1 -- 172.21.15.57:0/1710303627 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000f6e0 con 0x565075d8c2e0 2023-10-06T22:14:35.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35957fa700 1 -- 172.21.15.57:0/1710303627 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a001d040 con 0x565075d8c2e0 2023-10-06T22:14:35.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.795+0000 7f35957fa700 1 -- 172.21.15.57:0/1710303627 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0005570 con 0x565075d8c2e0 2023-10-06T22:14:35.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1710303627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 msgr2=0x565075d8c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d8c700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f35a000fa40 tx=0x7f35a0005dc0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1710303627 shutdown_connections 2023-10-06T22:14:35.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d8df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d8c700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1710303627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d8d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1710303627 >> 172.21.15.57:0/1710303627 conn(0x565075d002e0 msgr2=0x565075d871c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1710303627 shutdown_connections 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.796+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1710303627 wait complete. 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.797+0000 7f35c8f4cb80 1 -- start start 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.797+0000 7f35c8f4cb80 1 Processor -- start 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.797+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8c2e0 0x565075d96030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d96570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8dae0 0x565075d9d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075cdeeb0 con 0x565075d8dae0 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075c7b790 con 0x565075d8cc40 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075c7b430 con 0x565075d8c2e0 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35c8f4cb80 10 client.0 _ll_register_callbacks cb 0x565075ad8880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:35.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8dae0 0x565075d9d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8c2e0 0x565075d96030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8dae0 0x565075d9d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33866/0 (socket says 172.21.15.57:33866) 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 -- 172.21.15.57:0/3381461498 learned_addr learned my addr 172.21.15.57:0/3381461498 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d96570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 -- 172.21.15.57:0/3381461498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8c2e0 msgr2=0x565075d96030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8c2e0 0x565075d96030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 -- 172.21.15.57:0/3381461498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 msgr2=0x565075d96570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d96570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.798+0000 7f35af7fe700 1 -- 172.21.15.57:0/3381461498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b010bf20 con 0x565075d8dae0 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f35af7fe700 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8dae0 0x565075d9d5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f35a8099920 tx=0x7f35a809a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9046786d440137b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f3592ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:35.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f3592ff5700 1 -- 172.21.15.57:0/3381461498 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a801b320 con 0x565075d8dae0 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565075d7ef90 con 0x565075d8dae0 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f3592ff5700 1 -- 172.21.15.57:0/3381461498 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f35a809a320 con 0x565075d8dae0 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f3592ff5700 1 -- 172.21.15.57:0/3381461498 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a8099c20 con 0x565075d8dae0 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f35c8f4cb80 10 client.15354 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x565075af2ce0 con 0x565075d8dae0 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f3592ff5700 1 -- 172.21.15.57:0/3381461498 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a807c020 con 0x565075d8dae0 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f3592ff5700 1 -- 172.21.15.57:0/3381461498 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35a80a1a90 con 0x565075d8dae0 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f35c8f4cb80 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.799+0000 7f35c8f4cb80 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.800+0000 7f35c8f4cb80 20 client.15354 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.800+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075da1c90 0x565075da40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.800+0000 7f35c8f4cb80 4 client.15354 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:14:35.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.800+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565075cd7e90 con 0x565075da1c90 2023-10-06T22:14:35.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.800+0000 7f35aeffd700 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075da1c90 0x565075da40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.801+0000 7f35aeffd700 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075da1c90 0x565075da40b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a39daf9afc796d2 server_cookie=8aadd1280848475b in_seq=0 out_seq=0 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.801+0000 7f3592ff5700 10 client.15354 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.801+0000 7f3592ff5700 1 -- 172.21.15.57:0/3381461498 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565075cd7e90 con 0x565075da1c90 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.801+0000 7f3592ff5700 10 client.15354 handle_command_reply: tid=0 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.801+0000 7f35c8f4cb80 1 client.15354 shutdown 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.801+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8dae0 msgr2=0x565075d9d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.801+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8dae0 0x565075d9d5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f35a8099920 tx=0x7f35a809a160 comp rx=0 tx=0).stop 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 shutdown_connections 2023-10-06T22:14:35.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075da1c90 0x565075da40b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8dae0 0x565075d9d5b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8c2e0 0x565075d96030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/3381461498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d96570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f3592ff5700 0 client.15354 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 >> 172.21.15.57:0/3381461498 conn(0x565075d002e0 msgr2=0x565075d86fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 shutdown_connections 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/3381461498 wait complete. 2023-10-06T22:14:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.802+0000 7f35c8f4cb80 20 client.15354 trim_cache size 0 max 16384 2023-10-06T22:14:35.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 Processor -- start 2023-10-06T22:14:35.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 -- start start 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d93f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d9d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d86fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075b5f9a0 con 0x565075d8c2e0 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075da1a60 con 0x565075d8cc40 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075d96cb0 con 0x565075d8dae0 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35ae7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d9d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d93f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35af7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d86fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35ae7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d9d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39328/0 (socket says 172.21.15.57:39328) 2023-10-06T22:14:35.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.821+0000 7f35ae7fc700 1 -- 172.21.15.57:0/756579689 learned_addr learned my addr 172.21.15.57:0/756579689 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35ae7fc700 1 -- 172.21.15.57:0/756579689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 msgr2=0x565075d86fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d86fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35ae7fc700 1 -- 172.21.15.57:0/756579689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 msgr2=0x565075d93f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d93f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35ae7fc700 1 -- 172.21.15.57:0/756579689 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565075da8ce0 con 0x565075d8cc40 2023-10-06T22:14:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35af7fe700 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d86fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:35.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35aeffd700 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d93f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d9d570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f35a40208b0 tx=0x7f35a4021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a378805fea8c047 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f3595ffb700 1 -- 172.21.15.57:0/756579689 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4004a30 con 0x565075d8cc40 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f3595ffb700 1 -- 172.21.15.57:0/756579689 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a40054e0 con 0x565075d8cc40 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.822+0000 7f3595ffb700 1 -- 172.21.15.57:0/756579689 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a402b8d0 con 0x565075d8cc40 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/756579689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 msgr2=0x565075d9d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d9d570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f35a40208b0 tx=0x7f35a4021000 comp rx=0 tx=0).stop 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/756579689 shutdown_connections 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075d86fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8c2e0 0x565075d93f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/756579689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8cc40 0x565075d9d570 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/756579689 >> 172.21.15.57:0/756579689 conn(0x565075d002e0 msgr2=0x565075d94cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/756579689 shutdown_connections 2023-10-06T22:14:35.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.823+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/756579689 wait complete. 2023-10-06T22:14:35.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 -- start start 2023-10-06T22:14:35.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 Processor -- start 2023-10-06T22:14:35.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8c2e0 0x565075d9c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8cc40 0x565075d9cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075df5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075df6250 con 0x565075d8cc40 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075df6390 con 0x565075d8c2e0 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565075df6520 con 0x565075d8dae0 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.824+0000 7f35c8f4cb80 10 client.0 _ll_register_callbacks cb 0x7f35a8099760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8c2e0 0x565075d9c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35af7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075df5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8c2e0 0x565075d9c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39338/0 (socket says 172.21.15.57:39338) 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35aeffd700 1 -- 172.21.15.57:0/1970225452 learned_addr learned my addr 172.21.15.57:0/1970225452 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8cc40 0x565075d9cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35af7fe700 1 -- 172.21.15.57:0/1970225452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8c2e0 msgr2=0x565075d9c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35af7fe700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8c2e0 0x565075d9c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35af7fe700 1 -- 172.21.15.57:0/1970225452 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8cc40 msgr2=0x565075d9cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:35.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35af7fe700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8cc40 0x565075d9cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35af7fe700 1 -- 172.21.15.57:0/1970225452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565075b09e60 con 0x565075d8dae0 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35aeffd700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8c2e0 0x565075d9c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35ae7fc700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8cc40 0x565075d9cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35af7fe700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075df5d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35a8096120 tx=0x7f35a8003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf2139f08077d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f3592ff5700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f3592ff5700 1 -- 172.21.15.57:0/1970225452 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8095cb0 con 0x565075d8dae0 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35a001d040 con 0x565075d8dae0 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.825+0000 7f3592ff5700 1 -- 172.21.15.57:0/1970225452 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f35a807d080 con 0x565075d8dae0 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f3592ff5700 1 -- 172.21.15.57:0/1970225452 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a801b320 con 0x565075d8dae0 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 10 client.25381 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:35.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x565075af2ce0 con 0x565075d8dae0 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f3592ff5700 1 -- 172.21.15.57:0/1970225452 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a001d040 con 0x565075d8dae0 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f3592ff5700 1 -- 172.21.15.57:0/1970225452 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35a809eab0 con 0x565075d8dae0 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 10 client.25381 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 10 client.25381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 20 client.25381 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075df7bc0 0x565075df9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 4 client.25381 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x565075cd7bb0 con 0x565075df7bc0 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.826+0000 7f35aeffd700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075df7bc0 0x565075df9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.827+0000 7f35aeffd700 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075df7bc0 0x565075df9fe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f07fadf6de591b server_cookie=54c16483f657f7b1 in_seq=0 out_seq=0 2023-10-06T22:14:35.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.827+0000 7f3592ff5700 10 client.25381 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:35.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f3592ff5700 1 -- 172.21.15.57:0/1970225452 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x565075cd7bb0 con 0x565075df7bc0 2023-10-06T22:14:35.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f3592ff5700 10 client.25381 handle_command_reply: tid=0 2023-10-06T22:14:35.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 client.25381 shutdown 2023-10-06T22:14:35.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 msgr2=0x565075df5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075df5d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35a8096120 tx=0x7f35a8003f10 comp rx=0 tx=0).stop 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 shutdown_connections 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x565075df7bc0 0x565075df9fe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565075d8dae0 0x565075df5d10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565075d8c2e0 0x565075d9c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 --2- 172.21.15.57:0/1970225452 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565075d8cc40 0x565075d9cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.828+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 >> 172.21.15.57:0/1970225452 conn(0x565075d002e0 msgr2=0x565075d90a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f3592ff5700 0 client.25381 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 shutdown_connections 2023-10-06T22:14:35.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f35c8f4cb80 1 -- 172.21.15.57:0/1970225452 wait complete. 2023-10-06T22:14:35.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f35c8f4cb80 20 client.25381 trim_cache size 0 max 16384 2023-10-06T22:14:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:14:35.836 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-06T22:14:35.836 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "8feec67a-51cf-437b-b144-9f805790e0cb", 2023-10-06T22:14:35.836 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-06T22:14:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35980a1500 msgr2=0x7f35980a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35980a1500 0x7f35980a39c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009d30 tx=0x7f35a4011720 comp rx=0 tx=0).stop 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 msgr2=0x7f35b01a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.829+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 0x7f35b01a0430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35a800eb80 tx=0x7f35a800ee90 comp rx=0 tx=0).stop 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 shutdown_connections 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f35980a1500 0x7f35980a39c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35b0100ab0 0x7f35b01a0430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35b00ff790 0x7f35b0198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 --2- 172.21.15.57:0/1980090768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b01000f0 0x7f35b01993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:35.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 >> 172.21.15.57:0/1980090768 conn(0x7f35b00fb370 msgr2=0x7f35b00fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:35.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 shutdown_connections 2023-10-06T22:14:35.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:35.830+0000 7f35b55ff700 1 -- 172.21.15.57:0/1980090768 wait complete. 2023-10-06T22:14:35.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:35 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1032197199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:35.847 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:35 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3631234988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:35.850 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:14:36.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:35 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1032197199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:36.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:35 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3631234988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:36.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 -- 172.21.15.57:0/3142889397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af4074ef0 msgr2=0x7f7af4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/3142889397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af4074ef0 0x7f7af4073340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009990 tx=0x7f7ae4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 -- 172.21.15.57:0/3142889397 shutdown_connections 2023-10-06T22:14:36.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/3142889397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af410d4b0 0x7f7af410f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/3142889397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af4074ef0 0x7f7af4073340 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/3142889397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af4073910 0x7f7af4073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 -- 172.21.15.57:0/3142889397 >> 172.21.15.57:0/3142889397 conn(0x7f7af40fc430 msgr2=0x7f7af40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:36.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 -- 172.21.15.57:0/3142889397 shutdown_connections 2023-10-06T22:14:36.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.052+0000 7f7afa9ff700 1 -- 172.21.15.57:0/3142889397 wait complete. 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.053+0000 7f7afa9ff700 1 Processor -- start 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7afa9ff700 1 -- start start 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7afa9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af4073910 0x7f7af419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7afa9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af4074ef0 0x7f7af419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7afa9ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af410d4b0 0x7f7af41a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7afa9ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af41a4d60 con 0x7f7af410d4b0 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7afa9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af41a4ea0 con 0x7f7af4074ef0 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7afa9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af41a5030 con 0x7f7af4073910 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7af8f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af410d4b0 0x7f7af41a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af4073910 0x7f7af419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7af8f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af410d4b0 0x7f7af41a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33910/0 (socket says 172.21.15.57:33910) 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7af8f9c700 1 -- 172.21.15.57:0/759894697 learned_addr learned my addr 172.21.15.57:0/759894697 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.054+0000 7f7af3fff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af4074ef0 0x7f7af419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7aebfff700 1 -- 172.21.15.57:0/759894697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af4074ef0 msgr2=0x7f7af419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7aebfff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af4074ef0 0x7f7af419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7aebfff700 1 -- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af410d4b0 msgr2=0x7f7af41a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7aebfff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af410d4b0 0x7f7af41a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7aebfff700 1 -- 172.21.15.57:0/759894697 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc009520 con 0x7f7af4073910 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7aebfff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af4073910 0x7f7af419d1e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40098e0 tx=0x7f7ae4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7af1ffb700 1 -- 172.21.15.57:0/759894697 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae401d070 con 0x7f7af4073910 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae40095f0 con 0x7f7af4073910 2023-10-06T22:14:36.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7af1ffb700 1 -- 172.21.15.57:0/759894697 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae4021410 con 0x7f7af4073910 2023-10-06T22:14:36.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7af1ffb700 1 -- 172.21.15.57:0/759894697 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4005720 con 0x7f7af4073910 2023-10-06T22:14:36.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.055+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af41a5550 con 0x7f7af4073910 2023-10-06T22:14:36.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.059+0000 7f7af1ffb700 1 -- 172.21.15.57:0/759894697 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7ae4021580 con 0x7f7af4073910 2023-10-06T22:14:36.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.060+0000 7f7b0e327b80 1 Processor -- start 2023-10-06T22:14:36.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.060+0000 7f7b0e327b80 1 -- start start 2023-10-06T22:14:36.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.060+0000 7f7af1ffb700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7ae00a1480 0x7f7ae00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e33680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8d9b460 con 0x55d0e8e33260 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8d8cc20 con 0x55d0e8e34a60 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8d8a540 con 0x55d0e8e33bc0 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7af1ffb700 1 -- 172.21.15.57:0/759894697 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f7ae40ce060 con 0x7f7af4073910 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e33680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e34480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44228/0 (socket says 172.21.15.57:44228) 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7af3fff700 1 -- 172.21.15.57:0/2507287524 learned_addr learned my addr 172.21.15.57:0/2507287524 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e33680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33914/0 (socket says 172.21.15.57:33914) 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7af8f9c700 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.061+0000 7f7af8f9c700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7ae00a1480 0x7f7ae00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7af3fff700 1 -- 172.21.15.57:0/2507287524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 msgr2=0x55d0e8e34ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7af3fff700 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e34ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7af3fff700 1 -- 172.21.15.57:0/2507287524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 msgr2=0x55d0e8e33680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7af3fff700 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e33680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7af3fff700 1 -- 172.21.15.57:0/2507287524 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e8bb0de0 con 0x55d0e8e33bc0 2023-10-06T22:14:36.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7aebfff700 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e33680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7af3fff700 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e34480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009eb0 tx=0x7f7adc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28da2fbf7581790f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7af8f9c700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7ae00a1480 0x7f7ae00a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009d30 tx=0x7f7aec00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7aea7fc700 1 -- 172.21.15.57:0/2507287524 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc019630 con 0x55d0e8e33bc0 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7aea7fc700 1 -- 172.21.15.57:0/2507287524 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7adc019c50 con 0x55d0e8e33bc0 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.062+0000 7f7aea7fc700 1 -- 172.21.15.57:0/2507287524 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00ec50 con 0x55d0e8e33bc0 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2507287524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33bc0 msgr2=0x55d0e8e34480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e34480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009eb0 tx=0x7f7adc011720 comp rx=0 tx=0).stop 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2507287524 shutdown_connections 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e34ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e33680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2507287524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e34480 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2507287524 >> 172.21.15.57:0/2507287524 conn(0x55d0e8da7260 msgr2=0x55d0e8e2e140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:36.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2507287524 shutdown_connections 2023-10-06T22:14:36.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.063+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2507287524 wait complete. 2023-10-06T22:14:36.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 -- start start 2023-10-06T22:14:36.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 Processor -- start 2023-10-06T22:14:36.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e3d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e34a60 0x55d0e8e44590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8d85e30 con 0x55d0e8e34a60 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8d22710 con 0x55d0e8e33bc0 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8d223b0 con 0x55d0e8e33260 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.064+0000 7f7b0e327b80 10 client.0 _ll_register_callbacks cb 0x55d0e8b7f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af8f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e34a60 0x55d0e8e44590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e3d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e3d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39360/0 (socket says 172.21.15.57:39360) 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 -- 172.21.15.57:0/1828698045 learned_addr learned my addr 172.21.15.57:0/1828698045 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af8f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e34a60 0x55d0e8e44590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33926/0 (socket says 172.21.15.57:33926) 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 -- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 msgr2=0x55d0e8e3d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 -- 172.21.15.57:0/1828698045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e34a60 msgr2=0x55d0e8e44590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e34a60 0x55d0e8e44590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 -- 172.21.15.57:0/1828698045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af41a5550 con 0x55d0e8e33bc0 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af8f9c700 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e34a60 0x55d0e8e44590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7aebfff700 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7af3fff700 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e3d550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7adc011b20 tx=0x7f7adc00ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9425e0b2368c03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.065+0000 7f7ad37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1828698045 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc019840 con 0x55d0e8e33bc0 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0e8e25f10 con 0x55d0e8e33bc0 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1828698045 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f7adc0199a0 con 0x55d0e8e33bc0 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1828698045 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7adc009960 con 0x55d0e8e33bc0 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7b0e327b80 10 client.25379 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:36.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f7adc009ae0 con 0x55d0e8e33bc0 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1828698045 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0e8e25f10 con 0x55d0e8e33bc0 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.066+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1828698045 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7adc028020 con 0x55d0e8e33bc0 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7b0e327b80 10 client.25379 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7b0e327b80 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7b0e327b80 20 client.25379 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e48c00 0x55d0e8e4b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7b0e327b80 4 client.25379 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0e8d7ee10 con 0x55d0e8e48c00 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7aebfff700 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e48c00 0x55d0e8e4b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.067+0000 7f7aebfff700 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e48c00 0x55d0e8e4b020 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cca1e6bdffec076 server_cookie=634ce7e90ee8f274 in_seq=0 out_seq=0 2023-10-06T22:14:36.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7ad37fe700 10 client.25379 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1828698045 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d0e8d7ee10 con 0x55d0e8e48c00 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7ad37fe700 10 client.25379 handle_command_reply: tid=0 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7b0e327b80 1 client.25379 shutdown 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 msgr2=0x55d0e8e3d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e3d550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7adc011b20 tx=0x7f7adc00ee20 comp rx=0 tx=0).stop 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 shutdown_connections 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e48c00 0x55d0e8e4b020 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e34a60 0x55d0e8e44590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.069+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1828698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e3d550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.068+0000 7f7ad37fe700 0 client.25379 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:36.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.069+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 >> 172.21.15.57:0/1828698045 conn(0x55d0e8da7260 msgr2=0x55d0e8e2df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:36.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.069+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 shutdown_connections 2023-10-06T22:14:36.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.069+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1828698045 wait complete. 2023-10-06T22:14:36.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.069+0000 7f7b0e327b80 20 client.25379 trim_cache size 0 max 16384 2023-10-06T22:14:36.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:35 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1032197199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:14:36.076 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:35 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3631234988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:14:36.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.076+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f780004ec90 con 0x7f7800073910 2023-10-06T22:14:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.086+0000 7f7b0e327b80 1 Processor -- start 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7b0e327b80 1 -- start start 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e44550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e34a60 0x55d0e8e3d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8e3cbe0 con 0x55d0e8e33260 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8c06920 con 0x55d0e8e33bc0 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8e486f0 con 0x55d0e8e34a60 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33930/0 (socket says 172.21.15.57:33930) 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 -- 172.21.15.57:0/2053038259 learned_addr learned my addr 172.21.15.57:0/2053038259 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7af3fff700 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e44550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7af8f9c700 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e34a60 0x55d0e8e3d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 -- 172.21.15.57:0/2053038259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e34a60 msgr2=0x55d0e8e3d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e34a60 0x55d0e8e3d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 -- 172.21.15.57:0/2053038259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 msgr2=0x55d0e8e44550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e44550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.087+0000 7f7aebfff700 1 -- 172.21.15.57:0/2053038259 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e8b99c60 con 0x55d0e8e33260 2023-10-06T22:14:36.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.088+0000 7f7aebfff700 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40cbb30 tx=0x7f7ae4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f84f6ad66abe1b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:36.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.088+0000 7f7aeaffd700 1 -- 172.21.15.57:0/2053038259 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40b0070 con 0x55d0e8e33260 2023-10-06T22:14:36.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.088+0000 7f7aeaffd700 1 -- 172.21.15.57:0/2053038259 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae409bae0 con 0x55d0e8e33260 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.088+0000 7f7aeaffd700 1 -- 172.21.15.57:0/2053038259 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4099780 con 0x55d0e8e33260 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2053038259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 msgr2=0x55d0e8e3d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40cbb30 tx=0x7f7ae4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2053038259 shutdown_connections 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e34a60 0x55d0e8e3d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33260 0x55d0e8e3d010 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/2053038259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e44550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2053038259 >> 172.21.15.57:0/2053038259 conn(0x55d0e8da7260 msgr2=0x55d0e8e3bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2053038259 shutdown_connections 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 -- 172.21.15.57:0/2053038259 wait complete. 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 -- start start 2023-10-06T22:14:36.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.089+0000 7f7b0e327b80 1 Processor -- start 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e41fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7b0e327b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8e43a30 con 0x55d0e8e33bc0 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8e43ba0 con 0x55d0e8e34a60 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7b0e327b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e8e43d60 con 0x55d0e8e33260 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7b0e327b80 10 client.0 _ll_register_callbacks cb 0x55d0e8b7f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33944/0 (socket says 172.21.15.57:33944) 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 -- 172.21.15.57:0/1415652212 learned_addr learned my addr 172.21.15.57:0/1415652212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af8f9c700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7aebfff700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e41fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 -- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 msgr2=0x55d0e8e41fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e41fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 -- 172.21.15.57:0/1415652212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 msgr2=0x55d0e8e434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7af3fff700 1 -- 172.21.15.57:0/1415652212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae40095f0 con 0x55d0e8e33bc0 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.090+0000 7f7aebfff700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e41fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7af3fff700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e424e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7adc01c040 tx=0x7f7adc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff56a305249f324 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7ad37fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1415652212 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00e6d0 con 0x55d0e8e33bc0 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7adc019c50 con 0x55d0e8e33bc0 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1415652212 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f7adc000e40 con 0x55d0e8e33bc0 2023-10-06T22:14:36.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1415652212 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7adc004dd0 con 0x55d0e8e33bc0 2023-10-06T22:14:36.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7b0e327b80 10 client.15378 fetch_fsmap learned FSMap version 26 2023-10-06T22:14:36.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d0e8bd7250 con 0x55d0e8e33bc0 2023-10-06T22:14:36.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.091+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1415652212 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7adc03e020 con 0x55d0e8e33bc0 2023-10-06T22:14:36.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1415652212 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7adc011840 con 0x55d0e8e33bc0 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7b0e327b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7b0e327b80 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7b0e327b80 20 client.15378 populate_metadata read hostname 'smithi057' 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e9d1e0 0x55d0e8e9f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7b0e327b80 4 client.15378 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0e8d7eb30 con 0x55d0e8e9d1e0 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.092+0000 7f7aebfff700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e9d1e0 0x55d0e8e9f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7aebfff700 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e9d1e0 0x55d0e8e9f5d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97d401fd60806c5 server_cookie=3c637c9a5d85d567 in_seq=0 out_seq=0 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7ad37fe700 10 client.15378 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7ad37fe700 1 -- 172.21.15.57:0/1415652212 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d0e8d7eb30 con 0x55d0e8e9d1e0 2023-10-06T22:14:36.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7ad37fe700 10 client.15378 handle_command_reply: tid=0 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 client.15378 shutdown 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33bc0 msgr2=0x55d0e8e424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e424e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7adc01c040 tx=0x7f7adc005e50 comp rx=0 tx=0).stop 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 shutdown_connections 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d0e8e9d1e0 0x55d0e8e9f5d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0e8e34a60 0x55d0e8e434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0e8e33260 0x55d0e8e41fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7ad37fe700 0 client.15378 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 --2- 172.21.15.57:0/1415652212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d0e8e33bc0 0x55d0e8e424e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 >> 172.21.15.57:0/1415652212 conn(0x55d0e8da7260 msgr2=0x55d0e8e3cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 shutdown_connections 2023-10-06T22:14:36.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 1 -- 172.21.15.57:0/1415652212 wait complete. 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.093+0000 7f7b0e327b80 20 client.15378 trim_cache size 0 max 16384 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout: "8feec67a-51cf-437b-b144-9f805790e0cb": { 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "8feec67a-51cf-437b-b144-9f805790e0cb", 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:14:36.101 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:14:36.102 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:14:36.102 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:14:36.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7ae00a1480 msgr2=0x7f7ae00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7ae00a1480 0x7f7ae00a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009d30 tx=0x7f7aec00b040 comp rx=0 tx=0).stop 2023-10-06T22:14:36.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af4073910 msgr2=0x7f7af419d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af4073910 0x7f7af419d1e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40098e0 tx=0x7f7ae4009580 comp rx=0 tx=0).stop 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 shutdown_connections 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f7ae00a1480 0x7f7ae00a3940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7af410d4b0 0x7f7af41a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af4073910 0x7f7af419d1e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 --2- 172.21.15.57:0/759894697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af4074ef0 0x7f7af419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 >> 172.21.15.57:0/759894697 conn(0x7f7af40fc430 msgr2=0x7f7af40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 shutdown_connections 2023-10-06T22:14:36.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.094+0000 7f7afa9ff700 1 -- 172.21.15.57:0/759894697 wait complete. 2023-10-06T22:14:36.113 INFO:tasks.cephfs.filesystem:scrub status for tag:8feec67a-51cf-437b-b144-9f805790e0cb - {'path': '/', 'tag': '8feec67a-51cf-437b-b144-9f805790e0cb', 'options': 'recursive,force'} 2023-10-06T22:14:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.764+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f77fc010880 con 0x7f7800073910 2023-10-06T22:14:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.764+0000 7f77f77fe700 1 -- 172.21.15.57:0/3516908029 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f77fc05ba20 con 0x7f7800073910 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f77ec0a1480 msgr2=0x7f77ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f77ec0a1480 0x7f77ec0a3940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b2e0 tx=0x7f77f0005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 msgr2=0x7f780019d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f780019d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9a0 tx=0x7f77fc00ecb0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 shutdown_connections 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f77ec0a1480 0x7f77ec0a3940 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f780010d4c0 0x7f780019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7800073910 0x7f780019d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 --2- 172.21.15.57:0/3516908029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800074ef0 0x7f780019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:36.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.768+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 >> 172.21.15.57:0/3516908029 conn(0x7f78000fc480 msgr2=0x7f78000fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:36.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.769+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 shutdown_connections 2023-10-06T22:14:36.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:36.769+0000 7f78087ff700 1 -- 172.21.15.57:0/3516908029 wait complete. 2023-10-06T22:14:36.801 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-10-06T22:14:37.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: ignoring --setuser ceph since I am not root 2023-10-06T22:14:37.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: ignoring --setgroup ceph since I am not root 2023-10-06T22:14:37.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mgr[135842]: -- 172.21.15.156:0/2359963953 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b3d2bd1e0 con 0x559b3d292800 2023-10-06T22:14:37.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:36.931+0000 7fb3c0b81200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:14:37.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mon[134358]: pgmap v330: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-06T22:14:37.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mon[134358]: scrub queued for path: / 2023-10-06T22:14:37.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:14:37.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:14:37.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:14:37.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3516908029' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T22:14:37.005 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:36 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T22:14:37.012 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mon[134891]: pgmap v330: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-06T22:14:37.012 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mon[134891]: scrub queued for path: / 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3516908029' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: ignoring --setuser ceph since I am not root 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: ignoring --setgroup ceph since I am not root 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-mgr[137039]: -- 172.21.15.190:0/2012059562 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f12cba11e0 con 0x55f12cb77000 2023-10-06T22:14:37.013 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:36.930+0000 7f24915d7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:14:37.014 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:37.010+0000 7f24915d7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:14:37.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 -- 172.21.15.57:0/129382108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 msgr2=0x7f4138073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 --2- 172.21.15.57:0/129382108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138073d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4120009990 tx=0x7f4120009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 -- 172.21.15.57:0/129382108 shutdown_connections 2023-10-06T22:14:37.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 --2- 172.21.15.57:0/129382108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138108dc0 0x7f413810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 --2- 172.21.15.57:0/129382108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4138074ef0 0x7f4138073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 --2- 172.21.15.57:0/129382108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138073d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 -- 172.21.15.57:0/129382108 >> 172.21.15.57:0/129382108 conn(0x7f41380fc430 msgr2=0x7f41380fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 -- 172.21.15.57:0/129382108 shutdown_connections 2023-10-06T22:14:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.013+0000 7f413cdff700 1 -- 172.21.15.57:0/129382108 wait complete. 2023-10-06T22:14:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 Processor -- start 2023-10-06T22:14:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 -- start start 2023-10-06T22:14:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138074ef0 0x7f4138199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4138108dc0 0x7f413819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819a960 con 0x7f4138108dc0 2023-10-06T22:14:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819aad0 con 0x7f4138073910 2023-10-06T22:14:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.014+0000 7f413cdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819ac90 con 0x7f4138074ef0 2023-10-06T22:14:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f413659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138074ef0 0x7f4138199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39414/0 (socket says 172.21.15.57:39414) 2023-10-06T22:14:37.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f413659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138074ef0 0x7f4138199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44260/0 (socket says 172.21.15.57:44260) 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 -- 172.21.15.57:0/3390325635 learned_addr learned my addr 172.21.15.57:0/3390325635 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f4136d9d700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4138108dc0 0x7f413819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 -- 172.21.15.57:0/3390325635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138074ef0 msgr2=0x7f4138199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138074ef0 0x7f4138199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 -- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4138108dc0 msgr2=0x7f413819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4138108dc0 0x7f413819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 -- 172.21.15.57:0/3390325635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200095f0 con 0x7f4138073910 2023-10-06T22:14:37.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f413659c700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138074ef0 0x7f4138199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f4136d9d700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4138108dc0 0x7f413819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.015+0000 7f412e59c700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138198f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f412800e9a0 tx=0x7f412800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.016+0000 7f412ffff700 1 -- 172.21.15.57:0/3390325635 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128009790 con 0x7f4138073910 2023-10-06T22:14:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.016+0000 7f412ffff700 1 -- 172.21.15.57:0/3390325635 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4128010430 con 0x7f4138073910 2023-10-06T22:14:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.016+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413819af70 con 0x7f4138073910 2023-10-06T22:14:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.016+0000 7f412ffff700 1 -- 172.21.15.57:0/3390325635 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128018680 con 0x7f4138073910 2023-10-06T22:14:37.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.016+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41381a1730 con 0x7f4138073910 2023-10-06T22:14:37.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.017+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138107470 con 0x7f4138073910 2023-10-06T22:14:37.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.019+0000 7f412ffff700 1 -- 172.21.15.57:0/3390325635 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f41280105a0 con 0x7f4138073910 2023-10-06T22:14:37.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.020+0000 7f412ffff700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 0x7f41240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.020+0000 7f413659c700 1 -- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 msgr2=0x7f41240a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:37.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.020+0000 7f412ffff700 1 -- 172.21.15.57:0/3390325635 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f4128014070 con 0x7f4138073910 2023-10-06T22:14:37.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.020+0000 7f413659c700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 0x7f41240a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:14:37.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.022+0000 7f412ffff700 1 -- 172.21.15.57:0/3390325635 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f41280965a0 con 0x7f4138073910 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: ignoring --setuser ceph since I am not root 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: ignoring --setgroup ceph since I am not root 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:37.056+0000 7fde09419200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-mon[98492]: pgmap v330: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-mon[98492]: scrub queued for path: / 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:14:37.059 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:14:37.060 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3516908029' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T22:14:37.060 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:36 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T22:14:37.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.221+0000 7f413659c700 1 -- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 msgr2=0x7f41240a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:37.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.221+0000 7f413659c700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 0x7f41240a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T22:14:37.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.236+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f413804ec90 con 0x7f4138073910 2023-10-06T22:14:37.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.243+0000 7f412ffff700 1 -- 172.21.15.57:0/3390325635 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4128099bc0 con 0x7f4138073910 2023-10-06T22:14:37.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 msgr2=0x7f41240a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:14:37.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 0x7f41240a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 msgr2=0x7f4138198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138198f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f412800e9a0 tx=0x7f412800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 shutdown_connections 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f41240a14d0 0x7f41240a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4138108dc0 0x7f413819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4138073910 0x7f4138198f40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 --2- 172.21.15.57:0/3390325635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4138074ef0 0x7f4138199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 >> 172.21.15.57:0/3390325635 conn(0x7f41380fc430 msgr2=0x7f41380fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 shutdown_connections 2023-10-06T22:14:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.245+0000 7f413cdff700 1 -- 172.21.15.57:0/3390325635 wait complete. 2023-10-06T22:14:37.266 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-10-06T22:14:37.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:37.101+0000 7f24915d7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:14:37.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:37.193+0000 7f24915d7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:14:37.333 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:37.164+0000 7fde09419200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:14:37.333 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:37.290+0000 7fde09419200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:14:37.374 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:37.012+0000 7fb3c0b81200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T22:14:37.374 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:37.104+0000 7fb3c0b81200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T22:14:37.374 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:37.198+0000 7fb3c0b81200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:14:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.473+0000 7faa925ff700 1 -- 172.21.15.57:0/4204123534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c074ef0 msgr2=0x7faa8c073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.473+0000 7faa925ff700 1 --2- 172.21.15.57:0/4204123534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c074ef0 0x7faa8c073340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009990 tx=0x7faa7c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.474+0000 7faa925ff700 1 -- 172.21.15.57:0/4204123534 shutdown_connections 2023-10-06T22:14:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.474+0000 7faa925ff700 1 --2- 172.21.15.57:0/4204123534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa8c108dc0 0x7faa8c10b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.474+0000 7faa925ff700 1 --2- 172.21.15.57:0/4204123534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c074ef0 0x7faa8c073340 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.474+0000 7faa925ff700 1 --2- 172.21.15.57:0/4204123534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa8c073910 0x7faa8c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.474+0000 7faa925ff700 1 -- 172.21.15.57:0/4204123534 >> 172.21.15.57:0/4204123534 conn(0x7faa8c0fc430 msgr2=0x7faa8c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.474+0000 7faa925ff700 1 -- 172.21.15.57:0/4204123534 shutdown_connections 2023-10-06T22:14:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.474+0000 7faa925ff700 1 -- 172.21.15.57:0/4204123534 wait complete. 2023-10-06T22:14:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.475+0000 7faa925ff700 1 Processor -- start 2023-10-06T22:14:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.475+0000 7faa925ff700 1 -- start start 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa925ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa8c073910 0x7faa8c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa925ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa8c074ef0 0x7faa8c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa925ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c108dc0 0x7faa8c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa925ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c19a940 con 0x7faa8c074ef0 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa925ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c19aab0 con 0x7faa8c108dc0 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa925ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c19ac70 con 0x7faa8c073910 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa8bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa8c073910 0x7faa8c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa8bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa8c073910 0x7faa8c198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44288/0 (socket says 172.21.15.57:44288) 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa90b9c700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c108dc0 0x7faa8c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa8bfff700 1 -- 172.21.15.57:0/2505156668 learned_addr learned my addr 172.21.15.57:0/2505156668 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.476+0000 7faa8b7fe700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa8c074ef0 0x7faa8c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa90b9c700 1 -- 172.21.15.57:0/2505156668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa8c073910 msgr2=0x7faa8c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa90b9c700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa8c073910 0x7faa8c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa90b9c700 1 -- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa8c074ef0 msgr2=0x7faa8c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa90b9c700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa8c074ef0 0x7faa8c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa90b9c700 1 -- 172.21.15.57:0/2505156668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0095f0 con 0x7faa8c108dc0 2023-10-06T22:14:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa90b9c700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c108dc0 0x7faa8c19a2e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa8400efc0 tx=0x7faa8400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa897fa700 1 -- 172.21.15.57:0/2505156668 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84018070 con 0x7faa8c108dc0 2023-10-06T22:14:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa897fa700 1 -- 172.21.15.57:0/2505156668 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa8400f040 con 0x7faa8c108dc0 2023-10-06T22:14:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c19aef0 con 0x7faa8c108dc0 2023-10-06T22:14:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa897fa700 1 -- 172.21.15.57:0/2505156668 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8400cd40 con 0x7faa8c108dc0 2023-10-06T22:14:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.477+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c1a1710 con 0x7faa8c108dc0 2023-10-06T22:14:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.478+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c107470 con 0x7faa8c108dc0 2023-10-06T22:14:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.480+0000 7faa897fa700 1 -- 172.21.15.57:0/2505156668 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faa84003a00 con 0x7faa8c108dc0 2023-10-06T22:14:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.481+0000 7faa897fa700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 0x7faa740a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.482+0000 7faa8bfff700 1 -- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 msgr2=0x7faa740a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.482+0000 7faa897fa700 1 -- 172.21.15.57:0/2505156668 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7faa840cc8d0 con 0x7faa8c108dc0 2023-10-06T22:14:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.482+0000 7faa8bfff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 0x7faa740a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:14:37.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.485+0000 7faa897fa700 1 -- 172.21.15.57:0/2505156668 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faa84095f90 con 0x7faa8c108dc0 2023-10-06T22:14:37.659 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:37.422+0000 7fde09419200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T22:14:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.683+0000 7faa8bfff700 1 -- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 msgr2=0x7faa740a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:37.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.683+0000 7faa8bfff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 0x7faa740a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T22:14:37.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.696+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7faa8c04ec90 con 0x7faa8c108dc0 2023-10-06T22:14:37.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.703+0000 7faa897fa700 1 -- 172.21.15.57:0/2505156668 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7faa8401b320 con 0x7faa8c108dc0 2023-10-06T22:14:37.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 msgr2=0x7faa740a3a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:14:37.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 0x7faa740a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c108dc0 msgr2=0x7faa8c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c108dc0 0x7faa8c19a2e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa8400efc0 tx=0x7faa8400c3a0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 shutdown_connections 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7faa740a1550 0x7faa740a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c108dc0 0x7faa8c19a2e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa8c073910 0x7faa8c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 --2- 172.21.15.57:0/2505156668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa8c074ef0 0x7faa8c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 >> 172.21.15.57:0/2505156668 conn(0x7faa8c0fc430 msgr2=0x7faa8c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 shutdown_connections 2023-10-06T22:14:37.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.706+0000 7faa925ff700 1 -- 172.21.15.57:0/2505156668 wait complete. 2023-10-06T22:14:37.728 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-10-06T22:14:37.730 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:37.364+0000 7f24915d7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:14:37.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:37.657+0000 7fde09419200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:14:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T22:14:37.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-mon[98492]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-10-06T22:14:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' 2023-10-06T22:14:37.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:37 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' 2023-10-06T22:14:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 -- 172.21.15.57:0/1033775760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c074ef0 msgr2=0x7f524c073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 --2- 172.21.15.57:0/1033775760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c074ef0 0x7f524c073340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f523c009990 tx=0x7f523c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 -- 172.21.15.57:0/1033775760 shutdown_connections 2023-10-06T22:14:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 --2- 172.21.15.57:0/1033775760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f524c108dc0 0x7f524c10b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 --2- 172.21.15.57:0/1033775760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c074ef0 0x7f524c073340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 --2- 172.21.15.57:0/1033775760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f524c073910 0x7f524c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 -- 172.21.15.57:0/1033775760 >> 172.21.15.57:0/1033775760 conn(0x7f524c0fc430 msgr2=0x7f524c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 -- 172.21.15.57:0/1033775760 shutdown_connections 2023-10-06T22:14:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.933+0000 7f5251fff700 1 -- 172.21.15.57:0/1033775760 wait complete. 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.934+0000 7f5251fff700 1 Processor -- start 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.934+0000 7f5251fff700 1 -- start start 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f5251fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c073910 0x7f524c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f5251fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f524c074ef0 0x7f524c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f5251fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f524c108dc0 0x7f524c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f5251fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c19a940 con 0x7f524c108dc0 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f5251fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c19aab0 con 0x7f524c073910 2023-10-06T22:14:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f5251fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c19ac70 con 0x7f524c074ef0 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f524bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f524c108dc0 0x7f524c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f524b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f524c074ef0 0x7f524c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c073910 0x7f524c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c073910 0x7f524c198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39440/0 (socket says 172.21.15.57:39440) 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 -- 172.21.15.57:0/3508431723 learned_addr learned my addr 172.21.15.57:0/3508431723 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f524b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f524c074ef0 0x7f524c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44310/0 (socket says 172.21.15.57:44310) 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f524c074ef0 msgr2=0x7f524c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f524c074ef0 0x7f524c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f524c108dc0 msgr2=0x7f524c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f524c108dc0 0x7f524c19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.935+0000 7f52437fe700 1 -- 172.21.15.57:0/3508431723 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0095f0 con 0x7f524c073910 2023-10-06T22:14:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.936+0000 7f52437fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c073910 0x7f524c198f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f523c00b680 tx=0x7f523c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.936+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c01d070 con 0x7f524c073910 2023-10-06T22:14:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.936+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523c022470 con 0x7f524c073910 2023-10-06T22:14:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.936+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c19aef0 con 0x7f524c073910 2023-10-06T22:14:37.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.936+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c017650 con 0x7f524c073910 2023-10-06T22:14:37.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.936+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c1a1710 con 0x7f524c073910 2023-10-06T22:14:37.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.937+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c107470 con 0x7f524c073910 2023-10-06T22:14:37.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.939+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f523c0225e0 con 0x7f524c073910 2023-10-06T22:14:37.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.940+0000 7f52497fa700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:37.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.940+0000 7f524b7fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 msgr2=0x7f52380a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:37.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.940+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8190+0+0 (secure 0 0 0) 0x7f523c0ce390 con 0x7f524c073910 2023-10-06T22:14:37.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.940+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T22:14:37.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:37.942+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f523c097a50 con 0x7f524c073910 2023-10-06T22:14:38.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T22:14:38.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-mon[134358]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-10-06T22:14:38.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' 2023-10-06T22:14:38.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' 2023-10-06T22:14:38.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:37.372+0000 7fb3c0b81200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T22:14:38.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:37.740+0000 7fb3c0b81200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:14:38.004 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:37.874+0000 7fb3c0b81200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:14:38.005 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:37.964+0000 7fb3c0b81200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:14:38.051 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:37.728+0000 7f24915d7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:14:38.051 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:37.864+0000 7f24915d7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:14:38.051 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:37.954+0000 7f24915d7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:14:38.051 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T22:14:38.052 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-mon[134891]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-10-06T22:14:38.052 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' 2023-10-06T22:14:38.052 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:37 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' 2023-10-06T22:14:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:38.141+0000 7f524b7fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 msgr2=0x7f52380a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:38.141+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T22:14:38.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:38.184+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 --> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f524c0612f0 con 0x7f52380a14d0 2023-10-06T22:14:38.319 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:38 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:38.049+0000 7f24915d7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:14:38.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:38.142+0000 7fde09419200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T22:14:38.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:38.311+0000 7fde09419200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T22:14:38.487 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:38 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:38.061+0000 7fb3c0b81200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:14:38.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:38.542+0000 7f524b7fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 msgr2=0x7f52380a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:38.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:38.542+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T22:14:38.753 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:38 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:38.485+0000 7fb3c0b81200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:14:38.819 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:38 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:38.470+0000 7f24915d7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:14:38.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:38.431+0000 7fde09419200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T22:14:38.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:38.560+0000 7fde09419200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T22:14:39.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:39.343+0000 7f524b7fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 msgr2=0x7f52380a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:39.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:39.343+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T22:14:39.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:39 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:39.114+0000 7fde09419200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T22:14:39.503 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:39 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:39.153+0000 7fb3c0b81200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:14:39.503 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:39 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:39.235+0000 7fb3c0b81200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:14:39.569 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:39.128+0000 7f24915d7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:14:39.569 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:39.209+0000 7f24915d7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:14:39.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:39 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:14:39.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:39 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:14:40.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:39 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:14:40.250 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:39.983+0000 7f24915d7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:14:40.251 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:40.075+0000 7f24915d7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:14:40.288 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:40.019+0000 7fb3c0b81200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:14:40.288 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:40.112+0000 7fb3c0b81200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:14:40.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:39 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:39.992+0000 7fde09419200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T22:14:40.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:40 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:40.102+0000 7fde09419200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T22:14:40.529 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:40.248+0000 7f24915d7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:14:40.572 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:40.286+0000 7fb3c0b81200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:14:40.819 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:40.527+0000 7f24915d7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:14:40.819 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:40.757+0000 7f24915d7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:14:40.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:40.945+0000 7f524b7fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 msgr2=0x7f52380a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:40.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:40.945+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T22:14:40.955 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:40.569+0000 7fb3c0b81200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:14:40.955 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:40.801+0000 7fb3c0b81200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:14:41.173 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:40.907+0000 7f24915d7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:14:41.221 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:40.953+0000 7fb3c0b81200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:14:41.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:41.095+0000 7fde09419200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:14:41.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:41.219+0000 7fde09419200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T22:14:41.503 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:41 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:41.219+0000 7fb3c0b81200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:14:41.569 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:41.171+0000 7f24915d7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:14:41.833 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:41.452+0000 7fde09419200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T22:14:42.117 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:41.831+0000 7fde09419200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T22:14:42.205 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:41.876+0000 7f24915d7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:14:42.205 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:42.007+0000 7f24915d7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:14:42.253 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:41 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:41.935+0000 7fb3c0b81200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:14:42.253 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:42.068+0000 7fb3c0b81200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:14:42.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:42.114+0000 7fde09419200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T22:14:42.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:42.316+0000 7fde09419200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T22:14:42.535 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:42.203+0000 7f24915d7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:14:42.535 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:42.315+0000 7f24915d7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:14:42.535 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:42.533+0000 7f24915d7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:14:42.600 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:42.264+0000 7fb3c0b81200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:14:42.600 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:42.379+0000 7fb3c0b81200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:14:43.003 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:42.597+0000 7fb3c0b81200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:14:43.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:42.673+0000 7fde09419200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T22:14:43.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:43.479+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f523c097e20 con 0x7f524c073910 2023-10-06T22:14:43.753 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:14:43 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y[135817]: 2023-10-06T22:14:43.479+0000 7fb3c0b81200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:14:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:43 smithi156 ceph-mon[134358]: Standby manager daemon z restarted 2023-10-06T22:14:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:43 smithi156 ceph-mon[134358]: Standby manager daemon z started 2023-10-06T22:14:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:43 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T22:14:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:43 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:14:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:43 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T22:14:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:43 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:14:43.819 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:14:43 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z[136972]: 2023-10-06T22:14:43.401+0000 7f24915d7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:14:43.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:43 smithi190 ceph-mon[134891]: Standby manager daemon z restarted 2023-10-06T22:14:43.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:43 smithi190 ceph-mon[134891]: Standby manager daemon z started 2023-10-06T22:14:43.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:43 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T22:14:43.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:43 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:14:43.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:43 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T22:14:43.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:43 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:14:43.834 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:43.647+0000 7fde09419200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T22:14:43.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-mon[98492]: Standby manager daemon z restarted 2023-10-06T22:14:43.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-mon[98492]: Standby manager daemon z started 2023-10-06T22:14:43.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T22:14:43.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:14:43.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T22:14:43.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.190:0/663462694' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:14:44.093 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:43.831+0000 7fde09419200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T22:14:44.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:44.148+0000 7f524b7fe700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 msgr2=0x7f52380a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/409950778 2023-10-06T22:14:44.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:44.148+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T22:14:44.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:44.091+0000 7fde09419200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T22:14:44.423 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:44.244+0000 7fde09419200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T22:14:44.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:44.496+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 8 ==== mgrmap(e 21) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f523c0cbee0 con 0x7f524c073910 2023-10-06T22:14:44.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:44 smithi156 ceph-mon[134358]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-10-06T22:14:44.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:44 smithi156 ceph-mon[134358]: Standby manager daemon y restarted 2023-10-06T22:14:44.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:44 smithi156 ceph-mon[134358]: Standby manager daemon y started 2023-10-06T22:14:44.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:44 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T22:14:44.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:44 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:14:44.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:44 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T22:14:44.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:44 smithi156 ceph-mon[134358]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:14:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:44 smithi190 ceph-mon[134891]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-10-06T22:14:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:44 smithi190 ceph-mon[134891]: Standby manager daemon y restarted 2023-10-06T22:14:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:44 smithi190 ceph-mon[134891]: Standby manager daemon y started 2023-10-06T22:14:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:44 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T22:14:44.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:44 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:14:44.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:44 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T22:14:44.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:44 smithi190 ceph-mon[134891]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:14:44.923 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:44.536+0000 7fde09419200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T22:14:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-mon[98492]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-10-06T22:14:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-mon[98492]: Standby manager daemon y restarted 2023-10-06T22:14:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-mon[98492]: Standby manager daemon y started 2023-10-06T22:14:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T22:14:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T22:14:44.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T22:14:44.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:44 smithi057 ceph-mon[98492]: from='mgr.? 172.21.15.156:0/3523169854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T22:14:45.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:45.678+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 9 ==== mgrmap(e 22) v1 ==== 135688+0+0 (secure 0 0 0) 0x7f523c0228e0 con 0x7f524c073910 2023-10-06T22:14:45.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:45.678+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 msgr2=0x7f52380a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:14:45.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:45.678+0000 7f52497fa700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/409950778,v1:172.21.15.57:6801/409950778] conn(0x7f52380a14d0 0x7f52380a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:45 smithi156 ceph-mon[134358]: mgrmap e21: x(active, since 8m), standbys: y, z 2023-10-06T22:14:45.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:45 smithi190 ceph-mon[134891]: mgrmap e21: x(active, since 8m), standbys: y, z 2023-10-06T22:14:45.861 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.658+0000 7fde09419200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T22:14:45.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-mon[98492]: mgrmap e21: x(active, since 8m), standbys: y, z 2023-10-06T22:14:46.114 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd6c4fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.114 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd6c4fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd6c4fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd6c4fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd6c4fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd68cf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd68cf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd68cf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd68cf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.115 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:14:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:14:45.893+0000 7fdd68cf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:14:46.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.708+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mon.2 v2:172.21.15.156:3300/0 10 ==== mgrmap(e 23) v1 ==== 135815+0+0 (secure 0 0 0) 0x7f523c0986f0 con 0x7f524c073910 2023-10-06T22:14:46.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.709+0000 7f52497fa700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5238066390 0x7f5238066770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:46.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.709+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 --> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f524c0612f0 con 0x7f5238066390 2023-10-06T22:14:46.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.712+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5238066390 0x7f5238066770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:46.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.716+0000 7f524b7fe700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5238066390 0x7f5238066770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f524c074c10 tx=0x7f5234009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: Active manager daemon x restarted 2023-10-06T22:14:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: Activating manager daemon x 2023-10-06T22:14:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T22:14:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: mgrmap e22: x(active, starting, since 0.0133229s), standbys: y, z 2023-10-06T22:14:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:14:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:14:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T22:14:46.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: Manager daemon x is now available 2023-10-06T22:14:46.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:46.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:46.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T22:14:46.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T22:14:46.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.783+0000 7f52497fa700 1 -- 172.21.15.57:0/3508431723 <== mgr.25414 v2:172.21.15.57:6800/3210017833 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f524c0612f0 con 0x7f5238066390 2023-10-06T22:14:46.786 INFO:teuthology.orchestra.run.smithi057.stdout:Schedule set for path / 2023-10-06T22:14:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5238066390 msgr2=0x7f5238066770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5238066390 0x7f5238066770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f524c074c10 tx=0x7f5234009260 comp rx=0 tx=0).stop 2023-10-06T22:14:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c073910 msgr2=0x7f524c198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:46.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 --2- 172.21.15.57:0/3508431723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f524c073910 0x7f524c198f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f523c00b680 tx=0x7f523c005740 comp rx=0 tx=0).stop 2023-10-06T22:14:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f524bfff700 1 -- 172.21.15.57:0/3508431723 reap_dead start 2023-10-06T22:14:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 shutdown_connections 2023-10-06T22:14:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 >> 172.21.15.57:0/3508431723 conn(0x7f524c0fc430 msgr2=0x7f524c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 shutdown_connections 2023-10-06T22:14:46.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:46.786+0000 7f5251fff700 1 -- 172.21.15.57:0/3508431723 wait complete. 2023-10-06T22:14:46.805 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: Active manager daemon x restarted 2023-10-06T22:14:46.805 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: Activating manager daemon x 2023-10-06T22:14:46.805 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T22:14:46.806 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: mgrmap e22: x(active, starting, since 0.0133229s), standbys: y, z 2023-10-06T22:14:46.806 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:14:46.806 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:14:46.806 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:14:46.806 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T22:14:46.806 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T22:14:46.806 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:14:46.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:14:46.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:14:46.809 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T22:14:46.810 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T22:14:46.810 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T22:14:46.810 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: Manager daemon x is now available 2023-10-06T22:14:46.810 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:46.810 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:46.810 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T22:14:46.810 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T22:14:46.812 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-10-06T22:14:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: Active manager daemon x restarted 2023-10-06T22:14:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: Activating manager daemon x 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: mgrmap e22: x(active, starting, since 0.0133229s), standbys: y, z 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T22:14:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:14:46.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T22:14:46.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T22:14:46.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T22:14:46.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: Manager daemon x is now available 2023-10-06T22:14:46.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:46.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:46.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T22:14:46.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.022+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/3870908850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 msgr2=0x7f8f38106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.022+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/3870908850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 0x7f8f38106690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009990 tx=0x7f8f28009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.022+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/3870908850 shutdown_connections 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.022+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/3870908850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38106bd0 0x7f8f38109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.022+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/3870908850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f38101970 0x7f8f38103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.022+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/3870908850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 0x7f8f38106690 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.022+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/3870908850 >> 172.21.15.57:0/3870908850 conn(0x7f8f380fb340 msgr2=0x7f8f380fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:47.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.023+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/3870908850 shutdown_connections 2023-10-06T22:14:47.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.023+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/3870908850 wait complete. 2023-10-06T22:14:47.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.023+0000 7f8f3f9ff700 1 Processor -- start 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3f9ff700 1 -- start start 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3f9ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38101970 0x7f8f380ff920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3f9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 0x7f8f380ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3f9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f38106bd0 0x7f8f381003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3f9ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381a0c30 con 0x7f8f38101970 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3f9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381a0d70 con 0x7f8f38106bd0 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3f9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381a0eb0 con 0x7f8f381042a0 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3d79b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38101970 0x7f8f380ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3d79b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38101970 0x7f8f380ff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54566/0 (socket says 172.21.15.57:54566) 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3cf9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 0x7f8f380ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3d79b700 1 -- 172.21.15.57:0/4070791674 learned_addr learned my addr 172.21.15.57:0/4070791674 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.024+0000 7f8f3df9c700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f38106bd0 0x7f8f381003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3d79b700 1 -- 172.21.15.57:0/4070791674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 msgr2=0x7f8f380ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3d79b700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 0x7f8f380ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3d79b700 1 -- 172.21.15.57:0/4070791674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f38106bd0 msgr2=0x7f8f381003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3d79b700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f38106bd0 0x7f8f381003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3d79b700 1 -- 172.21.15.57:0/4070791674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280095f0 con 0x7f8f38101970 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3d79b700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38101970 0x7f8f380ff920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400e9a0 tx=0x7f8f3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f2e7fc700 1 -- 172.21.15.57:0/4070791674 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34009790 con 0x7f8f38101970 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f2e7fc700 1 -- 172.21.15.57:0/4070791674 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f34010430 con 0x7f8f38101970 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f2e7fc700 1 -- 172.21.15.57:0/4070791674 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34018680 con 0x7f8f38101970 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f381a1140 con 0x7f8f38101970 2023-10-06T22:14:47.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.026+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f381a1600 con 0x7f8f38101970 2023-10-06T22:14:47.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.027+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f380fcec0 con 0x7f8f38101970 2023-10-06T22:14:47.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.028+0000 7f8f2e7fc700 1 -- 172.21.15.57:0/4070791674 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 23) v1 ==== 135815+0+0 (secure 0 0 0) 0x7f8f340105a0 con 0x7f8f38101970 2023-10-06T22:14:47.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.031+0000 7f8f2e7fc700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8f240a1280 0x7f8f240a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.031+0000 7f8f2e7fc700 1 -- 172.21.15.57:0/4070791674 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f8f34014070 con 0x7f8f38101970 2023-10-06T22:14:47.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.032+0000 7f8f3cf9a700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8f240a1280 0x7f8f240a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.033+0000 7f8f3cf9a700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8f240a1280 0x7f8f240a3740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b2e0 tx=0x7f8f28005fb0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:47.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.036+0000 7f8f2e7fc700 1 -- 172.21.15.57:0/4070791674 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f34052100 con 0x7f8f38101970 2023-10-06T22:14:47.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.281+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 --> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f3802d0b0 con 0x7f8f240a1280 2023-10-06T22:14:47.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.343+0000 7f8f2e7fc700 1 -- 172.21.15.57:0/4070791674 <== mgr.25414 v2:172.21.15.57:6800/3210017833 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f8f3802d0b0 con 0x7f8f240a1280 2023-10-06T22:14:47.345 INFO:teuthology.orchestra.run.smithi057.stdout:Retention added to path / 2023-10-06T22:14:47.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8f240a1280 msgr2=0x7f8f240a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8f240a1280 0x7f8f240a3740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b2e0 tx=0x7f8f28005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38101970 msgr2=0x7f8f380ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38101970 0x7f8f380ff920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400e9a0 tx=0x7f8f3400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 shutdown_connections 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8f240a1280 0x7f8f240a3740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f38106bd0 0x7f8f381003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f38101970 0x7f8f380ff920 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 --2- 172.21.15.57:0/4070791674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f381042a0 0x7f8f380ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 >> 172.21.15.57:0/4070791674 conn(0x7f8f380fb340 msgr2=0x7f8f381025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 shutdown_connections 2023-10-06T22:14:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.345+0000 7f8f3f9ff700 1 -- 172.21.15.57:0/4070791674 wait complete. 2023-10-06T22:14:47.365 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-06T22:14:47.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.575+0000 7fb5301ff700 1 -- 172.21.15.57:0/1284994872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528102060 msgr2=0x7fb5281024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.575+0000 7fb5301ff700 1 --2- 172.21.15.57:0/1284994872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528102060 0x7fb5281024e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb518009990 tx=0x7fb518009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.576+0000 7fb5301ff700 1 -- 172.21.15.57:0/1284994872 shutdown_connections 2023-10-06T22:14:47.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.576+0000 7fb5301ff700 1 --2- 172.21.15.57:0/1284994872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb528102a20 0x7fb528109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.576+0000 7fb5301ff700 1 --2- 172.21.15.57:0/1284994872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb528100f10 0x7fb528101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.576+0000 7fb5301ff700 1 --2- 172.21.15.57:0/1284994872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528102060 0x7fb5281024e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.576+0000 7fb5301ff700 1 -- 172.21.15.57:0/1284994872 >> 172.21.15.57:0/1284994872 conn(0x7fb5280fc460 msgr2=0x7fb5280fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:47.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.576+0000 7fb5301ff700 1 -- 172.21.15.57:0/1284994872 shutdown_connections 2023-10-06T22:14:47.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.576+0000 7fb5301ff700 1 -- 172.21.15.57:0/1284994872 wait complete. 2023-10-06T22:14:47.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.577+0000 7fb5301ff700 1 Processor -- start 2023-10-06T22:14:47.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.577+0000 7fb5301ff700 1 -- start start 2023-10-06T22:14:47.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb5301ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528100f10 0x7fb5280732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb5301ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb528102060 0x7fb5280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb5301ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb528102a20 0x7fb528073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb5301ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281a0cb0 con 0x7fb528102060 2023-10-06T22:14:47.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb5301ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281a0df0 con 0x7fb528100f10 2023-10-06T22:14:47.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb5301ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281a0f30 con 0x7fb528102a20 2023-10-06T22:14:47.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb52df9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528100f10 0x7fb5280732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb52d79a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb528102060 0x7fb5280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb52df9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528100f10 0x7fb5280732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:51468/0 (socket says 172.21.15.57:51468) 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb52df9b700 1 -- 172.21.15.57:0/3218001824 learned_addr learned my addr 172.21.15.57:0/3218001824 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.578+0000 7fb52e79c700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb528102a20 0x7fb528073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb52df9b700 1 -- 172.21.15.57:0/3218001824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb528102a20 msgr2=0x7fb528073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb52df9b700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb528102a20 0x7fb528073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb52df9b700 1 -- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb528102060 msgr2=0x7fb5280737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb52df9b700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb528102060 0x7fb5280737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb52df9b700 1 -- 172.21.15.57:0/3218001824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180095f0 con 0x7fb528100f10 2023-10-06T22:14:47.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb52d79a700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb528102060 0x7fb5280737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:14:47.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb52df9b700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528100f10 0x7fb5280732a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb52400e9a0 tx=0x7fb52400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:47.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524009790 con 0x7fb528100f10 2023-10-06T22:14:47.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5281a11c0 con 0x7fb528100f10 2023-10-06T22:14:47.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb524010430 con 0x7fb528100f10 2023-10-06T22:14:47.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524018680 con 0x7fb528100f10 2023-10-06T22:14:47.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.579+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5281a16d0 con 0x7fb528100f10 2023-10-06T22:14:47.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.580+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52804ec90 con 0x7fb528100f10 2023-10-06T22:14:47.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.582+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 23) v1 ==== 135815+0+0 (secure 0 0 0) 0x7fb5240105a0 con 0x7fb528100f10 2023-10-06T22:14:47.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.584+0000 7fb51effd700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5140a1280 0x7fb5140a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:47.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.584+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb524014070 con 0x7fb528100f10 2023-10-06T22:14:47.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.584+0000 7fb52d79a700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5140a1280 0x7fb5140a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:47.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.585+0000 7fb52d79a700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5140a1280 0x7fb5140a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b2e0 tx=0x7fb518005fb0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:47.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.586+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb5240963e0 con 0x7fb528100f10 2023-10-06T22:14:47.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.718+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 24) v1 ==== 136123+0+0 (secure 0 0 0) 0x7fb524010880 con 0x7fb528100f10 2023-10-06T22:14:47.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.854+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 --> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5280612f0 con 0x7fb5140a1280 2023-10-06T22:14:47.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.862+0000 7fb51effd700 1 -- 172.21.15.57:0/3218001824 <== mgr.25414 v2:172.21.15.57:6800/3210017833 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fb5280612f0 con 0x7fb5140a1280 2023-10-06T22:14:47.864 INFO:teuthology.orchestra.run.smithi057.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T22:14:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-10-06T22:14:47.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5140a1280 msgr2=0x7fb5140a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5140a1280 0x7fb5140a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b2e0 tx=0x7fb518005fb0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528100f10 msgr2=0x7fb5280732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528100f10 0x7fb5280732a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb52400e9a0 tx=0x7fb52400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 shutdown_connections 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5140a1280 0x7fb5140a3740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb528102a20 0x7fb528073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb528100f10 0x7fb5280732a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 --2- 172.21.15.57:0/3218001824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb528102060 0x7fb5280737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 >> 172.21.15.57:0/3218001824 conn(0x7fb5280fc460 msgr2=0x7fb5280fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 shutdown_connections 2023-10-06T22:14:47.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:47.865+0000 7fb5301ff700 1 -- 172.21.15.57:0/3218001824 wait complete. 2023-10-06T22:14:47.887 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-06T22:14:47.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:47 smithi057 ceph-mon[98492]: [06/Oct/2023:22:14:46] ENGINE Bus STARTING 2023-10-06T22:14:47.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:47 smithi057 ceph-mon[98492]: [06/Oct/2023:22:14:46] ENGINE Serving on https://172.21.15.57:7150 2023-10-06T22:14:47.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:47 smithi057 ceph-mon[98492]: [06/Oct/2023:22:14:46] ENGINE Serving on http://172.21.15.57:8765 2023-10-06T22:14:47.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:47 smithi057 ceph-mon[98492]: [06/Oct/2023:22:14:46] ENGINE Bus STARTED 2023-10-06T22:14:47.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:47 smithi057 ceph-mon[98492]: mgrmap e23: x(active, since 1.04747s), standbys: y, z 2023-10-06T22:14:47.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:47 smithi057 ceph-mon[98492]: from='client.25418 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:47.956 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:47 smithi057 ceph-mon[98492]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:48.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:47 smithi156 ceph-mon[134358]: [06/Oct/2023:22:14:46] ENGINE Bus STARTING 2023-10-06T22:14:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:47 smithi156 ceph-mon[134358]: [06/Oct/2023:22:14:46] ENGINE Serving on https://172.21.15.57:7150 2023-10-06T22:14:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:47 smithi156 ceph-mon[134358]: [06/Oct/2023:22:14:46] ENGINE Serving on http://172.21.15.57:8765 2023-10-06T22:14:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:47 smithi156 ceph-mon[134358]: [06/Oct/2023:22:14:46] ENGINE Bus STARTED 2023-10-06T22:14:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:47 smithi156 ceph-mon[134358]: mgrmap e23: x(active, since 1.04747s), standbys: y, z 2023-10-06T22:14:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:47 smithi156 ceph-mon[134358]: from='client.25418 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:48.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:47 smithi156 ceph-mon[134358]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:48.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:47 smithi190 ceph-mon[134891]: [06/Oct/2023:22:14:46] ENGINE Bus STARTING 2023-10-06T22:14:48.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:47 smithi190 ceph-mon[134891]: [06/Oct/2023:22:14:46] ENGINE Serving on https://172.21.15.57:7150 2023-10-06T22:14:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:47 smithi190 ceph-mon[134891]: [06/Oct/2023:22:14:46] ENGINE Serving on http://172.21.15.57:8765 2023-10-06T22:14:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:47 smithi190 ceph-mon[134891]: [06/Oct/2023:22:14:46] ENGINE Bus STARTED 2023-10-06T22:14:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:47 smithi190 ceph-mon[134891]: mgrmap e23: x(active, since 1.04747s), standbys: y, z 2023-10-06T22:14:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:47 smithi190 ceph-mon[134891]: from='client.25418 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:48.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:47 smithi190 ceph-mon[134891]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:48.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 -- 172.21.15.57:0/1096246197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104300 msgr2=0x7fb0f0104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:48.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/1096246197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104300 0x7fb0f0104760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009990 tx=0x7fb0d8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 -- 172.21.15.57:0/1096246197 shutdown_connections 2023-10-06T22:14:48.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/1096246197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f0104ca0 0x7fb0f010b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/1096246197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f0103110 0x7fb0f0103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/1096246197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104300 0x7fb0f0104760 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 -- 172.21.15.57:0/1096246197 >> 172.21.15.57:0/1096246197 conn(0x7fb0f00fe660 msgr2=0x7fb0f0100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:48.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 -- 172.21.15.57:0/1096246197 shutdown_connections 2023-10-06T22:14:48.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.102+0000 7fb0f51ff700 1 -- 172.21.15.57:0/1096246197 wait complete. 2023-10-06T22:14:48.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.103+0000 7fb0f51ff700 1 Processor -- start 2023-10-06T22:14:48.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0f51ff700 1 -- start start 2023-10-06T22:14:48.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0f51ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f0103110 0x7fb0f0198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:48.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0f51ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f0104300 0x7fb0f0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:48.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0f51ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104ca0 0x7fb0f019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:48.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0f51ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f019a900 con 0x7fb0f0103110 2023-10-06T22:14:48.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0f51ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f019aa70 con 0x7fb0f0104300 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0f51ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f019ac30 con 0x7fb0f0104ca0 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0eed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f0103110 0x7fb0f0198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0ef59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104ca0 0x7fb0f019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0ef59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104ca0 0x7fb0f019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:54696/0 (socket says 172.21.15.57:54696) 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0ef59e700 1 -- 172.21.15.57:0/4237656420 learned_addr learned my addr 172.21.15.57:0/4237656420 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.104+0000 7fb0e7fff700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f0104300 0x7fb0f0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ef59e700 1 -- 172.21.15.57:0/4237656420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f0104300 msgr2=0x7fb0f0199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ef59e700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f0104300 0x7fb0f0199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ef59e700 1 -- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f0103110 msgr2=0x7fb0f0198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:48.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ef59e700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f0103110 0x7fb0f0198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ef59e700 1 -- 172.21.15.57:0/4237656420 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d80095f0 con 0x7fb0f0104ca0 2023-10-06T22:14:48.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ef59e700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104ca0 0x7fb0f019a2a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800ef30 tx=0x7fb0e800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:48.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ecd99700 1 -- 172.21.15.57:0/4237656420 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8018070 con 0x7fb0f0104ca0 2023-10-06T22:14:48.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ecd99700 1 -- 172.21.15.57:0/4237656420 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e800f040 con 0x7fb0f0104ca0 2023-10-06T22:14:48.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0ecd99700 1 -- 172.21.15.57:0/4237656420 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800cd00 con 0x7fb0f0104ca0 2023-10-06T22:14:48.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f019aeb0 con 0x7fb0f0104ca0 2023-10-06T22:14:48.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.105+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f01a16d0 con 0x7fb0f0104ca0 2023-10-06T22:14:48.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.106+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f004ec90 con 0x7fb0f0104ca0 2023-10-06T22:14:48.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.108+0000 7fb0ecd99700 1 -- 172.21.15.57:0/4237656420 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 24) v1 ==== 136123+0+0 (secure 0 0 0) 0x7fb0e8003a00 con 0x7fb0f0104ca0 2023-10-06T22:14:48.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.110+0000 7fb0ecd99700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:14:48.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.110+0000 7fb0ecd99700 1 -- 172.21.15.57:0/4237656420 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb0e80ccd10 con 0x7fb0f0104ca0 2023-10-06T22:14:48.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.110+0000 7fb0eed9d700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:14:48.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.111+0000 7fb0eed9d700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0104160 tx=0x7fb0e0009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:14:48.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.112+0000 7fb0ecd99700 1 -- 172.21.15.57:0/4237656420 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb0e80960d0 con 0x7fb0f0104ca0 2023-10-06T22:14:48.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.366+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 --> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb0f00612f0 con 0x7fb0dc0a1670 2023-10-06T22:14:48.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.374+0000 7fb0ecd99700 1 -- 172.21.15.57:0/4237656420 <== mgr.25414 v2:172.21.15.57:6800/3210017833 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fb0f00612f0 con 0x7fb0dc0a1670 2023-10-06T22:14:48.376 INFO:teuthology.orchestra.run.smithi057.stdout:/ 1m 6m3h 2023-10-06T22:14:48.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb0dc0a1670 msgr2=0x7fb0dc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:48.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0104160 tx=0x7fb0e0009260 comp rx=0 tx=0).stop 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104ca0 msgr2=0x7fb0f019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104ca0 0x7fb0f019a2a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800ef30 tx=0x7fb0e800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 shutdown_connections 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb0dc0a1670 0x7fb0dc0a3b30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f0104ca0 0x7fb0f019a2a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f0103110 0x7fb0f0198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 --2- 172.21.15.57:0/4237656420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0f0104300 0x7fb0f0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 >> 172.21.15.57:0/4237656420 conn(0x7fb0f00fe660 msgr2=0x7fb0f0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 shutdown_connections 2023-10-06T22:14:48.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:14:48.376+0000 7fb0f51ff700 1 -- 172.21.15.57:0/4237656420 wait complete. 2023-10-06T22:14:48.399 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-10-06T22:14:48.472 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-10-06T22:14:48.483 INFO:teuthology.run_tasks:Running task background_exec... 2023-10-06T22:14:48.516 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:14:48.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-10-06T22:14:48.521 INFO:teuthology.run_tasks:Running task workunit... 2023-10-06T22:14:48.569 INFO:tasks.workunit:Pulling workunits from ref 14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:14:48.569 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-06T22:14:48.569 INFO:tasks.workunit:timeout=3h 2023-10-06T22:14:48.569 INFO:tasks.workunit:cleanup=True 2023-10-06T22:14:48.569 DEBUG:teuthology.orchestra.run.smithi057:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:48.620 INFO:teuthology.orchestra.run.smithi057.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:48.620 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-06T22:14:48.620 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-10-06T22:14:48.620 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T22:14:48.621 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-06T22:14:48.621 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-06 22:12:50.802997875 +0000 2023-10-06T22:14:48.621 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-06 22:12:50.802997875 +0000 2023-10-06T22:14:48.621 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-06 22:14:26.590565311 +0000 2023-10-06T22:14:48.621 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2023-10-06 22:12:50.802997875 +0000 2023-10-06T22:14:48.621 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-06T22:14:48.621 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-06T22:14:48.704 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T22:14:48.754 INFO:tasks.workunit.client.0.smithi057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-06T22:14:49.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:48 smithi156 ceph-mon[134358]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:48 smithi156 ceph-mon[134358]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:48 smithi156 ceph-mon[134358]: mgrmap e24: x(active, since 2s), standbys: y, z 2023-10-06T22:14:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:48 smithi156 ceph-mon[134358]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:48 smithi190 ceph-mon[134891]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:48 smithi190 ceph-mon[134891]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:48 smithi190 ceph-mon[134891]: mgrmap e24: x(active, since 2s), standbys: y, z 2023-10-06T22:14:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:48 smithi190 ceph-mon[134891]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:48 smithi057 ceph-mon[98492]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:48 smithi057 ceph-mon[98492]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:48 smithi057 ceph-mon[98492]: mgrmap e24: x(active, since 2s), standbys: y, z 2023-10-06T22:14:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:48 smithi057 ceph-mon[98492]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='client.25486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.071 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='client.25486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='client.25486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:50.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:50.255 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-10-06T22:14:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T22:14:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-10-06T22:14:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: Unable to set osd_memory_target on smithi190 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-06T22:14:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:51.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:50 smithi190 ceph-mon[134891]: mgrmap e25: x(active, since 5s), standbys: y, z 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: Unable to set osd_memory_target on smithi190 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:50 smithi057 ceph-mon[98492]: mgrmap e25: x(active, since 5s), standbys: y, z 2023-10-06T22:14:51.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-10-06T22:14:51.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T22:14:51.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:51.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-10-06T22:14:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: Unable to set osd_memory_target on smithi190 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-06T22:14:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:51.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:50 smithi156 ceph-mon[134358]: mgrmap e25: x(active, since 5s), standbys: y, z 2023-10-06T22:14:52.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:52.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:52.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi057 to 222.0M 2023-10-06T22:14:52.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Unable to set osd_memory_target on smithi057 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T22:14:52.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:14:52.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:14:52.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-06T22:14:52.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:14:52.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:14:52.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:52.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:52.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:52.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:52 smithi156 ceph-mon[134358]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:52.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:52.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi057 to 222.0M 2023-10-06T22:14:52.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Unable to set osd_memory_target on smithi057 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T22:14:52.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:14:52.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:14:52.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-06T22:14:52.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:14:52.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:14:52.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:52.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:52.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:52.572 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:52 smithi190 ceph-mon[134891]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi057 to 222.0M 2023-10-06T22:14:52.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Unable to set osd_memory_target on smithi057 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi156:/etc/ceph/ceph.conf 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi190:/etc/ceph/ceph.conf 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:52.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:52 smithi057 ceph-mon[98492]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T22:14:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:53.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:53 smithi190 ceph-mon[134891]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:53 smithi057 ceph-mon[98492]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: Updating smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: Updating smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:53.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:53.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-06T22:14:53.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:53 smithi156 ceph-mon[134358]: Updating smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:54 smithi057 ceph-mon[98492]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 133 KiB/s wr, 72 op/s 2023-10-06T22:14:54.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:14:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:14:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:54 smithi156 ceph-mon[134358]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 133 KiB/s wr, 72 op/s 2023-10-06T22:14:54.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:14:54.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:14:54.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:14:54.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:14:54.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:54 smithi190 ceph-mon[134891]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 133 KiB/s wr, 72 op/s 2023-10-06T22:14:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:56 smithi156 ceph-mon[134358]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 56 op/s 2023-10-06T22:14:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:56 smithi190 ceph-mon[134891]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 56 op/s 2023-10-06T22:14:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:56 smithi057 ceph-mon[98492]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 56 op/s 2023-10-06T22:14:59.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:14:58 smithi156 ceph-mon[134358]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-10-06T22:14:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:14:58 smithi190 ceph-mon[134891]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-10-06T22:14:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:14:58 smithi057 ceph-mon[98492]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-10-06T22:15:00.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:15:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:15:00.357+0000 7fdd4a0fa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:15:00.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:15:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:15:00.357+0000 7fdd4a0fa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:15:00.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:15:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:15:00.357+0000 7fdd4a0fa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:15:00.673 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:15:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:15:00.357+0000 7fdd4a0fa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:15:00.674 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:15:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:15:00.357+0000 7fdd4a0fa700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T22:15:01.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:00 smithi156 ceph-mon[134358]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-10-06T22:15:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:00 smithi190 ceph-mon[134891]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-10-06T22:15:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:00 smithi057 ceph-mon[98492]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-10-06T22:15:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2327530050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4101a80 msgr2=0x7ff4a4105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2327530050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4101a80 0x7ff4a4105ac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2327530050 shutdown_connections 2023-10-06T22:15:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2327530050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41061f0 0x7ff4a410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2327530050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41010d0 0x7ff4a41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2327530050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a4101a80 0x7ff4a4105ac0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2327530050 >> 172.21.15.57:0/2327530050 conn(0x7ff4a40fc930 msgr2=0x7ff4a40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.784+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2327530050 shutdown_connections 2023-10-06T22:15:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.785+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2327530050 wait complete. 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 Processor -- start 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 -- start start 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41010d0 0x7ff4a4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41061f0 0x7ff4a40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a41a31b0 0x7ff4a41a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41a5ae0 con 0x7ff4a41010d0 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41a5c20 con 0x7ff4a41061f0 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.786+0000 7ff4a89ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41a5d90 con 0x7ff4a41a31b0 2023-10-06T22:15:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41010d0 0x7ff4a4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41061f0 0x7ff4a40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41010d0 0x7ff4a4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59724/0 (socket says 172.21.15.57:59724) 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 -- 172.21.15.57:0/2179249158 learned_addr learned my addr 172.21.15.57:0/2179249158 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a41a31b0 0x7ff4a41a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 -- 172.21.15.57:0/2179249158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a41a31b0 msgr2=0x7ff4a41a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a41a31b0 0x7ff4a41a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 -- 172.21.15.57:0/2179249158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41061f0 msgr2=0x7ff4a40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41061f0 0x7ff4a40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 -- 172.21.15.57:0/2179249158 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980095f0 con 0x7ff4a41010d0 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a41a31b0 0x7ff4a41a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41061f0 0x7ff4a40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4a259c700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41010d0 0x7ff4a4073390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e9a0 tx=0x7ff49400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.787+0000 7ff4937fe700 1 -- 172.21.15.57:0/2179249158 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494009790 con 0x7ff4a41010d0 2023-10-06T22:15:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.788+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a41a6070 con 0x7ff4a41010d0 2023-10-06T22:15:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.788+0000 7ff4937fe700 1 -- 172.21.15.57:0/2179249158 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49400cbd0 con 0x7ff4a41010d0 2023-10-06T22:15:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.788+0000 7ff4937fe700 1 -- 172.21.15.57:0/2179249158 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494018680 con 0x7ff4a41010d0 2023-10-06T22:15:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.788+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a41a6580 con 0x7ff4a41010d0 2023-10-06T22:15:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.793+0000 7ff4937fe700 1 -- 172.21.15.57:0/2179249158 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7ff49400cd40 con 0x7ff4a41010d0 2023-10-06T22:15:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 Processor -- start 2023-10-06T22:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 -- start start 2023-10-06T22:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf6e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf64a400 con 0x5641bf6e0c00 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf63bbc0 con 0x5641bf6ec3d0 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf6394e0 con 0x5641bf6e1520 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.794+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf6e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf6e0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59740/0 (socket says 172.21.15.57:59740) 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1021614727 learned_addr learned my addr 172.21.15.57:0/1021614727 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a259c700 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1021614727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 msgr2=0x5641bf6ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6ebe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1021614727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 msgr2=0x5641bf6ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6ee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1021614727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641bf45fd80 con 0x5641bf6e0c00 2023-10-06T22:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4937fe700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff48c0a1710 0x7ff48c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4937fe700 1 -- 172.21.15.57:0/2179249158 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7ff494014070 con 0x7ff4a41010d0 2023-10-06T22:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf6e0fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009dd0 tx=0x7ff49c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9173dc5e556f848 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.795+0000 7ff4a259c700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff48c0a1710 0x7ff48c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff490ff9700 1 -- 172.21.15.57:0/1021614727 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c014610 con 0x5641bf6e0c00 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff490ff9700 1 -- 172.21.15.57:0/1021614727 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c00f050 con 0x5641bf6e0c00 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff490ff9700 1 -- 172.21.15.57:0/1021614727 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0139d0 con 0x5641bf6e0c00 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1021614727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 msgr2=0x5641bf6e0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4a259c700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff48c0a1710 0x7ff48c0a3bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cac90 tx=0x7ff494096040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf6e0fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009dd0 tx=0x7ff49c0077e0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1021614727 shutdown_connections 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf6e0fe0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1021614727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.796+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1021614727 >> 172.21.15.57:0/1021614727 conn(0x5641bf656200 msgr2=0x5641bf6dd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.797+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1021614727 shutdown_connections 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.797+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1021614727 wait complete. 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.797+0000 7ff4bc351b80 1 -- start start 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.797+0000 7ff4bc351b80 1 Processor -- start 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e0c00 0x5641bf6ebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e1520 0x5641bf6e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf634dd0 con 0x5641bf6e1520 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf5d16b0 con 0x5641bf6ec3d0 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf5d1350 con 0x5641bf6e0c00 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4bc351b80 10 client.0 _ll_register_callbacks cb 0x5641bf42e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4a1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e1520 0x5641bf6e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4a1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e1520 0x5641bf6e9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59748/0 (socket says 172.21.15.57:59748) 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4a1d9b700 1 -- 172.21.15.57:0/2953974142 learned_addr learned my addr 172.21.15.57:0/2953974142 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e0c00 0x5641bf6ebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.798+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4a1d9b700 1 -- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e0c00 msgr2=0x5641bf6ebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e0c00 0x5641bf6ebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4a1d9b700 1 -- 172.21.15.57:0/2953974142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 msgr2=0x5641bf6e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4a1d9b700 1 -- 172.21.15.57:0/2953974142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c009a00 con 0x5641bf6e1520 2023-10-06T22:15:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e1520 0x5641bf6e9070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8163c12772d8e74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff489ffb700 1 -- 172.21.15.57:0/2953974142 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49801d070 con 0x5641bf6e1520 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff489ffb700 1 -- 172.21.15.57:0/2953974142 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7ff498004910 con 0x5641bf6e1520 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641bf6d4d50 con 0x5641bf6e1520 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff489ffb700 1 -- 172.21.15.57:0/2953974142 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff498004ad0 con 0x5641bf6e1520 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4bc351b80 10 client.15489 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.799+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5641bf448c00 con 0x5641bf6e1520 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff489ffb700 1 -- 172.21.15.57:0/2953974142 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff498034020 con 0x5641bf6e1520 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff489ffb700 1 -- 172.21.15.57:0/2953974142 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff49800f460 con 0x5641bf6e1520 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff4bc351b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff4bc351b80 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff4bc351b80 20 client.15489 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf6fbec0 0x5641bf6fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff4bc351b80 4 client.15489 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641bf62ddb0 con 0x5641bf6fbec0 2023-10-06T22:15:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.800+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf6fbec0 0x5641bf6fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.801+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf6fbec0 0x5641bf6fe2e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9781fb894f76044 server_cookie=60b3c39cf1114c2c in_seq=0 out_seq=0 2023-10-06T22:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.801+0000 7ff489ffb700 10 client.15489 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.801+0000 7ff489ffb700 1 -- 172.21.15.57:0/2953974142 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641bf62ddb0 con 0x5641bf6fbec0 2023-10-06T22:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.801+0000 7ff489ffb700 10 client.15489 handle_command_reply: tid=0 2023-10-06T22:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 client.15489 shutdown 2023-10-06T22:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e1520 msgr2=0x5641bf6e9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e1520 0x5641bf6e9070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498005320 comp rx=0 tx=0).stop 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 shutdown_connections 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf6fbec0 0x5641bf6fe2e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e0c00 0x5641bf6ebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff489ffb700 0 client.15489 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/2953974142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e1520 0x5641bf6e9070 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 >> 172.21.15.57:0/2953974142 conn(0x5641bf656200 msgr2=0x5641bf6dcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 shutdown_connections 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 1 -- 172.21.15.57:0/2953974142 wait complete. 2023-10-06T22:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.802+0000 7ff4bc351b80 20 client.15489 trim_cache size 0 max 16384 2023-10-06T22:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.817+0000 7ff4bc351b80 1 Processor -- start 2023-10-06T22:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.817+0000 7ff4bc351b80 1 -- start start 2023-10-06T22:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6e0c00 0x5641bf6f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf4b58c0 con 0x5641bf6ec3d0 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf6d72c0 con 0x5641bf6e0c00 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf6f3460 con 0x5641bf6e1520 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59758/0 (socket says 172.21.15.57:59758) 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1352329472 learned_addr learned my addr 172.21.15.57:0/1352329472 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6e0c00 0x5641bf6f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.818+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1352329472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 msgr2=0x5641bf6e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6e6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1352329472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6e0c00 msgr2=0x5641bf6f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6e0c00 0x5641bf6f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/1352329472 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641bf4861f0 con 0x5641bf6ec3d0 2023-10-06T22:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e6860 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff49c018000 tx=0x7ff49c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe7dd289f36033 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4917fa700 1 -- 172.21.15.57:0/1352329472 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0293e0 con 0x5641bf6ec3d0 2023-10-06T22:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4917fa700 1 -- 172.21.15.57:0/1352329472 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c0060a0 con 0x5641bf6ec3d0 2023-10-06T22:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.819+0000 7ff4917fa700 1 -- 172.21.15.57:0/1352329472 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c029e60 con 0x5641bf6ec3d0 2023-10-06T22:15:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1352329472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6ec3d0 msgr2=0x5641bf6e6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e6860 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff49c018000 tx=0x7ff49c000f40 comp rx=0 tx=0).stop 2023-10-06T22:15:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1352329472 shutdown_connections 2023-10-06T22:15:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6ec3d0 0x5641bf6e6860 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6e0c00 0x5641bf6f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/1352329472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf6e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1352329472 >> 172.21.15.57:0/1352329472 conn(0x5641bf656200 msgr2=0x5641bf6e43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1352329472 shutdown_connections 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 -- 172.21.15.57:0/1352329472 wait complete. 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 -- start start 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.820+0000 7ff4bc351b80 1 Processor -- start 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf747f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf7484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4bc351b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf748d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf749330 con 0x5641bf6e0c00 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf7494a0 con 0x5641bf6ec3d0 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4bc351b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf749660 con 0x5641bf6e1520 2023-10-06T22:15:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4bc351b80 10 client.0 _ll_register_callbacks cb 0x5641bf42e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf747f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf747f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59772/0 (socket says 172.21.15.57:59772) 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/3396309769 learned_addr learned my addr 172.21.15.57:0/3396309769 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf7484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.821+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf748d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 msgr2=0x5641bf7484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf7484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/3396309769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 msgr2=0x5641bf748d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf748d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4a2d9d700 1 -- 172.21.15.57:0/3396309769 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641bf448c00 con 0x5641bf6e0c00 2023-10-06T22:15:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4a2d9d700 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf747f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009360 tx=0x7ff49c007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaa13f02560a5c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff489ffb700 1 -- 172.21.15.57:0/3396309769 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0291e0 con 0x5641bf6e0c00 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff49c00f050 con 0x5641bf6e0c00 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff489ffb700 1 -- 172.21.15.57:0/3396309769 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7ff49c029340 con 0x5641bf6e0c00 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff489ffb700 1 -- 172.21.15.57:0/3396309769 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49c029500 con 0x5641bf6e0c00 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4bc351b80 10 client.15495 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.822+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff49c031040 con 0x5641bf6e0c00 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff489ffb700 1 -- 172.21.15.57:0/3396309769 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c00f050 con 0x5641bf6e0c00 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff489ffb700 1 -- 172.21.15.57:0/3396309769 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff49c0146e0 con 0x5641bf6e0c00 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff4bc351b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff4bc351b80 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff4bc351b80 20 client.15495 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf750ae0 0x5641bf752ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff4bc351b80 4 client.15495 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:15:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5641bf62dad0 con 0x5641bf750ae0 2023-10-06T22:15:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.823+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf750ae0 0x5641bf752ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.824+0000 7ff4a1d9b700 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf750ae0 0x5641bf752ed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d3b93930fbd9330 server_cookie=3f8a3c141ab49fe7 in_seq=0 out_seq=0 2023-10-06T22:15:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:01.824+0000 7ff489ffb700 10 client.15495 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:02 smithi190 ceph-mon[134891]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-10-06T22:15:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:02 smithi190 ceph-mon[134891]: mgrmap e26: x(active, since 16s), standbys: y, z 2023-10-06T22:15:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:02 smithi057 ceph-mon[98492]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-10-06T22:15:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:02 smithi057 ceph-mon[98492]: mgrmap e26: x(active, since 16s), standbys: y, z 2023-10-06T22:15:03.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:02 smithi156 ceph-mon[134358]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-10-06T22:15:03.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:02 smithi156 ceph-mon[134358]: mgrmap e26: x(active, since 16s), standbys: y, z 2023-10-06T22:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.661+0000 7ff489ffb700 1 -- 172.21.15.57:0/3396309769 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5641bf62dad0 con 0x5641bf750ae0 2023-10-06T22:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.661+0000 7ff489ffb700 10 client.15495 handle_command_reply: tid=0 2023-10-06T22:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.661+0000 7ff4bc351b80 1 client.15495 shutdown 2023-10-06T22:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 msgr2=0x5641bf747f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf747f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009360 tx=0x7ff49c007b30 comp rx=0 tx=0).stop 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 shutdown_connections 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5641bf750ae0 0x5641bf752ed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641bf6ec3d0 0x5641bf748d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641bf6e0c00 0x5641bf747f70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 --2- 172.21.15.57:0/3396309769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641bf6e1520 0x5641bf7484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 >> 172.21.15.57:0/3396309769 conn(0x5641bf656200 msgr2=0x5641bf6ea980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff489ffb700 0 client.15495 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 shutdown_connections 2023-10-06T22:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 1 -- 172.21.15.57:0/3396309769 wait complete. 2023-10-06T22:15:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.662+0000 7ff4bc351b80 20 client.15495 trim_cache size 0 max 16384 2023-10-06T22:15:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:15:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:15:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:15:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:15:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff48c0a1710 msgr2=0x7ff48c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff48c0a1710 0x7ff48c0a3bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cac90 tx=0x7ff494096040 comp rx=0 tx=0).stop 2023-10-06T22:15:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41010d0 msgr2=0x7ff4a4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41010d0 0x7ff4a4073390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e9a0 tx=0x7ff49400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 shutdown_connections 2023-10-06T22:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff48c0a1710 0x7ff48c0a3bd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4a41a31b0 0x7ff4a41a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4a41010d0 0x7ff4a4073390 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 --2- 172.21.15.57:0/2179249158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41061f0 0x7ff4a40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.663+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 >> 172.21.15.57:0/2179249158 conn(0x7ff4a40fc930 msgr2=0x7ff4a40fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.664+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 shutdown_connections 2023-10-06T22:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:04.664+0000 7ff4a89ff700 1 -- 172.21.15.57:0/2179249158 wait complete. 2023-10-06T22:15:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:04 smithi190 ceph-mon[134891]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 62 op/s 2023-10-06T22:15:05.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:04 smithi057 ceph-mon[98492]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 62 op/s 2023-10-06T22:15:05.245 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-mon[134358]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 99 KiB/s wr, 62 op/s 2023-10-06T22:15:05.245 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: *** Caught signal (Segmentation fault) ** 2023-10-06T22:15:05.245 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: in thread 7f0287fc9700 thread_name:md_log_replay 2023-10-06T22:15:05.245 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:15:05.245 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0298b78cf0] 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2: /usr/bin/ceph-mds(+0x4f0102) [0x5574af2cc102] 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5574af2ce353] 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 4: (MDLog::_replay_thread()+0x647) [0x5574af329bf7] 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5574af333141] 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0298b6e1ca] 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 7: clone() 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2023-10-06T22:15:04.864+0000 7f0287fc9700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: in thread 7f0287fc9700 thread_name:md_log_replay 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:15:05.246 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0298b78cf0] 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2: /usr/bin/ceph-mds(+0x4f0102) [0x5574af2cc102] 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5574af2ce353] 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 4: (MDLog::_replay_thread()+0x647) [0x5574af329bf7] 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5574af333141] 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0298b6e1ca] 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 7: clone() 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 0> 2023-10-06T22:15:04.864+0000 7f0287fc9700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: in thread 7f0287fc9700 thread_name:md_log_replay 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2023-10-06T22:15:05.247 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0298b78cf0] 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2: /usr/bin/ceph-mds(+0x4f0102) [0x5574af2cc102] 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5574af2ce353] 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 4: (MDLog::_replay_thread()+0x647) [0x5574af329bf7] 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5574af333141] 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0298b6e1ca] 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 7: clone() 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 0> 2023-10-06T22:15:04.864+0000 7f0287fc9700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: in thread 7f0287fc9700 thread_name:md_log_replay 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2023-10-06T22:15:05.248 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0298b78cf0] 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2: /usr/bin/ceph-mds(+0x4f0102) [0x5574af2cc102] 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5574af2ce353] 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 4: (MDLog::_replay_thread()+0x647) [0x5574af329bf7] 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5574af333141] 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0298b6e1ca] 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 7: clone() 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[171247]: 2023-10-06T22:15:05.249 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:05 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Main process exited, code=exited, status=139/n/a 2023-10-06T22:15:05.504 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:05 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Failed with result 'exit-code'. 2023-10-06T22:15:06.114 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 -- 172.21.15.57:0/2106162357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d10100700 msgr2=0x7f9d10104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2106162357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d10100700 0x7f9d10104bc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009990 tx=0x7f9d00009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 -- 172.21.15.57:0/2106162357 shutdown_connections 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2106162357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d101051e0 0x7f9d10105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2106162357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d100ffde0 0x7f9d101001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2106162357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d10100700 0x7f9d10104bc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 -- 172.21.15.57:0/2106162357 >> 172.21.15.57:0/2106162357 conn(0x7f9d100fb830 msgr2=0x7f9d100fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.330+0000 7f9d179ff700 1 -- 172.21.15.57:0/2106162357 shutdown_connections 2023-10-06T22:15:06.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.331+0000 7f9d179ff700 1 -- 172.21.15.57:0/2106162357 wait complete. 2023-10-06T22:15:06.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.332+0000 7f9d179ff700 1 Processor -- start 2023-10-06T22:15:06.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.332+0000 7f9d179ff700 1 -- start start 2023-10-06T22:15:06.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d179ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d100ffde0 0x7f9d1019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d179ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d10100700 0x7f9d1019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d179ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d101051e0 0x7f9d101a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d179ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101a25e0 con 0x7f9d101051e0 2023-10-06T22:15:06.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d179ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101a2750 con 0x7f9d10100700 2023-10-06T22:15:06.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d179ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101a2910 con 0x7f9d100ffde0 2023-10-06T22:15:06.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d101051e0 0x7f9d101a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d101051e0 0x7f9d101a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37198/0 (socket says 172.21.15.57:37198) 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 -- 172.21.15.57:0/2242782650 learned_addr learned my addr 172.21.15.57:0/2242782650 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d1579b700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d100ffde0 0x7f9d1019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d10100700 0x7f9d1019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 -- 172.21.15.57:0/2242782650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d100ffde0 msgr2=0x7f9d1019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d100ffde0 0x7f9d1019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 -- 172.21.15.57:0/2242782650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d10100700 msgr2=0x7f9d1019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d10100700 0x7f9d1019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.333+0000 7f9d15f9c700 1 -- 172.21.15.57:0/2242782650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000095f0 con 0x7f9d101051e0 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.334+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d101051e0 0x7f9d101a1fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800ef30 tx=0x7f9d0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.334+0000 7f9d067fc700 1 -- 172.21.15.57:0/2242782650 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08018070 con 0x7f9d101051e0 2023-10-06T22:15:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.334+0000 7f9d067fc700 1 -- 172.21.15.57:0/2242782650 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d0800f040 con 0x7f9d101051e0 2023-10-06T22:15:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.334+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d101a2b90 con 0x7f9d101051e0 2023-10-06T22:15:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.334+0000 7f9d067fc700 1 -- 172.21.15.57:0/2242782650 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800cd00 con 0x7f9d101051e0 2023-10-06T22:15:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.334+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d101aa430 con 0x7f9d101051e0 2023-10-06T22:15:06.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.337+0000 7f9d067fc700 1 -- 172.21.15.57:0/2242782650 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f9d08003a00 con 0x7f9d101051e0 2023-10-06T22:15:06.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.339+0000 7f9d2b3c8b80 1 Processor -- start 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.339+0000 7f9d2b3c8b80 1 -- start start 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d067fc700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9cfc0a1790 0x7f9cfc0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de4a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a9e0 0x55e81de55350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de57c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ddb3460 con 0x55e81de4a0c0 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81dda4c20 con 0x55e81de55890 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81dda2540 con 0x55e81de4a9e0 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d067fc700 1 -- 172.21.15.57:0/2242782650 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9d080cce30 con 0x7f9d101051e0 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9cfc0a1790 0x7f9cfc0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d1579b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de4a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d14f9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a9e0 0x55e81de55350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d1579b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de4a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37200/0 (socket says 172.21.15.57:37200) 2023-10-06T22:15:06.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d1579b700 1 -- 172.21.15.57:0/1287822280 learned_addr learned my addr 172.21.15.57:0/1287822280 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:06.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.340+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de57c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d1579b700 1 -- 172.21.15.57:0/1287822280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a9e0 msgr2=0x55e81de55350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d1579b700 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a9e0 0x55e81de55350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d1579b700 1 -- 172.21.15.57:0/1287822280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 msgr2=0x55e81de57c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d1579b700 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de57c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d1579b700 1 -- 172.21.15.57:0/1287822280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81dbc8de0 con 0x55e81de4a0c0 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de57c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9cfc0a1790 0x7f9cfc0a3c50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d000095a0 tx=0x7f9d00005c80 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9d1579b700 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de4a4a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009d80 tx=0x7f9d0c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe000a56f15158d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9cfb7fe700 1 -- 172.21.15.57:0/1287822280 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004230 con 0x55e81de4a0c0 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9cfb7fe700 1 -- 172.21.15.57:0/1287822280 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d0c004390 con 0x55e81de4a0c0 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.341+0000 7f9cfb7fe700 1 -- 172.21.15.57:0/1287822280 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0197c0 con 0x55e81de4a0c0 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/1287822280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 msgr2=0x55e81de4a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de4a4a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009d80 tx=0x7f9d0c011720 comp rx=0 tx=0).stop 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/1287822280 shutdown_connections 2023-10-06T22:15:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de57c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de4a4a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/1287822280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a9e0 0x55e81de55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/1287822280 >> 172.21.15.57:0/1287822280 conn(0x55e81ddbf260 msgr2=0x55e81de46450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/1287822280 shutdown_connections 2023-10-06T22:15:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.342+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/1287822280 wait complete. 2023-10-06T22:15:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.343+0000 7f9d2b3c8b80 1 -- start start 2023-10-06T22:15:06.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.343+0000 7f9d2b3c8b80 1 Processor -- start 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a9e0 0x55e81de52b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81dd9de30 con 0x55e81de4a0c0 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81dd3a710 con 0x55e81de4a9e0 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81dd3a3b0 con 0x55e81de55890 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d2b3c8b80 10 client.0 _ll_register_callbacks cb 0x55e81db97800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:06.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37204/0 (socket says 172.21.15.57:37204) 2023-10-06T22:15:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 -- 172.21.15.57:0/2214733581 learned_addr learned my addr 172.21.15.57:0/2214733581 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a9e0 0x55e81de52b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 -- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 msgr2=0x55e81de53050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de53050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 -- 172.21.15.57:0/2214733581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a9e0 msgr2=0x55e81de52b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a9e0 0x55e81de52b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.344+0000 7f9d1579b700 1 -- 172.21.15.57:0/2214733581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d101aa430 con 0x55e81de4a0c0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9d1579b700 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de525d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009310 tx=0x7f9d0c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca379ab4e854349 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9cf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/2214733581 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c005ce0 con 0x55e81de4a0c0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e81de3df10 con 0x55e81de4a0c0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/2214733581 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9d0c005040 con 0x55e81de4a0c0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/2214733581 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d0c005200 con 0x55e81de4a0c0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9d2b3c8b80 10 client.15507 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.345+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f9d0c0099e0 con 0x55e81de4a0c0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/2214733581 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e81de3df10 con 0x55e81de4a0c0 2023-10-06T22:15:06.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/2214733581 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9d0c021040 con 0x55e81de4a0c0 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9d2b3c8b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9d2b3c8b80 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9d2b3c8b80 20 client.15507 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81de652f0 0x55e81de67710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9d2b3c8b80 4 client.15507 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e81dd96e10 con 0x55e81de652f0 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.346+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81de652f0 0x55e81de67710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.347+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81de652f0 0x55e81de67710 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1286ef32679d246 server_cookie=ad1d7f54a2b1cab0 in_seq=0 out_seq=0 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.347+0000 7f9cf8ff9700 10 client.15507 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/2214733581 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e81dd96e10 con 0x55e81de652f0 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9cf8ff9700 10 client.15507 handle_command_reply: tid=0 2023-10-06T22:15:06.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 client.15507 shutdown 2023-10-06T22:15:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 msgr2=0x55e81de525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de525d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009310 tx=0x7f9d0c003910 comp rx=0 tx=0).stop 2023-10-06T22:15:06.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 shutdown_connections 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81de652f0 0x55e81de67710 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de53050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a0c0 0x55e81de525d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/2214733581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a9e0 0x55e81de52b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9cf8ff9700 0 client.15507 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 >> 172.21.15.57:0/2214733581 conn(0x55e81ddbf260 msgr2=0x55e81de46040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 shutdown_connections 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/2214733581 wait complete. 2023-10-06T22:15:06.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.348+0000 7f9d2b3c8b80 20 client.15507 trim_cache size 0 max 16384 2023-10-06T22:15:06.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.366+0000 7f9d2b3c8b80 1 Processor -- start 2023-10-06T22:15:06.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.366+0000 7f9d2b3c8b80 1 -- start start 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.366+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a0c0 0x55e81de52e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de51240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de51780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81dc1e920 con 0x55e81de4a9e0 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81de4b020 con 0x55e81de4a0c0 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81de4e140 con 0x55e81de55890 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de51240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de51240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37216/0 (socket says 172.21.15.57:37216) 2023-10-06T22:15:06.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 -- 172.21.15.57:0/3155061960 learned_addr learned my addr 172.21.15.57:0/3155061960 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d1579b700 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a0c0 0x55e81de52e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de51780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 -- 172.21.15.57:0/3155061960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 msgr2=0x55e81de51780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de51780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 -- 172.21.15.57:0/3155061960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a0c0 msgr2=0x55e81de52e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a0c0 0x55e81de52e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d14f9a700 1 -- 172.21.15.57:0/3155061960 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81dbb1c60 con 0x55e81de4a9e0 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de51780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.367+0000 7f9d1579b700 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a0c0 0x55e81de52e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de51240 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009990 tx=0x7f9d0000b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb1ed63e1eca145 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9cfbfff700 1 -- 172.21.15.57:0/3155061960 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00034370 con 0x55e81de4a9e0 2023-10-06T22:15:06.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9cfbfff700 1 -- 172.21.15.57:0/3155061960 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d000344d0 con 0x55e81de4a9e0 2023-10-06T22:15:06.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9cfbfff700 1 -- 172.21.15.57:0/3155061960 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00004cf0 con 0x55e81de4a9e0 2023-10-06T22:15:06.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3155061960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 msgr2=0x55e81de51240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de51240 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009990 tx=0x7f9d0000b280 comp rx=0 tx=0).stop 2023-10-06T22:15:06.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3155061960 shutdown_connections 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de55890 0x55e81de51780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de4a0c0 0x55e81de52e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3155061960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de51240 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.368+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3155061960 >> 172.21.15.57:0/3155061960 conn(0x55e81ddbf260 msgr2=0x55e81de50b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.369+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3155061960 shutdown_connections 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.369+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3155061960 wait complete. 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.369+0000 7f9d2b3c8b80 1 -- start start 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.369+0000 7f9d2b3c8b80 1 Processor -- start 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a0c0 0x55e81de5c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d2b3c8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de5ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81de5b590 con 0x55e81de4a9e0 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81de5b700 con 0x55e81de55890 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d2b3c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81deb91d0 con 0x55e81de4a0c0 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d2b3c8b80 10 client.0 _ll_register_callbacks cb 0x7f9d08009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d14f9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d1579b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a0c0 0x55e81de5c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d14f9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de5a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37222/0 (socket says 172.21.15.57:37222) 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d14f9a700 1 -- 172.21.15.57:0/3153064726 learned_addr learned my addr 172.21.15.57:0/3153064726 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d15f9c700 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de5ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d1579b700 1 -- 172.21.15.57:0/3153064726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 msgr2=0x55e81de5ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d1579b700 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de5ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d1579b700 1 -- 172.21.15.57:0/3153064726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 msgr2=0x55e81de5a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d1579b700 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de5a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d1579b700 1 -- 172.21.15.57:0/3153064726 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0099e0 con 0x55e81de4a0c0 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.370+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9d1579b700 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a0c0 0x55e81de5c520 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c011760 tx=0x7f9d0c009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edfa34a3d366f7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9cf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/3153064726 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c023a70 con 0x55e81de4a0c0 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d0c004390 con 0x55e81de4a0c0 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/3153064726 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9d0c003d50 con 0x55e81de4a0c0 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/3153064726 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d0c023bd0 con 0x55e81de4a0c0 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9d2b3c8b80 10 client.25528 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:06.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.371+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f9d00034730 con 0x55e81de4a0c0 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/3153064726 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d0c004390 con 0x55e81de4a0c0 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/3153064726 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9d0c011790 con 0x55e81de4a0c0 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9d2b3c8b80 10 client.25528 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9d2b3c8b80 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9d2b3c8b80 20 client.25528 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81deba4b0 0x55e81debc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9d2b3c8b80 4 client.25528 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e81dd96b30 con 0x55e81deba4b0 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.372+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81deba4b0 0x55e81debc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d14f9a700 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81deba4b0 0x55e81debc8d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=932d351aad41525 server_cookie=fd2e578be635c84e in_seq=0 out_seq=0 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9cf8ff9700 10 client.25528 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:06.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9cf8ff9700 1 -- 172.21.15.57:0/3153064726 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e81dd96b30 con 0x55e81deba4b0 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9cf8ff9700 10 client.25528 handle_command_reply: tid=0 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 client.25528 shutdown 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a0c0 msgr2=0x55e81de5c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a0c0 0x55e81de5c520 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c011760 tx=0x7f9d0c009750 comp rx=0 tx=0).stop 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 shutdown_connections 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e81deba4b0 0x55e81debc8d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e81de55890 0x55e81de5ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e81de4a0c0 0x55e81de5c520 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9cf8ff9700 0 client.25528 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 --2- 172.21.15.57:0/3153064726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e81de4a9e0 0x55e81de5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 >> 172.21.15.57:0/3153064726 conn(0x55e81ddbf260 msgr2=0x55e81de513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 shutdown_connections 2023-10-06T22:15:06.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.373+0000 7f9d2b3c8b80 1 -- 172.21.15.57:0/3153064726 wait complete. 2023-10-06T22:15:06.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d2b3c8b80 20 client.25528 trim_cache size 0 max 16384 2023-10-06T22:15:06.381 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:06.381 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-06T22:15:06.381 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-06T22:15:06.382 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:06.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9cfc0a1790 msgr2=0x7f9cfc0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9cfc0a1790 0x7f9cfc0a3c50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d000095a0 tx=0x7f9d00005c80 comp rx=0 tx=0).stop 2023-10-06T22:15:06.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d101051e0 msgr2=0x7f9d101a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d101051e0 0x7f9d101a1fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800ef30 tx=0x7f9d0800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 shutdown_connections 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9cfc0a1790 0x7f9cfc0a3c50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d101051e0 0x7f9d101a1fa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d100ffde0 0x7f9d1019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 --2- 172.21.15.57:0/2242782650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d10100700 0x7f9d1019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 >> 172.21.15.57:0/2242782650 conn(0x7f9d100fb830 msgr2=0x7f9d10102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 shutdown_connections 2023-10-06T22:15:06.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.374+0000 7f9d179ff700 1 -- 172.21.15.57:0/2242782650 wait complete. 2023-10-06T22:15:06.394 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T22:15:06.394 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 -- 172.21.15.57:0/4059301068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 msgr2=0x7fe7e4114260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/4059301068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 0x7fe7e4114260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009f90 tx=0x7fe7dc00b330 comp rx=0 tx=0).stop 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 -- 172.21.15.57:0/4059301068 shutdown_connections 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/4059301068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 0x7fe7e4114260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/4059301068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073250 0x7fe7e4073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/4059301068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7e4073b70 0x7fe7e4111930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 -- 172.21.15.57:0/4059301068 >> 172.21.15.57:0/4059301068 conn(0x7fe7e40fc990 msgr2=0x7fe7e40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.597+0000 7fe7e97ff700 1 -- 172.21.15.57:0/4059301068 shutdown_connections 2023-10-06T22:15:06.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.598+0000 7fe7e97ff700 1 -- 172.21.15.57:0/4059301068 wait complete. 2023-10-06T22:15:06.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.598+0000 7fe7e97ff700 1 Processor -- start 2023-10-06T22:15:06.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e97ff700 1 -- start start 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e97ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7e4073250 0x7fe7e419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e97ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073b70 0x7fe7e419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e97ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 0x7fe7e41a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e97ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e41a15b0 con 0x7fe7e4073b70 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e97ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e41a1720 con 0x7fe7e4073250 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e97ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e41a18e0 con 0x7fe7e4111e70 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073b70 0x7fe7e419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073b70 0x7fe7e419dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37230/0 (socket says 172.21.15.57:37230) 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e27fc700 1 -- 172.21.15.57:0/2230596970 learned_addr learned my addr 172.21.15.57:0/2230596970 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e37fe700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 0x7fe7e41a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.599+0000 7fe7e2ffd700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7e4073250 0x7fe7e419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e27fc700 1 -- 172.21.15.57:0/2230596970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 msgr2=0x7fe7e41a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e27fc700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 0x7fe7e41a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e27fc700 1 -- 172.21.15.57:0/2230596970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7e4073250 msgr2=0x7fe7e419d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e27fc700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7e4073250 0x7fe7e419d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e27fc700 1 -- 172.21.15.57:0/2230596970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d40095f0 con 0x7fe7e4073b70 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e27fc700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073b70 0x7fe7e419dc00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800e9a0 tx=0x7fe7d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7cbfff700 1 -- 172.21.15.57:0/2230596970 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8009790 con 0x7fe7e4073b70 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc009c40 con 0x7fe7e4073b70 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7cbfff700 1 -- 172.21.15.57:0/2230596970 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7d8010430 con 0x7fe7e4073b70 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7cbfff700 1 -- 172.21.15.57:0/2230596970 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8018680 con 0x7fe7e4073b70 2023-10-06T22:15:06.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.600+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e41aa340 con 0x7fe7e4073b70 2023-10-06T22:15:06.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.601+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e4104390 con 0x7fe7e4073b70 2023-10-06T22:15:06.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.604+0000 7fe7cbfff700 1 -- 172.21.15.57:0/2230596970 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fe7d80105a0 con 0x7fe7e4073b70 2023-10-06T22:15:06.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.605+0000 7fe7cbfff700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe7cc0a1760 0x7fe7cc0a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:06.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.605+0000 7fe7cbfff700 1 -- 172.21.15.57:0/2230596970 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fe7d8014070 con 0x7fe7e4073b70 2023-10-06T22:15:06.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.605+0000 7fe7e2ffd700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe7cc0a1760 0x7fe7cc0a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:06.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.606+0000 7fe7e2ffd700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe7cc0a1760 0x7fe7cc0a3c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005fd0 tx=0x7fe7d4009310 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:06.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.609+0000 7fe7cbfff700 1 -- 172.21.15.57:0/2230596970 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe7d8052100 con 0x7fe7e4073b70 2023-10-06T22:15:06.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.880+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7e4067cd0 con 0x7fe7e4073b70 2023-10-06T22:15:06.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.881+0000 7fe7cbfff700 1 -- 172.21.15.57:0/2230596970 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fe7d80965e0 con 0x7fe7e4073b70 2023-10-06T22:15:06.883 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:15:06.884 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:15:06.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe7cc0a1760 msgr2=0x7fe7cc0a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe7cc0a1760 0x7fe7cc0a3c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005fd0 tx=0x7fe7d4009310 comp rx=0 tx=0).stop 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073b70 msgr2=0x7fe7e419dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073b70 0x7fe7e419dc00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800e9a0 tx=0x7fe7d800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 shutdown_connections 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe7cc0a1760 0x7fe7cc0a3c20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e4111e70 0x7fe7e41a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7e4073250 0x7fe7e419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 --2- 172.21.15.57:0/2230596970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe7e4073b70 0x7fe7e419dc00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:06.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 >> 172.21.15.57:0/2230596970 conn(0x7fe7e40fc990 msgr2=0x7fe7e40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:06.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 shutdown_connections 2023-10-06T22:15:06.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:06.884+0000 7fe7e97ff700 1 -- 172.21.15.57:0/2230596970 wait complete. 2023-10-06T22:15:06.888 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:15:06.906 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T22:15:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:06 smithi190 ceph-mon[134891]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2023-10-06T22:15:07.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 -- 172.21.15.57:0/58018472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194073b70 msgr2=0x7f4194111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 --2- 172.21.15.57:0/58018472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194073b70 0x7f4194111910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4184009990 tx=0x7f4184009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 -- 172.21.15.57:0/58018472 shutdown_connections 2023-10-06T22:15:07.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 --2- 172.21.15.57:0/58018472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194111e50 0x7f4194114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 --2- 172.21.15.57:0/58018472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194073250 0x7f4194073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 --2- 172.21.15.57:0/58018472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194073b70 0x7f4194111910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 -- 172.21.15.57:0/58018472 >> 172.21.15.57:0/58018472 conn(0x7f41940fc950 msgr2=0x7f41940fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 -- 172.21.15.57:0/58018472 shutdown_connections 2023-10-06T22:15:07.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.110+0000 7f419bfff700 1 -- 172.21.15.57:0/58018472 wait complete. 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 Processor -- start 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 -- start start 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194073250 0x7f419419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194073b70 0x7f419419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194111e50 0x7f41941a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a25d0 con 0x7f4194073250 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a2740 con 0x7f4194073b70 2023-10-06T22:15:07.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.111+0000 7f419bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a2900 con 0x7f4194111e50 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194073250 0x7f419419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194073250 0x7f419419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37248/0 (socket says 172.21.15.57:37248) 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 -- 172.21.15.57:0/3067413276 learned_addr learned my addr 172.21.15.57:0/3067413276 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f419959a700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194073b70 0x7f419419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f419a59c700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194111e50 0x7f41941a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 -- 172.21.15.57:0/3067413276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194111e50 msgr2=0x7f41941a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194111e50 0x7f41941a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 -- 172.21.15.57:0/3067413276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194073b70 msgr2=0x7f419419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194073b70 0x7f419419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 -- 172.21.15.57:0/3067413276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840095f0 con 0x7f4194073250 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f419a59c700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194111e50 0x7f41941a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:07.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.112+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194073250 0x7f419419d660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f419000e9a0 tx=0x7f419000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.113+0000 7f418affd700 1 -- 172.21.15.57:0/3067413276 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190009790 con 0x7f4194073250 2023-10-06T22:15:07.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.113+0000 7f418affd700 1 -- 172.21.15.57:0/3067413276 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419000cbd0 con 0x7f4194073250 2023-10-06T22:15:07.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.113+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41941a2be0 con 0x7f4194073250 2023-10-06T22:15:07.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.113+0000 7f418affd700 1 -- 172.21.15.57:0/3067413276 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190018680 con 0x7f4194073250 2023-10-06T22:15:07.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.113+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41941aa420 con 0x7f4194073250 2023-10-06T22:15:07.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.116+0000 7f418affd700 1 -- 172.21.15.57:0/3067413276 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f419000cd40 con 0x7f4194073250 2023-10-06T22:15:07.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.117+0000 7f41af861b80 1 Processor -- start 2023-10-06T22:15:07.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.117+0000 7f41af861b80 1 -- start start 2023-10-06T22:15:07.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.117+0000 7f418affd700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f41800a1790 0x7f41800a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f418affd700 1 -- 172.21.15.57:0/3067413276 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4190014070 con 0x7f4194073250 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f41af861b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e0c0 0x55a65fe5e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f41af861b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe69350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f41af861b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f41af861b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fdc7460 con 0x55a65fe5e0c0 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f41af861b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fdb8c20 con 0x55a65fe69890 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f41af861b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fdb6540 con 0x55a65fe5e9e0 2023-10-06T22:15:07.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f419a59c700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f41800a1790 0x7f41800a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e0c0 0x55a65fe5e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e0c0 0x55a65fe5e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37254/0 (socket says 172.21.15.57:37254) 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 -- 172.21.15.57:0/1781777319 learned_addr learned my addr 172.21.15.57:0/1781777319 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 -- 172.21.15.57:0/1781777319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 msgr2=0x55a65fe69350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f419a59c700 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f419959a700 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe69350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe69350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 -- 172.21.15.57:0/1781777319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 msgr2=0x55a65fe6bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f4199d9b700 1 -- 172.21.15.57:0/1781777319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a65fbdcde0 con 0x55a65fe5e0c0 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f419a59c700 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.118+0000 7f419959a700 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe69350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.119+0000 7f419a59c700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f41800a1790 0x7f41800a3c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f419406a470 tx=0x7f418c009920 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.119+0000 7f4199d9b700 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e0c0 0x55a65fe5e4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f419009a2b0 tx=0x7f4190099d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5623ee994a7354 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.119+0000 7f417ffff700 1 -- 172.21.15.57:0/1781777319 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419009e470 con 0x55a65fe5e0c0 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.119+0000 7f417ffff700 1 -- 172.21.15.57:0/1781777319 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41900a26b0 con 0x55a65fe5e0c0 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.119+0000 7f417ffff700 1 -- 172.21.15.57:0/1781777319 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900a2480 con 0x55a65fe5e0c0 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 -- 172.21.15.57:0/1781777319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e0c0 msgr2=0x55a65fe5e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e0c0 0x55a65fe5e4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f419009a2b0 tx=0x7f4190099d00 comp rx=0 tx=0).stop 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 -- 172.21.15.57:0/1781777319 shutdown_connections 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e0c0 0x55a65fe5e4a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 --2- 172.21.15.57:0/1781777319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe69350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 -- 172.21.15.57:0/1781777319 >> 172.21.15.57:0/1781777319 conn(0x55a65fdd3260 msgr2=0x55a65fe5a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 -- 172.21.15.57:0/1781777319 shutdown_connections 2023-10-06T22:15:07.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 -- 172.21.15.57:0/1781777319 wait complete. 2023-10-06T22:15:07.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.120+0000 7f41af861b80 1 -- start start 2023-10-06T22:15:07.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 1 Processor -- start 2023-10-06T22:15:07.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe5e0c0 0x55a65fe69330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe66540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe69890 0x55a65fe66c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fdb1e30 con 0x55a65fe69890 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fd4e710 con 0x55a65fe5e0c0 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fd4e3b0 con 0x55a65fe5e9e0 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f41af861b80 10 client.0 _ll_register_callbacks cb 0x55a65fbab800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.121+0000 7f419a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe69890 0x55a65fe66c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe5e0c0 0x55a65fe69330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe69890 0x55a65fe66c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37268/0 (socket says 172.21.15.57:37268) 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 -- 172.21.15.57:0/3059672196 learned_addr learned my addr 172.21.15.57:0/3059672196 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419959a700 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe66540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 -- 172.21.15.57:0/3059672196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 msgr2=0x55a65fe66540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe66540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 -- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe5e0c0 msgr2=0x55a65fe69330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe5e0c0 0x55a65fe69330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 -- 172.21.15.57:0/3059672196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41941aa420 con 0x55a65fe69890 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe5e0c0 0x55a65fe69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f419a59c700 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe69890 0x55a65fe66c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f418c01ae30 tx=0x7f418c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b538304fa2f96c8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.122+0000 7f417d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f417d7fa700 1 -- 172.21.15.57:0/3059672196 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c01f070 con 0x55a65fe69890 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a65fe51f10 con 0x55a65fe69890 2023-10-06T22:15:07.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f417d7fa700 1 -- 172.21.15.57:0/3059672196 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f418c01a0d0 con 0x55a65fe69890 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f417d7fa700 1 -- 172.21.15.57:0/3059672196 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f418c00e050 con 0x55a65fe69890 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 10 client.15531 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a65fbc5c60 con 0x55a65fe69890 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f417d7fa700 1 -- 172.21.15.57:0/3059672196 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a65fe51f10 con 0x55a65fe69890 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f417d7fa700 1 -- 172.21.15.57:0/3059672196 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f418c018a90 con 0x55a65fe69890 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 10 client.15531 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 20 client.15531 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fe79270 0x55a65fe7b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 4 client.15531 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.123+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a65fdaae10 con 0x55a65fe79270 2023-10-06T22:15:07.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.124+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fe79270 0x55a65fe7b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.124+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fe79270 0x55a65fe7b690 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=287c850e6c158e85 server_cookie=b206950054291772 in_seq=0 out_seq=0 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.124+0000 7f417d7fa700 10 client.15531 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f417d7fa700 1 -- 172.21.15.57:0/3059672196 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a65fdaae10 con 0x55a65fe79270 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f417d7fa700 10 client.15531 handle_command_reply: tid=0 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 client.15531 shutdown 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe69890 msgr2=0x55a65fe66c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe69890 0x55a65fe66c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f418c01ae30 tx=0x7f418c004b40 comp rx=0 tx=0).stop 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 shutdown_connections 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fe79270 0x55a65fe7b690 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe69890 0x55a65fe66c80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe5e0c0 0x55a65fe69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 --2- 172.21.15.57:0/3059672196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e9e0 0x55a65fe66540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f417d7fa700 0 client.15531 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:07.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 >> 172.21.15.57:0/3059672196 conn(0x55a65fdd3260 msgr2=0x55a65fe59fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.125+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 shutdown_connections 2023-10-06T22:15:07.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.126+0000 7f41af861b80 1 -- 172.21.15.57:0/3059672196 wait complete. 2023-10-06T22:15:07.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.126+0000 7f41af861b80 20 client.15531 trim_cache size 0 max 16384 2023-10-06T22:15:07.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 Processor -- start 2023-10-06T22:15:07.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 -- start start 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fc32920 con 0x55a65fe5e9e0 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fe76950 con 0x55a65fe69890 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.139+0000 7f41af861b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fe668a0 con 0x55a65fe5e0c0 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f419959a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe65e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe68e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59166/0 (socket says 172.21.15.57:59166) 2023-10-06T22:15:07.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 -- 172.21.15.57:0/4061515088 learned_addr learned my addr 172.21.15.57:0/4061515088 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f419959a700 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe65e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37284/0 (socket says 172.21.15.57:37284) 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f419a59c700 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 -- 172.21.15.57:0/4061515088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 msgr2=0x55a65fe66360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe66360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 -- 172.21.15.57:0/4061515088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 msgr2=0x55a65fe65e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe65e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 -- 172.21.15.57:0/4061515088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a65fc03250 con 0x55a65fe5e0c0 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.140+0000 7f4199d9b700 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe68e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f419009a020 tx=0x7f4190096790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3932c2f982acfcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f4188ff9700 1 -- 172.21.15.57:0/4061515088 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419009e170 con 0x55a65fe5e0c0 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f4188ff9700 1 -- 172.21.15.57:0/4061515088 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41900a2220 con 0x55a65fe5e0c0 2023-10-06T22:15:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f4188ff9700 1 -- 172.21.15.57:0/4061515088 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190096110 con 0x55a65fe5e0c0 2023-10-06T22:15:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 -- 172.21.15.57:0/4061515088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 msgr2=0x55a65fe68e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe68e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f419009a020 tx=0x7f4190096790 comp rx=0 tx=0).stop 2023-10-06T22:15:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 -- 172.21.15.57:0/4061515088 shutdown_connections 2023-10-06T22:15:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe66360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe68e60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 --2- 172.21.15.57:0/4061515088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 -- 172.21.15.57:0/4061515088 >> 172.21.15.57:0/4061515088 conn(0x55a65fdd3260 msgr2=0x55a65fe58bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 -- 172.21.15.57:0/4061515088 shutdown_connections 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.141+0000 7f41af861b80 1 -- 172.21.15.57:0/4061515088 wait complete. 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.142+0000 7f41af861b80 1 -- start start 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.142+0000 7f41af861b80 1 Processor -- start 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.142+0000 7f41af861b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe6e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f41af861b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe6e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f41af861b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f41af861b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fecc060 con 0x55a65fe5e9e0 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f41af861b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fecc1a0 con 0x55a65fe69890 2023-10-06T22:15:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f41af861b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65fecc2e0 con 0x55a65fe5e0c0 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f41af861b80 10 client.0 _ll_register_callbacks cb 0x7f418c01a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe6e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f4199d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe6e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe6e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37296/0 (socket says 172.21.15.57:37296) 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 -- 172.21.15.57:0/3314215076 learned_addr learned my addr 172.21.15.57:0/3314215076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419a59c700 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 -- 172.21.15.57:0/3314215076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 msgr2=0x55a65fe6e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe6e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 -- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 msgr2=0x55a65fe6ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419959a700 1 -- 172.21.15.57:0/3314215076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a65fbdcde0 con 0x55a65fe5e9e0 2023-10-06T22:15:07.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.143+0000 7f419a59c700 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f419959a700 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe6e900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4184015000 tx=0x7f418400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528700507030a405 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f417d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f417d7fa700 1 -- 172.21.15.57:0/3314215076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840056f0 con 0x55a65fe5e9e0 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41900a26b0 con 0x55a65fe5e9e0 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f417d7fa700 1 -- 172.21.15.57:0/3314215076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4184004500 con 0x55a65fe5e9e0 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f417d7fa700 1 -- 172.21.15.57:0/3314215076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41840046c0 con 0x55a65fe5e9e0 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f41af861b80 10 client.15537 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a65fbdb2b0 con 0x55a65fe5e9e0 2023-10-06T22:15:07.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f417d7fa700 1 -- 172.21.15.57:0/3314215076 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41900a26b0 con 0x55a65fe5e9e0 2023-10-06T22:15:07.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.144+0000 7f417d7fa700 1 -- 172.21.15.57:0/3314215076 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4184026930 con 0x55a65fe5e9e0 2023-10-06T22:15:07.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f41af861b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f41af861b80 10 client.15537 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:15:07.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f41af861b80 20 client.15537 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f41af861b80 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fecdb30 0x55a65fecff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f41af861b80 4 client.15537 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:07.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a65fdaab30 con 0x55a65fecdb30 2023-10-06T22:15:07.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fecdb30 0x55a65fecff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f4199d9b700 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fecdb30 0x55a65fecff50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c7ad25ab525ce95 server_cookie=106f3c642c0ef36a in_seq=0 out_seq=0 2023-10-06T22:15:07.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.145+0000 7f417d7fa700 10 client.15537 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:07.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f417d7fa700 1 -- 172.21.15.57:0/3314215076 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a65fdaab30 con 0x55a65fecdb30 2023-10-06T22:15:07.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f417d7fa700 10 client.15537 handle_command_reply: tid=0 2023-10-06T22:15:07.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 client.15537 shutdown 2023-10-06T22:15:07.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 msgr2=0x55a65fe6e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe6e900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4184015000 tx=0x7f418400bfa0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 shutdown_connections 2023-10-06T22:15:07.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55a65fecdb30 0x55a65fecff50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a65fe69890 0x55a65fe6ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a65fe5e0c0 0x55a65fe6e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f417d7fa700 0 client.15537 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:07.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 --2- 172.21.15.57:0/3314215076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a65fe5e9e0 0x55a65fe6e900 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 >> 172.21.15.57:0/3314215076 conn(0x55a65fdd3260 msgr2=0x55a65fe618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 shutdown_connections 2023-10-06T22:15:07.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 1 -- 172.21.15.57:0/3314215076 wait complete. 2023-10-06T22:15:07.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.146+0000 7f41af861b80 20 client.15537 trim_cache size 0 max 16384 2023-10-06T22:15:07.156 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:07.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f41800a1790 msgr2=0x7f41800a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f41800a1790 0x7f41800a3c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f419406a470 tx=0x7f418c009920 comp rx=0 tx=0).stop 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194073250 msgr2=0x7f419419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194073250 0x7f419419d660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f419000e9a0 tx=0x7f419000ecb0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 shutdown_connections 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f41800a1790 0x7f41800a3c50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4194111e50 0x7f41941a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4194073250 0x7f419419d660 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 --2- 172.21.15.57:0/3067413276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4194073b70 0x7f419419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 >> 172.21.15.57:0/3067413276 conn(0x7f41940fc950 msgr2=0x7f41940fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 shutdown_connections 2023-10-06T22:15:07.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.147+0000 7f419bfff700 1 -- 172.21.15.57:0/3067413276 wait complete. 2023-10-06T22:15:07.164 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:06 smithi057 ceph-mon[98492]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2023-10-06T22:15:07.167 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T22:15:07.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:06 smithi156 ceph-mon[134358]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2023-10-06T22:15:07.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 -- 172.21.15.57:0/1738037222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741010b0 msgr2=0x7f2e74101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 --2- 172.21.15.57:0/1738037222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741010b0 0x7f2e74101490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e680098f0 tx=0x7f2e68009c00 comp rx=0 tx=0).stop 2023-10-06T22:15:07.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 -- 172.21.15.57:0/1738037222 shutdown_connections 2023-10-06T22:15:07.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 --2- 172.21.15.57:0/1738037222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e741061d0 0x7f2e7410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 --2- 172.21.15.57:0/1738037222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741010b0 0x7f2e74101490 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 --2- 172.21.15.57:0/1738037222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e74101a60 0x7f2e74105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 -- 172.21.15.57:0/1738037222 >> 172.21.15.57:0/1738037222 conn(0x7f2e740fc950 msgr2=0x7f2e740fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.371+0000 7f2e7abff700 1 -- 172.21.15.57:0/1738037222 shutdown_connections 2023-10-06T22:15:07.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.372+0000 7f2e7abff700 1 -- 172.21.15.57:0/1738037222 wait complete. 2023-10-06T22:15:07.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 Processor -- start 2023-10-06T22:15:07.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 -- start start 2023-10-06T22:15:07.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e741010b0 0x7f2e74196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741061d0 0x7f2e74197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e7419b990 0x7f2e7419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7419c410 con 0x7f2e7419b990 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7419c580 con 0x7f2e741010b0 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7419c740 con 0x7f2e741061d0 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7919c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e7419b990 0x7f2e7419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7899b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e741010b0 0x7f2e74196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7919c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e7419b990 0x7f2e7419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37308/0 (socket says 172.21.15.57:37308) 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.373+0000 7f2e7919c700 1 -- 172.21.15.57:0/3058062791 learned_addr learned my addr 172.21.15.57:0/3058062791 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e73fff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741061d0 0x7f2e74197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7919c700 1 -- 172.21.15.57:0/3058062791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741061d0 msgr2=0x7f2e74197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7919c700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741061d0 0x7f2e74197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7919c700 1 -- 172.21.15.57:0/3058062791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e741010b0 msgr2=0x7f2e74196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7919c700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e741010b0 0x7f2e74196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7919c700 1 -- 172.21.15.57:0/3058062791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e600095f0 con 0x7f2e7419b990 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7899b700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e741010b0 0x7f2e74196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7919c700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e7419b990 0x7f2e7419be10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00ef30 tx=0x7f2e6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e71ffb700 1 -- 172.21.15.57:0/3058062791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018070 con 0x7f2e7419b990 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.374+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e68009520 con 0x7f2e7419b990 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.375+0000 7f2e71ffb700 1 -- 172.21.15.57:0/3058062791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6c00f040 con 0x7f2e7419b990 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.375+0000 7f2e71ffb700 1 -- 172.21.15.57:0/3058062791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00cd00 con 0x7f2e7419b990 2023-10-06T22:15:07.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.375+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e741a3f90 con 0x7f2e7419b990 2023-10-06T22:15:07.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.378+0000 7f2e71ffb700 1 -- 172.21.15.57:0/3058062791 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f2e6c003a00 con 0x7f2e7419b990 2023-10-06T22:15:07.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.379+0000 7f2e8e454b80 1 Processor -- start 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.379+0000 7f2e8e454b80 1 -- start start 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.379+0000 7f2e71ffb700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2e640a1710 0x7f2e640a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.379+0000 7f2e71ffb700 1 -- 172.21.15.57:0/3058062791 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2e6c0cce30 con 0x7f2e7419b990 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df8d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d9e0 0x562e8df98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8def6460 con 0x562e8df8d0c0 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8dee7c20 con 0x562e8df8d9e0 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8dee5540 con 0x562e8df98890 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e73fff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2e640a1710 0x7f2e640a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df8d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df8d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37320/0 (socket says 172.21.15.57:37320) 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 -- 172.21.15.57:0/2819442494 learned_addr learned my addr 172.21.15.57:0/2819442494 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e73fff700 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d9e0 0x562e8df98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7919c700 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 -- 172.21.15.57:0/2819442494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 msgr2=0x562e8df9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 -- 172.21.15.57:0/2819442494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d9e0 msgr2=0x562e8df98350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d9e0 0x562e8df98350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e7899b700 1 -- 172.21.15.57:0/2819442494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e8dd0bde0 con 0x562e8df8d0c0 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.380+0000 7f2e73fff700 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d9e0 0x562e8df98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.381+0000 7f2e73fff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2e640a1710 0x7f2e640a3bd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009fd0 tx=0x7f2e60013a60 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.381+0000 7f2e7899b700 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df8d4a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e68005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d8365ae853a4c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.381+0000 7f2e5effd700 1 -- 172.21.15.57:0/2819442494 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6801c070 con 0x562e8df8d0c0 2023-10-06T22:15:07.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.381+0000 7f2e5effd700 1 -- 172.21.15.57:0/2819442494 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6801d040 con 0x562e8df8d0c0 2023-10-06T22:15:07.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.381+0000 7f2e5effd700 1 -- 172.21.15.57:0/2819442494 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680052a0 con 0x562e8df8d0c0 2023-10-06T22:15:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 -- 172.21.15.57:0/2819442494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 msgr2=0x562e8df8d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df8d4a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e68005dc0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 -- 172.21.15.57:0/2819442494 shutdown_connections 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df8d4a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/2819442494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d9e0 0x562e8df98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 -- 172.21.15.57:0/2819442494 >> 172.21.15.57:0/2819442494 conn(0x562e8df02260 msgr2=0x562e8df89450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 -- 172.21.15.57:0/2819442494 shutdown_connections 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.382+0000 7f2e8e454b80 1 -- 172.21.15.57:0/2819442494 wait complete. 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 -- start start 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 Processor -- start 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df98290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df95510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df95b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8dee0e30 con 0x562e8df8d9e0 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8de7d710 con 0x562e8df8d0c0 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8de7d3b0 con 0x562e8df98890 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.383+0000 7f2e8e454b80 10 client.0 _ll_register_callbacks cb 0x562e8dcda800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e73fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df95510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e7919c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df95b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e7919c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df95b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59230/0 (socket says 172.21.15.57:59230) 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e7899b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df98290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e7919c700 1 -- 172.21.15.57:0/3769173547 learned_addr learned my addr 172.21.15.57:0/3769173547 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e73fff700 1 -- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 msgr2=0x562e8df95b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e73fff700 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df95b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e73fff700 1 -- 172.21.15.57:0/3769173547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 msgr2=0x562e8df98290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e73fff700 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df98290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e73fff700 1 -- 172.21.15.57:0/3769173547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7410e5c0 con 0x562e8df8d9e0 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e7899b700 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df98290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e7919c700 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df95b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e73fff700 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df95510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2e60013510 tx=0x7f2e600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6773bec322149fba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.384+0000 7f2e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e53fff700 1 -- 172.21.15.57:0/3769173547 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60018070 con 0x562e8df8d9e0 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e8df80f10 con 0x562e8df8d9e0 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e53fff700 1 -- 172.21.15.57:0/3769173547 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2e6000f680 con 0x562e8df8d9e0 2023-10-06T22:15:07.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e53fff700 1 -- 172.21.15.57:0/3769173547 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e600122e0 con 0x562e8df8d9e0 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e8e454b80 10 client.15549 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562e8dcf4c60 con 0x562e8df8d9e0 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e53fff700 1 -- 172.21.15.57:0/3769173547 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e8df80f10 con 0x562e8df8d9e0 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.385+0000 7f2e53fff700 1 -- 172.21.15.57:0/3769173547 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e600108e0 con 0x562e8df8d9e0 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.386+0000 7f2e8e454b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.386+0000 7f2e8e454b80 10 client.15549 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.386+0000 7f2e8e454b80 20 client.15549 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.386+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8dfa81f0 0x562e8dfaa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.386+0000 7f2e8e454b80 4 client.15549 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.386+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e8ded9e10 con 0x562e8dfa81f0 2023-10-06T22:15:07.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.386+0000 7f2e7899b700 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8dfa81f0 0x562e8dfaa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.387+0000 7f2e7899b700 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8dfa81f0 0x562e8dfaa610 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b13a3a9706d2f7a1 server_cookie=e1384b8068ec0c27 in_seq=0 out_seq=0 2023-10-06T22:15:07.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.387+0000 7f2e53fff700 10 client.15549 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e53fff700 1 -- 172.21.15.57:0/3769173547 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e8ded9e10 con 0x562e8dfa81f0 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e53fff700 10 client.15549 handle_command_reply: tid=0 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 client.15549 shutdown 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 msgr2=0x562e8df95510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df95510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2e60013510 tx=0x7f2e600047c0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 shutdown_connections 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8dfa81f0 0x562e8dfaa610 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df95b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df98290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/3769173547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df95510 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e53fff700 0 client.15549 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:07.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 >> 172.21.15.57:0/3769173547 conn(0x562e8df02260 msgr2=0x562e8df88fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 shutdown_connections 2023-10-06T22:15:07.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 1 -- 172.21.15.57:0/3769173547 wait complete. 2023-10-06T22:15:07.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.388+0000 7f2e8e454b80 20 client.15549 trim_cache size 0 max 16384 2023-10-06T22:15:07.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 Processor -- start 2023-10-06T22:15:07.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 -- start start 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df97e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8dd61920 con 0x562e8df8d9e0 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8df91380 con 0x562e8df8d0c0 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8df8ff60 con 0x562e8df98890 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e73fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df97e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e73fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df97e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37340/0 (socket says 172.21.15.57:37340) 2023-10-06T22:15:07.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e73fff700 1 -- 172.21.15.57:0/216238636 learned_addr learned my addr 172.21.15.57:0/216238636 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.402+0000 7f2e7899b700 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e7919c700 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e73fff700 1 -- 172.21.15.57:0/216238636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 msgr2=0x562e8df90810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e73fff700 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df90810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e73fff700 1 -- 172.21.15.57:0/216238636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 msgr2=0x562e8df9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e73fff700 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e73fff700 1 -- 172.21.15.57:0/216238636 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e8dd32250 con 0x562e8df8d9e0 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e73fff700 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df97e30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2e600045b0 tx=0x7f2e600045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1283a2893639ace4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e5f7fe700 1 -- 172.21.15.57:0/216238636 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60018070 con 0x562e8df8d9e0 2023-10-06T22:15:07.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e5f7fe700 1 -- 172.21.15.57:0/216238636 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e600122e0 con 0x562e8df8d9e0 2023-10-06T22:15:07.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.403+0000 7f2e5f7fe700 1 -- 172.21.15.57:0/216238636 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60020a50 con 0x562e8df8d9e0 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 -- 172.21.15.57:0/216238636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 msgr2=0x562e8df97e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df97e30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2e600045b0 tx=0x7f2e600045e0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 -- 172.21.15.57:0/216238636 shutdown_connections 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df98890 0x562e8df90810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df8d0c0 0x562e8df9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/216238636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d9e0 0x562e8df97e30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 -- 172.21.15.57:0/216238636 >> 172.21.15.57:0/216238636 conn(0x562e8df02260 msgr2=0x562e8dfa45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 -- 172.21.15.57:0/216238636 shutdown_connections 2023-10-06T22:15:07.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 -- 172.21.15.57:0/216238636 wait complete. 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.404+0000 7f2e8e454b80 1 -- start start 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 1 Processor -- start 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df9d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df8d9e0 0x562e8df9d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df98890 0x562e8df9dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8dfff450 con 0x562e8df8d0c0 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8dfff5c0 con 0x562e8df98890 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8dfff780 con 0x562e8df8d9e0 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e8e454b80 10 client.0 _ll_register_callbacks cb 0x7f2e60004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e7899b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df9d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e7899b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df9d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37344/0 (socket says 172.21.15.57:37344) 2023-10-06T22:15:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e7899b700 1 -- 172.21.15.57:0/961574245 learned_addr learned my addr 172.21.15.57:0/961574245 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.405+0000 7f2e73fff700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df8d9e0 0x562e8df9d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7919c700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df98890 0x562e8df9dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7899b700 1 -- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df8d9e0 msgr2=0x562e8df9d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7899b700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df8d9e0 0x562e8df9d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7899b700 1 -- 172.21.15.57:0/961574245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df98890 msgr2=0x562e8df9dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7899b700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df98890 0x562e8df9dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7899b700 1 -- 172.21.15.57:0/961574245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e8dcf4c60 con 0x562e8df8d0c0 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7919c700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df98890 0x562e8df9dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e7899b700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df9d360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800f8b0 tx=0x7f2e68009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be84a0e7f0b00bad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e53fff700 1 -- 172.21.15.57:0/961574245 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68031040 con 0x562e8df8d0c0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e6801d040 con 0x562e8df8d0c0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.406+0000 7f2e53fff700 1 -- 172.21.15.57:0/961574245 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2e680052e0 con 0x562e8df8d0c0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e53fff700 1 -- 172.21.15.57:0/961574245 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e6801c040 con 0x562e8df8d0c0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 10 client.15555 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562e8dd0a2b0 con 0x562e8df8d0c0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e53fff700 1 -- 172.21.15.57:0/961574245 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6801d040 con 0x562e8df8d0c0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e53fff700 1 -- 172.21.15.57:0/961574245 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e68009c70 con 0x562e8df8d0c0 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 10 client.15555 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 20 client.15555 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8e000b10 0x562e8e002f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 4 client.15555 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e8ded9b30 con 0x562e8e000b10 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.407+0000 7f2e73fff700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8e000b10 0x562e8e002f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e73fff700 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8e000b10 0x562e8e002f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7a8a492b2f6c2f04 server_cookie=b2119d66fca5f89e in_seq=0 out_seq=0 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e53fff700 10 client.15555 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e53fff700 1 -- 172.21.15.57:0/961574245 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e8ded9b30 con 0x562e8e000b10 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e53fff700 10 client.15555 handle_command_reply: tid=0 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e8e454b80 1 client.15555 shutdown 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 msgr2=0x562e8df9d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df9d360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800f8b0 tx=0x7f2e68009400 comp rx=0 tx=0).stop 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 shutdown_connections 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x562e8e000b10 0x562e8e002f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e8df98890 0x562e8df9dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e53fff700 0 client.15555 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.408+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e8df8d0c0 0x562e8df9d360 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e8e454b80 1 --2- 172.21.15.57:0/961574245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e8df8d9e0 0x562e8df9d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 >> 172.21.15.57:0/961574245 conn(0x562e8df02260 msgr2=0x562e8df90810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 shutdown_connections 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e8e454b80 1 -- 172.21.15.57:0/961574245 wait complete. 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e8e454b80 20 client.15555 trim_cache size 0 max 16384 2023-10-06T22:15:07.417 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:07.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2e640a1710 msgr2=0x7f2e640a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2e640a1710 0x7f2e640a3bd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009fd0 tx=0x7f2e60013a60 comp rx=0 tx=0).stop 2023-10-06T22:15:07.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e7419b990 msgr2=0x7f2e7419be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e7419b990 0x7f2e7419be10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00ef30 tx=0x7f2e6c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 shutdown_connections 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2e640a1710 0x7f2e640a3bd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e7419b990 0x7f2e7419be10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e741010b0 0x7f2e74196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 --2- 172.21.15.57:0/3058062791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e741061d0 0x7f2e74197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.409+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 >> 172.21.15.57:0/3058062791 conn(0x7f2e740fc950 msgr2=0x7f2e74104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.410+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 shutdown_connections 2023-10-06T22:15:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.410+0000 7f2e7abff700 1 -- 172.21.15.57:0/3058062791 wait complete. 2023-10-06T22:15:07.430 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 -- 172.21.15.57:0/1448760077 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b01010b0 msgr2=0x7fd6b0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/1448760077 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b01010b0 0x7fd6b0101490 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009990 tx=0x7fd6ac009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 -- 172.21.15.57:0/1448760077 shutdown_connections 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/1448760077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b01061d0 0x7fd6b010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/1448760077 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b01010b0 0x7fd6b0101490 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/1448760077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0101a60 0x7fd6b0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 -- 172.21.15.57:0/1448760077 >> 172.21.15.57:0/1448760077 conn(0x7fd6b00fc950 msgr2=0x7fd6b00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.634+0000 7fd6b7bff700 1 -- 172.21.15.57:0/1448760077 shutdown_connections 2023-10-06T22:15:07.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.635+0000 7fd6b7bff700 1 -- 172.21.15.57:0/1448760077 wait complete. 2023-10-06T22:15:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.635+0000 7fd6b7bff700 1 Processor -- start 2023-10-06T22:15:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.636+0000 7fd6b7bff700 1 -- start start 2023-10-06T22:15:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.636+0000 7fd6b7bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b01010b0 0x7fd6b01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.636+0000 7fd6b7bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b01061d0 0x7fd6b0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.636+0000 7fd6b7bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b019db80 0x7fd6b019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.636+0000 7fd6b7bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b019e600 con 0x7fd6b019db80 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.636+0000 7fd6b7bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b019e770 con 0x7fd6b01061d0 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.636+0000 7fd6b7bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b019e930 con 0x7fd6b01010b0 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b01061d0 0x7fd6b0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b619c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b019db80 0x7fd6b019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b01061d0 0x7fd6b0199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55582/0 (socket says 172.21.15.57:55582) 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b519a700 1 -- 172.21.15.57:0/4218599015 learned_addr learned my addr 172.21.15.57:0/4218599015 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b599b700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b01010b0 0x7fd6b01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b619c700 1 -- 172.21.15.57:0/4218599015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b01010b0 msgr2=0x7fd6b01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b619c700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b01010b0 0x7fd6b01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b619c700 1 -- 172.21.15.57:0/4218599015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b01061d0 msgr2=0x7fd6b0199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b619c700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b01061d0 0x7fd6b0199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b619c700 1 -- 172.21.15.57:0/4218599015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0095f0 con 0x7fd6b019db80 2023-10-06T22:15:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6b619c700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b019db80 0x7fd6b019e000 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8007f10 tx=0x7fd6a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.637+0000 7fd6a6ffd700 1 -- 172.21.15.57:0/4218599015 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8004db0 con 0x7fd6b019db80 2023-10-06T22:15:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.638+0000 7fd6a6ffd700 1 -- 172.21.15.57:0/4218599015 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6a800f040 con 0x7fd6b019db80 2023-10-06T22:15:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.638+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b01a5e90 con 0x7fd6b019db80 2023-10-06T22:15:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.638+0000 7fd6a6ffd700 1 -- 172.21.15.57:0/4218599015 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8005990 con 0x7fd6b019db80 2023-10-06T22:15:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.638+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01a6320 con 0x7fd6b019db80 2023-10-06T22:15:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.641+0000 7fd6a6ffd700 1 -- 172.21.15.57:0/4218599015 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fd6a800db20 con 0x7fd6b019db80 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.642+0000 7fd6cb53cb80 1 Processor -- start 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.642+0000 7fd6cb53cb80 1 -- start start 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.642+0000 7fd6a6ffd700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd69c0a1710 0x7fd69c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d4b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d58c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6a6ffd700 1 -- 172.21.15.57:0/4218599015 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd6a80ccbe0 con 0x7fd6b019db80 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1cb4460 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1ca5c20 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1ca3540 con 0x55dcd1d56890 2023-10-06T22:15:07.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6b519a700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd69c0a1710 0x7fd69c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6b599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d4b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6b519a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6b599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d4b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55586/0 (socket says 172.21.15.57:55586) 2023-10-06T22:15:07.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6b599b700 1 -- 172.21.15.57:0/982577394 learned_addr learned my addr 172.21.15.57:0/982577394 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.643+0000 7fd6b619c700 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d58c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b619c700 1 -- 172.21.15.57:0/982577394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b0c0 msgr2=0x55dcd1d4b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b619c700 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d4b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b619c700 1 -- 172.21.15.57:0/982577394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b9e0 msgr2=0x55dcd1d56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b619c700 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b619c700 1 -- 172.21.15.57:0/982577394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcd1ac9de0 con 0x55dcd1d56890 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b599b700 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d4b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b619c700 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d58c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80ca1c0 tx=0x7fd6a8005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ef2c2c48bddd63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd6b519a700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd69c0a1710 0x7fd69c0a3bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b750 tx=0x7fd6a000bc40 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd69bfff700 1 -- 172.21.15.57:0/982577394 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a809a080 con 0x55dcd1d56890 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd69bfff700 1 -- 172.21.15.57:0/982577394 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6a8099980 con 0x55dcd1d56890 2023-10-06T22:15:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.644+0000 7fd69bfff700 1 -- 172.21.15.57:0/982577394 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8099be0 con 0x55dcd1d56890 2023-10-06T22:15:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/982577394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 msgr2=0x55dcd1d58c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d58c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80ca1c0 tx=0x7fd6a8005bc0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/982577394 shutdown_connections 2023-10-06T22:15:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d58c80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d4b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/982577394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/982577394 >> 172.21.15.57:0/982577394 conn(0x55dcd1cc0260 msgr2=0x55dcd1d47450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/982577394 shutdown_connections 2023-10-06T22:15:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/982577394 wait complete. 2023-10-06T22:15:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.645+0000 7fd6cb53cb80 1 -- start start 2023-10-06T22:15:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 1 Processor -- start 2023-10-06T22:15:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d56310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d53520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d56890 0x55dcd1d53c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1c9ee30 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1c3b710 con 0x55dcd1d56890 2023-10-06T22:15:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1c3b3b0 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6cb53cb80 10 client.0 _ll_register_callbacks cb 0x55dcd1a98800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6b599b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d56310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6b519a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d53520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6b599b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d56310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37374/0 (socket says 172.21.15.57:37374) 2023-10-06T22:15:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.646+0000 7fd6b599b700 1 -- 172.21.15.57:0/896265545 learned_addr learned my addr 172.21.15.57:0/896265545 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b619c700 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d56890 0x55dcd1d53c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b519a700 1 -- 172.21.15.57:0/896265545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d56890 msgr2=0x55dcd1d53c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b519a700 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d56890 0x55dcd1d53c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b519a700 1 -- 172.21.15.57:0/896265545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 msgr2=0x55dcd1d56310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b519a700 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d56310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b519a700 1 -- 172.21.15.57:0/896265545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b010e5c0 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b599b700 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d56310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:07.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6b519a700 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d53520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd6b019a7f0 tx=0x7fd6a001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62578aef39682334 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6997fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:07.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6997fa700 1 -- 172.21.15.57:0/896265545 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a001e8b0 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.647+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcd1d3ef10 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6997fa700 1 -- 172.21.15.57:0/896265545 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd6a001ea10 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6997fa700 1 -- 172.21.15.57:0/896265545 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a001ebd0 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 10 client.25579 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fd6b01a6320 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6997fa700 1 -- 172.21.15.57:0/896265545 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcd1d3ef10 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6997fa700 1 -- 172.21.15.57:0/896265545 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6a0020ca0 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 10 client.25579 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 10 client.25579 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 20 client.25579 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1d66250 0x55dcd1d68670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 4 client.25579 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.648+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcd1c97e10 con 0x55dcd1d66250 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.649+0000 7fd6b599b700 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1d66250 0x55dcd1d68670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.649+0000 7fd6b599b700 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1d66250 0x55dcd1d68670 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb8f3076fd33b4fa server_cookie=43805853304d4cd4 in_seq=0 out_seq=0 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.649+0000 7fd6997fa700 10 client.25579 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.650+0000 7fd6997fa700 1 -- 172.21.15.57:0/896265545 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dcd1c97e10 con 0x55dcd1d66250 2023-10-06T22:15:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.650+0000 7fd6997fa700 10 client.25579 handle_command_reply: tid=0 2023-10-06T22:15:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.650+0000 7fd6cb53cb80 1 client.25579 shutdown 2023-10-06T22:15:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.650+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d4b9e0 msgr2=0x55dcd1d53520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.650+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d53520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd6b019a7f0 tx=0x7fd6a001c000 comp rx=0 tx=0).stop 2023-10-06T22:15:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 shutdown_connections 2023-10-06T22:15:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1d66250 0x55dcd1d68670 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d56890 0x55dcd1d53c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d56310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/896265545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d53520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6997fa700 0 client.25579 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 >> 172.21.15.57:0/896265545 conn(0x55dcd1cc0260 msgr2=0x55dcd1d46fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 shutdown_connections 2023-10-06T22:15:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/896265545 wait complete. 2023-10-06T22:15:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.651+0000 7fd6cb53cb80 20 client.25579 trim_cache size 0 max 16384 2023-10-06T22:15:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.664+0000 7fd6cb53cb80 1 Processor -- start 2023-10-06T22:15:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.664+0000 7fd6cb53cb80 1 -- start start 2023-10-06T22:15:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d55e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d52dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1b1f920 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1d48060 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1d53840 con 0x55dcd1d56890 2023-10-06T22:15:07.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d55e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d55e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37384/0 (socket says 172.21.15.57:37384) 2023-10-06T22:15:07.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 -- 172.21.15.57:0/3415301102 learned_addr learned my addr 172.21.15.57:0/3415301102 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b519a700 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d52dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b619c700 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 -- 172.21.15.57:0/3415301102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 msgr2=0x55dcd1d53300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d53300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 -- 172.21.15.57:0/3415301102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 msgr2=0x55dcd1d52dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d52dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 -- 172.21.15.57:0/3415301102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcd1ab2c60 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.665+0000 7fd6b599b700 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d55e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55dcd1d68f50 tx=0x7fd6ac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f719fbf6b9760a00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6a4bff700 1 -- 172.21.15.57:0/3415301102 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac01d070 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6a4bff700 1 -- 172.21.15.57:0/3415301102 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6ac01e040 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6a4bff700 1 -- 172.21.15.57:0/3415301102 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac004070 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/3415301102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 msgr2=0x55dcd1d55e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d55e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55dcd1d68f50 tx=0x7fd6ac009310 comp rx=0 tx=0).stop 2023-10-06T22:15:07.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/3415301102 shutdown_connections 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1d53300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d55e40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/3415301102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d52dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/3415301102 >> 172.21.15.57:0/3415301102 conn(0x55dcd1cc0260 msgr2=0x55dcd1d4e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.666+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/3415301102 shutdown_connections 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.667+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/3415301102 wait complete. 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.667+0000 7fd6cb53cb80 1 -- start start 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.667+0000 7fd6cb53cb80 1 Processor -- start 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6cb53cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1db30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1db3600 con 0x55dcd1d4b0c0 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1db3770 con 0x55dcd1d4b9e0 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6cb53cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd1db3930 con 0x55dcd1d56890 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6cb53cb80 10 client.0 _ll_register_callbacks cb 0x55dcd1a98800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1db30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1db30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59288/0 (socket says 172.21.15.57:59288) 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 -- 172.21.15.57:0/125738169 learned_addr learned my addr 172.21.15.57:0/125738169 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b519a700 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d5ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b599b700 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 -- 172.21.15.57:0/125738169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 msgr2=0x55dcd1d5ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d5ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 -- 172.21.15.57:0/125738169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 msgr2=0x55dcd1d5b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d5b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.668+0000 7fd6b619c700 1 -- 172.21.15.57:0/125738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac005510 con 0x55dcd1d56890 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6b619c700 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1db30c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8005c30 tx=0x7fd6a8007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afecb0be78df5436 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6997fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6997fa700 1 -- 172.21.15.57:0/125738169 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a809dad0 con 0x55dcd1d56890 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6a8099980 con 0x55dcd1d56890 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6997fa700 1 -- 172.21.15.57:0/125738169 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd6a809a080 con 0x55dcd1d56890 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6997fa700 1 -- 172.21.15.57:0/125738169 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a80a1d10 con 0x55dcd1d56890 2023-10-06T22:15:07.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 10 client.25585 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dcd1af0250 con 0x55dcd1d56890 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6997fa700 1 -- 172.21.15.57:0/125738169 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6a8063080 con 0x55dcd1d56890 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6997fa700 1 -- 172.21.15.57:0/125738169 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6a80b90b0 con 0x55dcd1d56890 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 10 client.25585 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 10 client.25585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 20 client.25585 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1dbae80 0x55dcd1dbd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 4 client.25585 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.669+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dcd1c97b30 con 0x55dcd1dbae80 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.670+0000 7fd6b599b700 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1dbae80 0x55dcd1dbd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.670+0000 7fd6b599b700 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1dbae80 0x55dcd1dbd2a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8e7ccf070e30977 server_cookie=6ae81aef2ce1cbfd in_seq=0 out_seq=0 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.670+0000 7fd6997fa700 10 client.25585 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.670+0000 7fd6997fa700 1 -- 172.21.15.57:0/125738169 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dcd1c97b30 con 0x55dcd1dbae80 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.670+0000 7fd6997fa700 10 client.25585 handle_command_reply: tid=0 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 client.25585 shutdown 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 msgr2=0x55dcd1db30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1db30c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8005c30 tx=0x7fd6a8007f40 comp rx=0 tx=0).stop 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 shutdown_connections 2023-10-06T22:15:07.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55dcd1dbae80 0x55dcd1dbd2a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcd1d56890 0x55dcd1db30c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dcd1d4b0c0 0x55dcd1d5b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 --2- 172.21.15.57:0/125738169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcd1d4b9e0 0x55dcd1d5ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6997fa700 0 client.25585 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 >> 172.21.15.57:0/125738169 conn(0x55dcd1cc0260 msgr2=0x55dcd1d4e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 shutdown_connections 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 1 -- 172.21.15.57:0/125738169 wait complete. 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.671+0000 7fd6cb53cb80 20 client.25585 trim_cache size 0 max 16384 2023-10-06T22:15:07.682 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:07.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd69c0a1710 msgr2=0x7fd69c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd69c0a1710 0x7fd69c0a3bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b750 tx=0x7fd6a000bc40 comp rx=0 tx=0).stop 2023-10-06T22:15:07.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b019db80 msgr2=0x7fd6b019e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b019db80 0x7fd6b019e000 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8007f10 tx=0x7fd6a800d3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 shutdown_connections 2023-10-06T22:15:07.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd69c0a1710 0x7fd69c0a3bd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6b019db80 0x7fd6b019e000 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b01010b0 0x7fd6b01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 --2- 172.21.15.57:0/4218599015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b01061d0 0x7fd6b0199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 >> 172.21.15.57:0/4218599015 conn(0x7fd6b00fc950 msgr2=0x7fd6b00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 shutdown_connections 2023-10-06T22:15:07.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.672+0000 7fd6b7bff700 1 -- 172.21.15.57:0/4218599015 wait complete. 2023-10-06T22:15:07.692 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T22:15:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.903+0000 7fc4387ff700 1 -- 172.21.15.57:0/741823427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 msgr2=0x7fc43010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.903+0000 7fc4387ff700 1 --2- 172.21.15.57:0/741823427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc43010adf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc420009990 tx=0x7fc420009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.904+0000 7fc4387ff700 1 -- 172.21.15.57:0/741823427 shutdown_connections 2023-10-06T22:15:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.904+0000 7fc4387ff700 1 --2- 172.21.15.57:0/741823427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43010b410 0x7fc43010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.904+0000 7fc4387ff700 1 --2- 172.21.15.57:0/741823427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc430101ff0 0x7fc4301023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.904+0000 7fc4387ff700 1 --2- 172.21.15.57:0/741823427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc43010adf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.904+0000 7fc4387ff700 1 -- 172.21.15.57:0/741823427 >> 172.21.15.57:0/741823427 conn(0x7fc4300fb830 msgr2=0x7fc4300fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.904+0000 7fc4387ff700 1 -- 172.21.15.57:0/741823427 shutdown_connections 2023-10-06T22:15:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.904+0000 7fc4387ff700 1 -- 172.21.15.57:0/741823427 wait complete. 2023-10-06T22:15:07.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 Processor -- start 2023-10-06T22:15:07.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 -- start start 2023-10-06T22:15:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc430101ff0 0x7fc4300ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc430100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43010b410 0x7fc4301a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301009b0 con 0x7fc430102910 2023-10-06T22:15:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301a56e0 con 0x7fc430101ff0 2023-10-06T22:15:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.905+0000 7fc4387ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301a5820 con 0x7fc43010b410 2023-10-06T22:15:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc43659b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc430101ff0 0x7fc4300ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc430100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc436d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43010b410 0x7fc4301a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc436d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43010b410 0x7fc4301a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59316/0 (socket says 172.21.15.57:59316) 2023-10-06T22:15:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc430100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37400/0 (socket says 172.21.15.57:37400) 2023-10-06T22:15:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc43659b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc430101ff0 0x7fc4300ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55632/0 (socket says 172.21.15.57:55632) 2023-10-06T22:15:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc436d9c700 1 -- 172.21.15.57:0/78440659 learned_addr learned my addr 172.21.15.57:0/78440659 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 -- 172.21.15.57:0/78440659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43010b410 msgr2=0x7fc4301a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43010b410 0x7fc4301a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 -- 172.21.15.57:0/78440659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc430101ff0 msgr2=0x7fc4300ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc430101ff0 0x7fc4300ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 -- 172.21.15.57:0/78440659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200095f0 con 0x7fc430102910 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.906+0000 7fc435d9a700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc430100180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc420000c00 tx=0x7fc420004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.907+0000 7fc4277fe700 1 -- 172.21.15.57:0/78440659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001d070 con 0x7fc430102910 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.907+0000 7fc4277fe700 1 -- 172.21.15.57:0/78440659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc420022470 con 0x7fc430102910 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.907+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4301a5a20 con 0x7fc430102910 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.907+0000 7fc4277fe700 1 -- 172.21.15.57:0/78440659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420017650 con 0x7fc430102910 2023-10-06T22:15:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.907+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4301a5ea0 con 0x7fc430102910 2023-10-06T22:15:07.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.910+0000 7fc4277fe700 1 -- 172.21.15.57:0/78440659 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fc4200225e0 con 0x7fc430102910 2023-10-06T22:15:07.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.912+0000 7fc44c191b80 1 Processor -- start 2023-10-06T22:15:07.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.912+0000 7fc4277fe700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc41c0a1710 0x7fc41c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.912+0000 7fc44c191b80 1 -- start start 2023-10-06T22:15:07.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.912+0000 7fc4277fe700 1 -- 172.21.15.57:0/78440659 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc4200ce540 con 0x7fc430102910 2023-10-06T22:15:07.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc436d9c700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc41c0a1710 0x7fc41c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda6314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda63c890 0x55acda63ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc44c191b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda59a460 con 0x55acda6310c0 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc44c191b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda58bc20 con 0x55acda63c890 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc44c191b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda589540 con 0x55acda6319e0 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc43659b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda6314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc43659b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda6314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37406/0 (socket says 172.21.15.57:37406) 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc43659b700 1 -- 172.21.15.57:0/857016865 learned_addr learned my addr 172.21.15.57:0/857016865 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc435d9a700 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc436d9c700 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda63c890 0x55acda63ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.913+0000 7fc436d9c700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc41c0a1710 0x7fc41c0a3bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc430101d50 tx=0x7fc42800b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc43659b700 1 -- 172.21.15.57:0/857016865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 msgr2=0x55acda63c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc43659b700 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc43659b700 1 -- 172.21.15.57:0/857016865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda63c890 msgr2=0x55acda63ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc43659b700 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda63c890 0x55acda63ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc43659b700 1 -- 172.21.15.57:0/857016865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acda3afde0 con 0x55acda6310c0 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc435d9a700 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc43659b700 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda6314a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009d80 tx=0x7fc42c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae3b34c8a92b951 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc424ff9700 1 -- 172.21.15.57:0/857016865 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c004230 con 0x55acda6310c0 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc424ff9700 1 -- 172.21.15.57:0/857016865 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc42c004390 con 0x55acda6310c0 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.914+0000 7fc424ff9700 1 -- 172.21.15.57:0/857016865 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c019860 con 0x55acda6310c0 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 -- 172.21.15.57:0/857016865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 msgr2=0x55acda6314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda6314a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009d80 tx=0x7fc42c011720 comp rx=0 tx=0).stop 2023-10-06T22:15:07.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 -- 172.21.15.57:0/857016865 shutdown_connections 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda63c890 0x55acda63ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda6314a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 --2- 172.21.15.57:0/857016865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 -- 172.21.15.57:0/857016865 >> 172.21.15.57:0/857016865 conn(0x55acda5a6260 msgr2=0x55acda62d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 -- 172.21.15.57:0/857016865 shutdown_connections 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.915+0000 7fc44c191b80 1 -- 172.21.15.57:0/857016865 wait complete. 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 -- start start 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 Processor -- start 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda63c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda6394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda63c890 0x55acda639ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda584e30 con 0x55acda6310c0 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda521710 con 0x55acda6319e0 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda5213b0 con 0x55acda63c890 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.916+0000 7fc44c191b80 10 client.0 _ll_register_callbacks cb 0x55acda37e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda63c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda63c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37416/0 (socket says 172.21.15.57:37416) 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 -- 172.21.15.57:0/3541838289 learned_addr learned my addr 172.21.15.57:0/3541838289 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc435d9a700 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda6394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc436d9c700 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda63c890 0x55acda639ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 -- 172.21.15.57:0/3541838289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda63c890 msgr2=0x55acda639ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda63c890 0x55acda639ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 -- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 msgr2=0x55acda6394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda6394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 -- 172.21.15.57:0/3541838289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4301a5ea0 con 0x55acda6310c0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc43659b700 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda63c2a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc42c011a70 tx=0x7fc42c003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44877cd2a40b193d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.917+0000 7fc419ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc419ffb700 1 -- 172.21.15.57:0/3541838289 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c005450 con 0x55acda6310c0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acda624f10 con 0x55acda6310c0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc419ffb700 1 -- 172.21.15.57:0/3541838289 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc42c004500 con 0x55acda6310c0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc419ffb700 1 -- 172.21.15.57:0/3541838289 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42c003da0 con 0x55acda6310c0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc44c191b80 10 client.15585 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc42c0099e0 con 0x55acda6310c0 2023-10-06T22:15:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc419ffb700 1 -- 172.21.15.57:0/3541838289 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acda624f10 con 0x55acda6310c0 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc419ffb700 1 -- 172.21.15.57:0/3541838289 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc42c021040 con 0x55acda6310c0 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc44c191b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc44c191b80 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.918+0000 7fc44c191b80 20 client.15585 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.919+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda64c190 0x55acda64e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.919+0000 7fc44c191b80 4 client.15585 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.919+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acda57de10 con 0x55acda64c190 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.919+0000 7fc435d9a700 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda64c190 0x55acda64e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.919+0000 7fc435d9a700 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda64c190 0x55acda64e5b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a69ad4e9e3b7c014 server_cookie=2d7d207c2dcd13c in_seq=0 out_seq=0 2023-10-06T22:15:07.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.919+0000 7fc419ffb700 10 client.15585 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:07.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.920+0000 7fc419ffb700 1 -- 172.21.15.57:0/3541838289 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55acda57de10 con 0x55acda64c190 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.920+0000 7fc419ffb700 10 client.15585 handle_command_reply: tid=0 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.920+0000 7fc44c191b80 1 client.15585 shutdown 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 msgr2=0x55acda63c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda63c2a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc42c011a70 tx=0x7fc42c003930 comp rx=0 tx=0).stop 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 shutdown_connections 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda64c190 0x55acda64e5b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda63c890 0x55acda639ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda6310c0 0x55acda63c2a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc419ffb700 0 client.15585 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3541838289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda6394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 >> 172.21.15.57:0/3541838289 conn(0x55acda5a6260 msgr2=0x55acda62cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 shutdown_connections 2023-10-06T22:15:07.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 1 -- 172.21.15.57:0/3541838289 wait complete. 2023-10-06T22:15:07.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.921+0000 7fc44c191b80 20 client.15585 trim_cache size 0 max 16384 2023-10-06T22:15:07.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.934+0000 7fc44c191b80 1 Processor -- start 2023-10-06T22:15:07.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.934+0000 7fc44c191b80 1 -- start start 2023-10-06T22:15:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6310c0 0x55acda6433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda638d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc44c191b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda405920 con 0x55acda63c890 2023-10-06T22:15:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc44c191b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda629a20 con 0x55acda6310c0 2023-10-06T22:15:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc44c191b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda63a0e0 con 0x55acda6319e0 2023-10-06T22:15:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc436d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda638d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc43659b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6310c0 0x55acda6433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc43659b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6310c0 0x55acda6433d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55652/0 (socket says 172.21.15.57:55652) 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc435d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc436d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda638d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37422/0 (socket says 172.21.15.57:37422) 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.935+0000 7fc43659b700 1 -- 172.21.15.57:0/2600370057 learned_addr learned my addr 172.21.15.57:0/2600370057 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc43659b700 1 -- 172.21.15.57:0/2600370057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 msgr2=0x55acda63bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc43659b700 1 --2- 172.21.15.57:0/2600370057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc43659b700 1 -- 172.21.15.57:0/2600370057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 msgr2=0x55acda638d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc43659b700 1 --2- 172.21.15.57:0/2600370057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda638d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc43659b700 1 -- 172.21.15.57:0/2600370057 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acda398c60 con 0x55acda6310c0 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc43659b700 1 --2- 172.21.15.57:0/2600370057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6310c0 0x55acda6433d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc42c004e90 tx=0x7fc42c0195e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a422c9c93f52e4a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc4257fa700 1 -- 172.21.15.57:0/2600370057 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c003cd0 con 0x55acda6310c0 2023-10-06T22:15:07.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc4257fa700 1 -- 172.21.15.57:0/2600370057 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc42c005710 con 0x55acda6310c0 2023-10-06T22:15:07.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.936+0000 7fc4257fa700 1 -- 172.21.15.57:0/2600370057 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c004500 con 0x55acda6310c0 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 -- 172.21.15.57:0/2600370057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6310c0 msgr2=0x55acda6433d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 --2- 172.21.15.57:0/2600370057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6310c0 0x55acda6433d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc42c004e90 tx=0x7fc42c0195e0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 -- 172.21.15.57:0/2600370057 shutdown_connections 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 --2- 172.21.15.57:0/2600370057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda638d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 --2- 172.21.15.57:0/2600370057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6310c0 0x55acda6433d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 --2- 172.21.15.57:0/2600370057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6319e0 0x55acda63bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 -- 172.21.15.57:0/2600370057 >> 172.21.15.57:0/2600370057 conn(0x55acda5a6260 msgr2=0x55acda6347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 -- 172.21.15.57:0/2600370057 shutdown_connections 2023-10-06T22:15:07.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 -- 172.21.15.57:0/2600370057 wait complete. 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 -- start start 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.937+0000 7fc44c191b80 1 Processor -- start 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6310c0 0x55acda636970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda636eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc44c191b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda635730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc44c191b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda635c70 con 0x55acda63c890 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc44c191b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda635de0 con 0x55acda6319e0 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc44c191b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acda635f50 con 0x55acda6310c0 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc44c191b80 10 client.0 _ll_register_callbacks cb 0x7fc42800f2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc436d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda635730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc436d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda635730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37424/0 (socket says 172.21.15.57:37424) 2023-10-06T22:15:07.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc436d9c700 1 -- 172.21.15.57:0/3127948805 learned_addr learned my addr 172.21.15.57:0/3127948805 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc435d9a700 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda636eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc43659b700 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6310c0 0x55acda636970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.938+0000 7fc436d9c700 1 -- 172.21.15.57:0/3127948805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6310c0 msgr2=0x55acda636970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc436d9c700 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6310c0 0x55acda636970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc436d9c700 1 -- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 msgr2=0x55acda636eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc436d9c700 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda636eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc436d9c700 1 -- 172.21.15.57:0/3127948805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c0099e0 con 0x55acda63c890 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc436d9c700 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda635730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc4280045f0 tx=0x7fc428004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a5ff3c879b8d26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc419ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc419ffb700 1 -- 172.21.15.57:0/3127948805 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42800e040 con 0x55acda63c890 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc42c004390 con 0x55acda63c890 2023-10-06T22:15:07.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc419ffb700 1 -- 172.21.15.57:0/3127948805 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc42801c040 con 0x55acda63c890 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc419ffb700 1 -- 172.21.15.57:0/3127948805 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc428017dc0 con 0x55acda63c890 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc44c191b80 10 client.15591 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.939+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55acda3d6250 con 0x55acda63c890 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc419ffb700 1 -- 172.21.15.57:0/3127948805 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc42c004390 con 0x55acda63c890 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc419ffb700 1 -- 172.21.15.57:0/3127948805 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc428036430 con 0x55acda63c890 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc44c191b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc44c191b80 10 client.15591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc44c191b80 20 client.15591 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda6a0f30 0x55acda6a3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc44c191b80 4 client.15591 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:07.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acda57db30 con 0x55acda6a0f30 2023-10-06T22:15:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.940+0000 7fc43659b700 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda6a0f30 0x55acda6a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc419ffb700 10 client.15591 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:07.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc43659b700 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda6a0f30 0x55acda6a3350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5664c29888eabe1 server_cookie=8fec5aae8cbbe11d in_seq=0 out_seq=0 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc419ffb700 1 -- 172.21.15.57:0/3127948805 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acda57db30 con 0x55acda6a0f30 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc419ffb700 10 client.15591 handle_command_reply: tid=0 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 client.15591 shutdown 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 msgr2=0x55acda635730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda635730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc4280045f0 tx=0x7fc428004ad0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 shutdown_connections 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55acda6a0f30 0x55acda6a3350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55acda63c890 0x55acda635730 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acda6310c0 0x55acda636970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 --2- 172.21.15.57:0/3127948805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acda6319e0 0x55acda636eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc419ffb700 0 client.15591 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 >> 172.21.15.57:0/3127948805 conn(0x55acda5a6260 msgr2=0x55acda638d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 shutdown_connections 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.941+0000 7fc44c191b80 1 -- 172.21.15.57:0/3127948805 wait complete. 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc44c191b80 20 client.15591 trim_cache size 0 max 16384 2023-10-06T22:15:07.949 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:07.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc41c0a1710 msgr2=0x7fc41c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc41c0a1710 0x7fc41c0a3bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc430101d50 tx=0x7fc42800b040 comp rx=0 tx=0).stop 2023-10-06T22:15:07.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 msgr2=0x7fc430100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:07.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc430100180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc420000c00 tx=0x7fc420004730 comp rx=0 tx=0).stop 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 shutdown_connections 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc41c0a1710 0x7fc41c0a3bd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43010b410 0x7fc4301a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc430101ff0 0x7fc4300ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 --2- 172.21.15.57:0/78440659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc430102910 0x7fc430100180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 >> 172.21.15.57:0/78440659 conn(0x7fc4300fb830 msgr2=0x7fc43010d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 shutdown_connections 2023-10-06T22:15:07.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:07.942+0000 7fc4387ff700 1 -- 172.21.15.57:0/78440659 wait complete. 2023-10-06T22:15:07.960 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:07 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2230596970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:07.963 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T22:15:08.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:07 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2230596970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:08.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/1303185398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf34101a60 msgr2=0x7fbf34105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/1303185398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf34101a60 0x7fbf34105aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009990 tx=0x7fbf24009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/1303185398 shutdown_connections 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/1303185398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341061d0 0x7fbf3410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/1303185398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf341010b0 0x7fbf34101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/1303185398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf34101a60 0x7fbf34105aa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/1303185398 >> 172.21.15.57:0/1303185398 conn(0x7fbf340fc950 msgr2=0x7fbf340fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/1303185398 shutdown_connections 2023-10-06T22:15:08.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.167+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/1303185398 wait complete. 2023-10-06T22:15:08.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.168+0000 7fbf3b7ff700 1 Processor -- start 2023-10-06T22:15:08.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.168+0000 7fbf3b7ff700 1 -- start start 2023-10-06T22:15:08.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3b7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341010b0 0x7fbf34196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3b7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf341061d0 0x7fbf341974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3419b940 0x7fbf3419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3b7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3419c3c0 con 0x7fbf341010b0 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3419c530 con 0x7fbf3419b940 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3b7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3419c6f0 con 0x7fbf341061d0 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341010b0 0x7fbf34196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341010b0 0x7fbf34196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37440/0 (socket says 172.21.15.57:37440) 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 -- 172.21.15.57:0/2103673669 learned_addr learned my addr 172.21.15.57:0/2103673669 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf39d9c700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3419b940 0x7fbf3419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf341061d0 0x7fbf341974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 -- 172.21.15.57:0/2103673669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf341061d0 msgr2=0x7fbf341974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf341061d0 0x7fbf341974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 -- 172.21.15.57:0/2103673669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3419b940 msgr2=0x7fbf3419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3419b940 0x7fbf3419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf3959b700 1 -- 172.21.15.57:0/2103673669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30009520 con 0x7fbf341010b0 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.169+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf341061d0 0x7fbf341974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.170+0000 7fbf3959b700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341010b0 0x7fbf34196fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000eb30 tx=0x7fbf3000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.170+0000 7fbf2a7fc700 1 -- 172.21.15.57:0/2103673669 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30009880 con 0x7fbf341010b0 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.170+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf240095f0 con 0x7fbf341010b0 2023-10-06T22:15:08.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.170+0000 7fbf2a7fc700 1 -- 172.21.15.57:0/2103673669 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf300044e0 con 0x7fbf341010b0 2023-10-06T22:15:08.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.170+0000 7fbf2a7fc700 1 -- 172.21.15.57:0/2103673669 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30003e90 con 0x7fbf341010b0 2023-10-06T22:15:08.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.170+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf341a3ff0 con 0x7fbf341010b0 2023-10-06T22:15:08.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.174+0000 7fbf2a7fc700 1 -- 172.21.15.57:0/2103673669 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fbf30004000 con 0x7fbf341010b0 2023-10-06T22:15:08.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.175+0000 7fbf4f0b6b80 1 Processor -- start 2023-10-06T22:15:08.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.175+0000 7fbf4f0b6b80 1 -- start start 2023-10-06T22:15:08.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382e20c0 0x5599382e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382ed890 0x5599382efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993824b460 con 0x5599382ed890 2023-10-06T22:15:08.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993823cc20 con 0x5599382e29e0 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993823a540 con 0x5599382e20c0 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf2a7fc700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fbf200a1710 0x7fbf200a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf3959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf38d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382ed890 0x5599382efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf39d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382e20c0 0x5599382e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf38d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382ed890 0x5599382efc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37446/0 (socket says 172.21.15.57:37446) 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf38d9a700 1 -- 172.21.15.57:0/307879187 learned_addr learned my addr 172.21.15.57:0/307879187 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf2a7fc700 1 -- 172.21.15.57:0/2103673669 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fbf30014070 con 0x7fbf341010b0 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.176+0000 7fbf39d9c700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fbf200a1710 0x7fbf200a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf3959b700 1 -- 172.21.15.57:0/307879187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382e20c0 msgr2=0x5599382e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf3959b700 1 --2- 172.21.15.57:0/307879187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382e20c0 0x5599382e24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf3959b700 1 -- 172.21.15.57:0/307879187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382ed890 msgr2=0x5599382efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf3959b700 1 --2- 172.21.15.57:0/307879187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382ed890 0x5599382efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf3959b700 1 -- 172.21.15.57:0/307879187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559938060de0 con 0x5599382e29e0 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf3959b700 1 --2- 172.21.15.57:0/307879187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382ed350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf300cadf0 tx=0x7fbf3008fd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2538380a8717857 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf39d9c700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fbf200a1710 0x7fbf200a3bd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009ec0 tx=0x7fbf2c00b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf1f7fe700 1 -- 172.21.15.57:0/307879187 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30099f10 con 0x5599382e29e0 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf1f7fe700 1 -- 172.21.15.57:0/307879187 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf30085040 con 0x5599382e29e0 2023-10-06T22:15:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.177+0000 7fbf1f7fe700 1 -- 172.21.15.57:0/307879187 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3009e770 con 0x5599382e29e0 2023-10-06T22:15:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/307879187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 msgr2=0x5599382ed350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/307879187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382ed350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf300cadf0 tx=0x7fbf3008fd20 comp rx=0 tx=0).stop 2023-10-06T22:15:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/307879187 shutdown_connections 2023-10-06T22:15:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/307879187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382ed890 0x5599382efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/307879187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382e20c0 0x5599382e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/307879187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382ed350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/307879187 >> 172.21.15.57:0/307879187 conn(0x559938257260 msgr2=0x5599382de450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/307879187 shutdown_connections 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.178+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/307879187 wait complete. 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.179+0000 7fbf4f0b6b80 1 -- start start 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.179+0000 7fbf4f0b6b80 1 Processor -- start 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559938235e30 con 0x5599382e29e0 2023-10-06T22:15:08.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599381d2710 con 0x5599382e20c0 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599381d23b0 con 0x5599382ed890 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf4f0b6b80 10 client.0 _ll_register_callbacks cb 0x55993802f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37454/0 (socket says 172.21.15.57:37454) 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 -- 172.21.15.57:0/1100055535 learned_addr learned my addr 172.21.15.57:0/1100055535 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf39d9c700 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf3959b700 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 -- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 msgr2=0x5599382eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 -- 172.21.15.57:0/1100055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 msgr2=0x5599382ea640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382ea640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.180+0000 7fbf38d9a700 1 -- 172.21.15.57:0/1100055535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf341a3ff0 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eab80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ba80 tx=0x7fbf2400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe8bce349f286bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf1cff9700 1 -- 172.21.15.57:0/1100055535 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400f930 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599382d5f10 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf1cff9700 1 -- 172.21.15.57:0/1100055535 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fbf2400fa90 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf1cff9700 1 -- 172.21.15.57:0/1100055535 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf2400fc50 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf4f0b6b80 10 client.15603 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559938049c60 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.181+0000 7fbf1cff9700 1 -- 172.21.15.57:0/1100055535 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599382d5f10 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf1cff9700 1 -- 172.21.15.57:0/1100055535 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf240173f0 con 0x5599382e29e0 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf4f0b6b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf4f0b6b80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf4f0b6b80 20 client.15603 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5599382fd340 0x5599382ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf4f0b6b80 4 client.15603 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55993822ee10 con 0x5599382fd340 2023-10-06T22:15:08.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.182+0000 7fbf3959b700 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5599382fd340 0x5599382ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.183+0000 7fbf3959b700 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5599382fd340 0x5599382ff760 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e38aa463b9b029 server_cookie=276e8c2777ad65aa in_seq=0 out_seq=0 2023-10-06T22:15:08.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.183+0000 7fbf1cff9700 10 client.15603 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:08.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.183+0000 7fbf1cff9700 1 -- 172.21.15.57:0/1100055535 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55993822ee10 con 0x5599382fd340 2023-10-06T22:15:08.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.183+0000 7fbf1cff9700 10 client.15603 handle_command_reply: tid=0 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 client.15603 shutdown 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 msgr2=0x5599382eab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eab80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ba80 tx=0x7fbf2400bab0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 shutdown_connections 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5599382fd340 0x5599382ff760 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382ea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/1100055535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eab80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf1cff9700 0 client.15603 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 >> 172.21.15.57:0/1100055535 conn(0x559938257260 msgr2=0x5599382ddfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 shutdown_connections 2023-10-06T22:15:08.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/1100055535 wait complete. 2023-10-06T22:15:08.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.184+0000 7fbf4f0b6b80 20 client.15603 trim_cache size 0 max 16384 2023-10-06T22:15:08.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 Processor -- start 2023-10-06T22:15:08.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 -- start start 2023-10-06T22:15:08.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382e9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599380b6920 con 0x5599382e29e0 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599382e8c00 con 0x5599382e20c0 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599382e64d0 con 0x5599382ed890 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf38d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf38d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37464/0 (socket says 172.21.15.57:37464) 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf38d9a700 1 -- 172.21.15.57:0/2529903336 learned_addr learned my addr 172.21.15.57:0/2529903336 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.198+0000 7fbf3959b700 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf39d9c700 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf38d9a700 1 -- 172.21.15.57:0/2529903336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 msgr2=0x5599382e9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382e9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf38d9a700 1 -- 172.21.15.57:0/2529903336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 msgr2=0x5599382e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf38d9a700 1 -- 172.21.15.57:0/2529903336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559938087250 con 0x5599382e29e0 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf3959b700 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eaef0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbf240098e0 tx=0x7fbf24005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcc1e1670678c67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf1ffff700 1 -- 172.21.15.57:0/2529903336 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf240173f0 con 0x5599382e29e0 2023-10-06T22:15:08.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf1ffff700 1 -- 172.21.15.57:0/2529903336 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2401e040 con 0x5599382e29e0 2023-10-06T22:15:08.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.199+0000 7fbf1ffff700 1 -- 172.21.15.57:0/2529903336 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf240246e0 con 0x5599382e29e0 2023-10-06T22:15:08.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/2529903336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 msgr2=0x5599382eaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eaef0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbf240098e0 tx=0x7fbf24005820 comp rx=0 tx=0).stop 2023-10-06T22:15:08.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/2529903336 shutdown_connections 2023-10-06T22:15:08.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x5599382e9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e20c0 0x5599382e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/2529903336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e29e0 0x5599382eaef0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/2529903336 >> 172.21.15.57:0/2529903336 conn(0x559938257260 msgr2=0x5599382e8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/2529903336 shutdown_connections 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/2529903336 wait complete. 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.200+0000 7fbf4f0b6b80 1 -- start start 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 1 Processor -- start 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e20c0 0x5599382f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x559938350190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599382f2bc0 con 0x5599382e20c0 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599383506d0 con 0x5599382e29e0 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559938350810 con 0x5599382ed890 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf4f0b6b80 10 client.0 _ll_register_callbacks cb 0x7fbf24009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf3959b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e20c0 0x5599382f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf3959b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e20c0 0x5599382f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37480/0 (socket says 172.21.15.57:37480) 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf3959b700 1 -- 172.21.15.57:0/3789269661 learned_addr learned my addr 172.21.15.57:0/3789269661 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf39d9c700 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x559938350190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.201+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf3959b700 1 -- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 msgr2=0x559938350190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf3959b700 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x559938350190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf3959b700 1 -- 172.21.15.57:0/3789269661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 msgr2=0x5599382f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:08.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf3959b700 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf3959b700 1 -- 172.21.15.57:0/3789269661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559938049c60 con 0x5599382e20c0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf3959b700 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e20c0 0x5599382f1da0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000eb00 tx=0x7fbf30003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0c563477b0844b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf1cff9700 1 -- 172.21.15.57:0/3789269661 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3009a5b0 con 0x5599382e20c0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf1cff9700 1 -- 172.21.15.57:0/3789269661 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fbf3009acb0 con 0x5599382e20c0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf30085040 con 0x5599382e20c0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf1cff9700 1 -- 172.21.15.57:0/3789269661 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf3000cc90 con 0x5599382e20c0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf4f0b6b80 10 client.15609 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.202+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55993805f2b0 con 0x5599382e20c0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf1cff9700 1 -- 172.21.15.57:0/3789269661 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf30085040 con 0x5599382e20c0 2023-10-06T22:15:08.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf1cff9700 1 -- 172.21.15.57:0/3789269661 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf3009a810 con 0x5599382e20c0 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf4f0b6b80 10 client.15609 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf4f0b6b80 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf4f0b6b80 20 client.15609 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559938351bb0 0x559938353fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf4f0b6b80 4 client.15609 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55993822eb30 con 0x559938351bb0 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.203+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559938351bb0 0x559938353fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf38d9a700 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559938351bb0 0x559938353fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594df397b4309424 server_cookie=b19f02b9b65e7981 in_seq=0 out_seq=0 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf1cff9700 10 client.15609 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf1cff9700 1 -- 172.21.15.57:0/3789269661 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55993822eb30 con 0x559938351bb0 2023-10-06T22:15:08.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf1cff9700 10 client.15609 handle_command_reply: tid=0 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 client.15609 shutdown 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e20c0 msgr2=0x5599382f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e20c0 0x5599382f1da0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000eb00 tx=0x7fbf30003930 comp rx=0 tx=0).stop 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 shutdown_connections 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559938351bb0 0x559938353fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599382ed890 0x559938350190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599382e20c0 0x5599382f1da0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 --2- 172.21.15.57:0/3789269661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599382e29e0 0x5599382f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf1cff9700 0 client.15609 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 >> 172.21.15.57:0/3789269661 conn(0x559938257260 msgr2=0x5599382eae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 shutdown_connections 2023-10-06T22:15:08.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.204+0000 7fbf4f0b6b80 1 -- 172.21.15.57:0/3789269661 wait complete. 2023-10-06T22:15:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf4f0b6b80 20 client.15609 trim_cache size 0 max 16384 2023-10-06T22:15:08.213 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:08.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fbf200a1710 msgr2=0x7fbf200a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fbf200a1710 0x7fbf200a3bd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009ec0 tx=0x7fbf2c00b040 comp rx=0 tx=0).stop 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341010b0 msgr2=0x7fbf34196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341010b0 0x7fbf34196fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000eb30 tx=0x7fbf3000ee40 comp rx=0 tx=0).stop 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 shutdown_connections 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fbf200a1710 0x7fbf200a3bd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3419b940 0x7fbf3419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf341010b0 0x7fbf34196fa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 --2- 172.21.15.57:0/2103673669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf341061d0 0x7fbf341974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 >> 172.21.15.57:0/2103673669 conn(0x7fbf340fc950 msgr2=0x7fbf340fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:08.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 shutdown_connections 2023-10-06T22:15:08.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:08.205+0000 7fbf3b7ff700 1 -- 172.21.15.57:0/2103673669 wait complete. 2023-10-06T22:15:08.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:07 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2230596970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:08 smithi190 ceph-mon[134891]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-10-06T22:15:09.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:08 smithi057 ceph-mon[98492]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-10-06T22:15:09.226 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T22:15:09.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T22:15:09.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:08 smithi156 ceph-mon[134358]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-10-06T22:15:09.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 -- 172.21.15.57:0/2608997691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c24101a60 msgr2=0x7f7c24105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2608997691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c24101a60 0x7f7c24105aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0098f0 tx=0x7f7c0c009c00 comp rx=0 tx=0).stop 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 -- 172.21.15.57:0/2608997691 shutdown_connections 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2608997691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c241061d0 0x7f7c2410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2608997691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241010b0 0x7f7c24101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2608997691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c24101a60 0x7f7c24105aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 -- 172.21.15.57:0/2608997691 >> 172.21.15.57:0/2608997691 conn(0x7f7c240fc970 msgr2=0x7f7c240fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 -- 172.21.15.57:0/2608997691 shutdown_connections 2023-10-06T22:15:09.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.444+0000 7f7c29dff700 1 -- 172.21.15.57:0/2608997691 wait complete. 2023-10-06T22:15:09.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.445+0000 7f7c29dff700 1 Processor -- start 2023-10-06T22:15:09.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.445+0000 7f7c29dff700 1 -- start start 2023-10-06T22:15:09.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c29dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c241010b0 0x7f7c24197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c29dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241061d0 0x7f7c241976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c29dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2419bb20 0x7f7c2419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c29dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2419c5a0 con 0x7f7c241061d0 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c29dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2419c710 con 0x7f7c241010b0 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c29dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2419c8d0 con 0x7f7c2419bb20 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241061d0 0x7f7c241976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241061d0 0x7f7c241976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37494/0 (socket says 172.21.15.57:37494) 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 -- 172.21.15.57:0/2916496519 learned_addr learned my addr 172.21.15.57:0/2916496519 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c237fe700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c241010b0 0x7f7c24197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c23fff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2419bb20 0x7f7c2419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 -- 172.21.15.57:0/2916496519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2419bb20 msgr2=0x7f7c2419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2419bb20 0x7f7c2419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 -- 172.21.15.57:0/2916496519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c241010b0 msgr2=0x7f7c24197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c241010b0 0x7f7c24197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.446+0000 7f7c1bfff700 1 -- 172.21.15.57:0/2916496519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c009520 con 0x7f7c241061d0 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.447+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241061d0 0x7f7c241976c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009230 tx=0x7f7c0c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.447+0000 7f7c217fa700 1 -- 172.21.15.57:0/2916496519 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c01c070 con 0x7f7c241061d0 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.447+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c241a3dd0 con 0x7f7c241061d0 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.447+0000 7f7c217fa700 1 -- 172.21.15.57:0/2916496519 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c0c00bba0 con 0x7f7c241061d0 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.447+0000 7f7c217fa700 1 -- 172.21.15.57:0/2916496519 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c004930 con 0x7f7c241061d0 2023-10-06T22:15:09.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.447+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c241a41f0 con 0x7f7c241061d0 2023-10-06T22:15:09.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.450+0000 7f7c217fa700 1 -- 172.21.15.57:0/2916496519 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f7c0c020070 con 0x7f7c241061d0 2023-10-06T22:15:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 Processor -- start 2023-10-06T22:15:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 -- start start 2023-10-06T22:15:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f6364e0 con 0x55dc8f6d8910 2023-10-06T22:15:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f627ca0 con 0x55dc8f6cd140 2023-10-06T22:15:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.452+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f6255c0 con 0x55dc8f6cda60 2023-10-06T22:15:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c217fa700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7c100a1710 0x7f7c100a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c1bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59444/0 (socket says 172.21.15.57:59444) 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 -- 172.21.15.57:0/4263538014 learned_addr learned my addr 172.21.15.57:0/4263538014 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c1bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6dad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37508/0 (socket says 172.21.15.57:37508) 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c23fff700 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c217fa700 1 -- 172.21.15.57:0/2916496519 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7c0c0ce070 con 0x7f7c241061d0 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c23fff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7c100a1710 0x7f7c100a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 -- 172.21.15.57:0/4263538014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 msgr2=0x55dc8f6cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 -- 172.21.15.57:0/4263538014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 msgr2=0x55dc8f6dad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6dad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c237fe700 1 -- 172.21.15.57:0/4263538014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc8f44be60 con 0x55dc8f6cda60 2023-10-06T22:15:09.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.453+0000 7f7c23fff700 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:09.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.454+0000 7f7c237fe700 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d83d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009d60 tx=0x7f7c14011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8305e809e5a707a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.454+0000 7f7c23fff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7c100a1710 0x7f7c100a3bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009fd0 tx=0x7f7c1c013040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.454+0000 7f7c19ffb700 1 -- 172.21.15.57:0/4263538014 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14019630 con 0x55dc8f6cda60 2023-10-06T22:15:09.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.454+0000 7f7c19ffb700 1 -- 172.21.15.57:0/4263538014 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c14019c50 con 0x55dc8f6cda60 2023-10-06T22:15:09.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.454+0000 7f7c19ffb700 1 -- 172.21.15.57:0/4263538014 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400ebb0 con 0x55dc8f6cda60 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/4263538014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 msgr2=0x55dc8f6d83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d83d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009d60 tx=0x7f7c14011720 comp rx=0 tx=0).stop 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/4263538014 shutdown_connections 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6dad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/4263538014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d83d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/4263538014 >> 172.21.15.57:0/4263538014 conn(0x55dc8f6422e0 msgr2=0x55dc8f6c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/4263538014 shutdown_connections 2023-10-06T22:15:09.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/4263538014 wait complete. 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.455+0000 7f7c3d6f3b80 1 -- start start 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 1 Processor -- start 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f620eb0 con 0x55dc8f6d8910 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f5bd790 con 0x55dc8f6cd140 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f5bd430 con 0x55dc8f6cda60 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c3d6f3b80 10 client.0 _ll_register_callbacks cb 0x55dc8f41a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c23fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6d5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c23fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6d5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37524/0 (socket says 172.21.15.57:37524) 2023-10-06T22:15:09.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c23fff700 1 -- 172.21.15.57:0/1486011926 learned_addr learned my addr 172.21.15.57:0/1486011926 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.456+0000 7f7c237fe700 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c23fff700 1 -- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 msgr2=0x55dc8f6d55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c23fff700 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c23fff700 1 -- 172.21.15.57:0/1486011926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 msgr2=0x55dc8f6d83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c23fff700 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6d83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c23fff700 1 -- 172.21.15.57:0/1486011926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c140099c0 con 0x55dc8f6d8910 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c23fff700 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6d5cf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c019950 tx=0x7f7c1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9e808e8d9ad41e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c02ffd700 1 -- 172.21.15.57:0/1486011926 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c004db0 con 0x55dc8f6d8910 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc8f6c0f90 con 0x55dc8f6d8910 2023-10-06T22:15:09.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.457+0000 7f7c02ffd700 1 -- 172.21.15.57:0/1486011926 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7c1c0078d0 con 0x55dc8f6d8910 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c02ffd700 1 -- 172.21.15.57:0/1486011926 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c1c007a90 con 0x55dc8f6d8910 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 10 client.15621 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dc8f434ce0 con 0x55dc8f6d8910 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c02ffd700 1 -- 172.21.15.57:0/1486011926 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc8f6c0f90 con 0x55dc8f6d8910 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c02ffd700 1 -- 172.21.15.57:0/1486011926 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c1c01f440 con 0x55dc8f6d8910 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 20 client.15621 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f6e8400 0x55dc8f6ea820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 4 client.15621 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.458+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc8f619e90 con 0x55dc8f6e8400 2023-10-06T22:15:09.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.459+0000 7f7c237fe700 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f6e8400 0x55dc8f6ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.459+0000 7f7c237fe700 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f6e8400 0x55dc8f6ea820 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a2afa138e66ade server_cookie=189bd1b4595a5692 in_seq=0 out_seq=0 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.459+0000 7f7c02ffd700 10 client.15621 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c02ffd700 1 -- 172.21.15.57:0/1486011926 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc8f619e90 con 0x55dc8f6e8400 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c02ffd700 10 client.15621 handle_command_reply: tid=0 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 client.15621 shutdown 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 msgr2=0x55dc8f6d5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6d5cf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c019950 tx=0x7f7c1c004ad0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 shutdown_connections 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f6e8400 0x55dc8f6ea820 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6d5cf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6d83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1486011926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cda60 0x55dc8f6d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c02ffd700 0 client.15621 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.460+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 >> 172.21.15.57:0/1486011926 conn(0x55dc8f6422e0 msgr2=0x55dc8f6c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.461+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 shutdown_connections 2023-10-06T22:15:09.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.461+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1486011926 wait complete. 2023-10-06T22:15:09.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.461+0000 7f7c3d6f3b80 20 client.15621 trim_cache size 0 max 16384 2023-10-06T22:15:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.478+0000 7f7c3d6f3b80 1 Processor -- start 2023-10-06T22:15:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c3d6f3b80 1 -- start start 2023-10-06T22:15:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6cda60 0x55dc8f6cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6d8910 0x55dc8f6cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f4a19a0 con 0x55dc8f6cda60 2023-10-06T22:15:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f6df9a0 con 0x55dc8f6cd140 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f6d1480 con 0x55dc8f6d8910 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6cda60 0x55dc8f6cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6cda60 0x55dc8f6cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37530/0 (socket says 172.21.15.57:37530) 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 -- 172.21.15.57:0/1256381621 learned_addr learned my addr 172.21.15.57:0/1256381621 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c237fe700 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c23fff700 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6d8910 0x55dc8f6cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 -- 172.21.15.57:0/1256381621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6d8910 msgr2=0x55dc8f6cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6d8910 0x55dc8f6cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 -- 172.21.15.57:0/1256381621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 msgr2=0x55dc8f6df460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6df460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c1bfff700 1 -- 172.21.15.57:0/1256381621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc8f6c51c0 con 0x55dc8f6cda60 2023-10-06T22:15:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.479+0000 7f7c23fff700 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6d8910 0x55dc8f6cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:09.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.480+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6cda60 0x55dc8f6cf740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0973a0 tx=0x7f7c0c0a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3192f9e40a5c9e0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.480+0000 7f7c1a7fc700 1 -- 172.21.15.57:0/1256381621 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c09aea0 con 0x55dc8f6cda60 2023-10-06T22:15:09.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.480+0000 7f7c1a7fc700 1 -- 172.21.15.57:0/1256381621 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c0c09eff0 con 0x55dc8f6cda60 2023-10-06T22:15:09.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.480+0000 7f7c1a7fc700 1 -- 172.21.15.57:0/1256381621 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c09b100 con 0x55dc8f6cda60 2023-10-06T22:15:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1256381621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6cda60 msgr2=0x55dc8f6cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6cda60 0x55dc8f6cf740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0973a0 tx=0x7f7c0c0a1040 comp rx=0 tx=0).stop 2023-10-06T22:15:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1256381621 shutdown_connections 2023-10-06T22:15:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6d8910 0x55dc8f6cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cd140 0x55dc8f6df460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/1256381621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6cda60 0x55dc8f6cf740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1256381621 >> 172.21.15.57:0/1256381621 conn(0x55dc8f6422e0 msgr2=0x55dc8f6cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1256381621 shutdown_connections 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/1256381621 wait complete. 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 -- start start 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.481+0000 7f7c3d6f3b80 1 Processor -- start 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cd140 0x55dc8f6ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cda60 0x55dc8f6de010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c3d6f3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f73c3c0 con 0x55dc8f6d8910 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f73c530 con 0x55dc8f6cda60 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c3d6f3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8f73c6f0 con 0x55dc8f6cd140 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c3d6f3b80 10 client.0 _ll_register_callbacks cb 0x7f7c1c019580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c23fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c237fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cd140 0x55dc8f6ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c23fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6de550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37544/0 (socket says 172.21.15.57:37544) 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c23fff700 1 -- 172.21.15.57:0/3210817481 learned_addr learned my addr 172.21.15.57:0/3210817481 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.482+0000 7f7c1bfff700 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cda60 0x55dc8f6de010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c23fff700 1 -- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cd140 msgr2=0x55dc8f6ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c23fff700 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cd140 0x55dc8f6ddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c23fff700 1 -- 172.21.15.57:0/3210817481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cda60 msgr2=0x55dc8f6de010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c23fff700 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cda60 0x55dc8f6de010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c23fff700 1 -- 172.21.15.57:0/3210817481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c241a41f0 con 0x55dc8f6d8910 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c237fe700 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cd140 0x55dc8f6ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c23fff700 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6de550 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c01fcf0 tx=0x7f7c1c0117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78172d469314c0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c02ffd700 1 -- 172.21.15.57:0/3210817481 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c007420 con 0x55dc8f6d8910 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c14019c50 con 0x55dc8f6d8910 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c02ffd700 1 -- 172.21.15.57:0/3210817481 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7c1c0078d0 con 0x55dc8f6d8910 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c02ffd700 1 -- 172.21.15.57:0/3210817481 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c1c007a90 con 0x55dc8f6d8910 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c3d6f3b80 10 client.15627 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.483+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dc8f4722d0 con 0x55dc8f6d8910 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c02ffd700 1 -- 172.21.15.57:0/3210817481 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c14019c50 con 0x55dc8f6d8910 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c02ffd700 1 -- 172.21.15.57:0/3210817481 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c1c005820 con 0x55dc8f6d8910 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c3d6f3b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c3d6f3b80 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c3d6f3b80 20 client.15627 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:09.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f73de00 0x55dc8f740220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c3d6f3b80 4 client.15627 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T22:15:09.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dc8f619bb0 con 0x55dc8f73de00 2023-10-06T22:15:09.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.484+0000 7f7c237fe700 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f73de00 0x55dc8f740220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.485+0000 7f7c237fe700 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f73de00 0x55dc8f740220 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e1bd0478fc5302 server_cookie=67f58bb2b3786bfa in_seq=0 out_seq=0 2023-10-06T22:15:09.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.485+0000 7f7c02ffd700 10 client.15627 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c02ffd700 1 -- 172.21.15.57:0/3210817481 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dc8f619bb0 con 0x55dc8f73de00 2023-10-06T22:15:09.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c02ffd700 10 client.15627 handle_command_reply: tid=0 2023-10-06T22:15:09.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 client.15627 shutdown 2023-10-06T22:15:09.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 msgr2=0x55dc8f6de550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6de550 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c01fcf0 tx=0x7f7c1c0117d0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 shutdown_connections 2023-10-06T22:15:09.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc8f73de00 0x55dc8f740220 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc8f6d8910 0x55dc8f6de550 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc8f6cd140 0x55dc8f6ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c02ffd700 0 client.15627 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 --2- 172.21.15.57:0/3210817481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc8f6cda60 0x55dc8f6de010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.486+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 >> 172.21.15.57:0/3210817481 conn(0x55dc8f6422e0 msgr2=0x55dc8f6cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 shutdown_connections 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c3d6f3b80 1 -- 172.21.15.57:0/3210817481 wait complete. 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c3d6f3b80 20 client.15627 trim_cache size 0 max 16384 2023-10-06T22:15:09.493 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-06T22:15:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "9acfa6c8-b80f-4c5c-8043-194e4487cbf5", 2023-10-06T22:15:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-06T22:15:09.494 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:09.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7c100a1710 msgr2=0x7f7c100a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7c100a1710 0x7f7c100a3bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009fd0 tx=0x7f7c1c013040 comp rx=0 tx=0).stop 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241061d0 msgr2=0x7f7c241976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241061d0 0x7f7c241976c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009230 tx=0x7f7c0c00bf40 comp rx=0 tx=0).stop 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 shutdown_connections 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7c100a1710 0x7f7c100a3bd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2419bb20 0x7f7c2419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c241010b0 0x7f7c24197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.487+0000 7f7c29dff700 1 --2- 172.21.15.57:0/2916496519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c241061d0 0x7f7c241976c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.488+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 >> 172.21.15.57:0/2916496519 conn(0x7f7c240fc970 msgr2=0x7f7c240fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.488+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 shutdown_connections 2023-10-06T22:15:09.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.488+0000 7f7c29dff700 1 -- 172.21.15.57:0/2916496519 wait complete. 2023-10-06T22:15:09.508 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:15:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.713+0000 7fb3875ff700 1 -- 172.21.15.57:0/1542134222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3801061d0 msgr2=0x7fb38010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.713+0000 7fb3875ff700 1 --2- 172.21.15.57:0/1542134222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3801061d0 0x7fb38010dd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb37800b0a0 tx=0x7fb37800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.714+0000 7fb3875ff700 1 -- 172.21.15.57:0/1542134222 shutdown_connections 2023-10-06T22:15:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.714+0000 7fb3875ff700 1 --2- 172.21.15.57:0/1542134222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3801061d0 0x7fb38010dd50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.714+0000 7fb3875ff700 1 --2- 172.21.15.57:0/1542134222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3801010b0 0x7fb380101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.714+0000 7fb3875ff700 1 --2- 172.21.15.57:0/1542134222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb380101a60 0x7fb380105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.714+0000 7fb3875ff700 1 -- 172.21.15.57:0/1542134222 >> 172.21.15.57:0/1542134222 conn(0x7fb3800fc950 msgr2=0x7fb3800fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.714+0000 7fb3875ff700 1 -- 172.21.15.57:0/1542134222 shutdown_connections 2023-10-06T22:15:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.714+0000 7fb3875ff700 1 -- 172.21.15.57:0/1542134222 wait complete. 2023-10-06T22:15:09.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 Processor -- start 2023-10-06T22:15:09.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 -- start start 2023-10-06T22:15:09.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3801010b0 0x7fb380199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3801061d0 0x7fb380199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb38019dcd0 0x7fb38019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38019e750 con 0x7fb38019dcd0 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38019e8c0 con 0x7fb3801061d0 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb3875ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38019ea80 con 0x7fb3801010b0 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.715+0000 7fb385b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb38019dcd0 0x7fb38019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb38019dcd0 0x7fb38019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37568/0 (socket says 172.21.15.57:37568) 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 -- 172.21.15.57:0/4089389305 learned_addr learned my addr 172.21.15.57:0/4089389305 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb38539b700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3801010b0 0x7fb380199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb384b9a700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3801061d0 0x7fb380199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 -- 172.21.15.57:0/4089389305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3801010b0 msgr2=0x7fb380199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3801010b0 0x7fb380199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 -- 172.21.15.57:0/4089389305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3801061d0 msgr2=0x7fb380199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3801061d0 0x7fb380199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 -- 172.21.15.57:0/4089389305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378009d00 con 0x7fb38019dcd0 2023-10-06T22:15:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb385b9c700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb38019dcd0 0x7fb38019e150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb37800d010 tx=0x7fb3780079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.716+0000 7fb3767fc700 1 -- 172.21.15.57:0/4089389305 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800e040 con 0x7fb38019dcd0 2023-10-06T22:15:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.717+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3801a5f80 con 0x7fb38019dcd0 2023-10-06T22:15:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.717+0000 7fb3767fc700 1 -- 172.21.15.57:0/4089389305 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37800bde0 con 0x7fb38019dcd0 2023-10-06T22:15:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.717+0000 7fb3767fc700 1 -- 172.21.15.57:0/4089389305 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37801b6b0 con 0x7fb38019dcd0 2023-10-06T22:15:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.717+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3801a6400 con 0x7fb38019dcd0 2023-10-06T22:15:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.720+0000 7fb3767fc700 1 -- 172.21.15.57:0/4089389305 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fb37801d480 con 0x7fb38019dcd0 2023-10-06T22:15:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.721+0000 7fb39afdbb80 1 Processor -- start 2023-10-06T22:15:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.721+0000 7fb39afdbb80 1 -- start start 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.721+0000 7fb3767fc700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb36c0a1710 0x7fb36c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.721+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.721+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df6f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df6f890 0x55c88df71c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88decd460 con 0x55c88df6f890 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88debec20 con 0x55c88df649e0 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88debc540 con 0x55c88df640c0 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb3767fc700 1 -- 172.21.15.57:0/4089389305 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb3780ceb20 con 0x7fb38019dcd0 2023-10-06T22:15:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb38539b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb385b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df6f890 0x55c88df71c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb384b9a700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb36c0a1710 0x7fb36c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb38539b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59480/0 (socket says 172.21.15.57:59480) 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb385b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df6f890 0x55c88df71c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37576/0 (socket says 172.21.15.57:37576) 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb38539b700 1 -- 172.21.15.57:0/170364954 learned_addr learned my addr 172.21.15.57:0/170364954 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb384b9a700 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df6f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb385b9c700 1 -- 172.21.15.57:0/170364954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 msgr2=0x55c88df644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb385b9c700 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb385b9c700 1 -- 172.21.15.57:0/170364954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 msgr2=0x55c88df6f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb385b9c700 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df6f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb385b9c700 1 -- 172.21.15.57:0/170364954 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c88dce2de0 con 0x55c88df6f890 2023-10-06T22:15:09.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.722+0000 7fb384b9a700 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df6f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:09.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.723+0000 7fb385b9c700 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df6f890 0x55c88df71c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3780cc100 tx=0x7fb37800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5977b9922092fe38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.723+0000 7fb384b9a700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb36c0a1710 0x7fb36c0a3bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb370009fd0 tx=0x7fb370009990 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.723+0000 7fb36b7fe700 1 -- 172.21.15.57:0/170364954 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37809b2b0 con 0x55c88df6f890 2023-10-06T22:15:09.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.723+0000 7fb36b7fe700 1 -- 172.21.15.57:0/170364954 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37809c050 con 0x55c88df6f890 2023-10-06T22:15:09.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.723+0000 7fb36b7fe700 1 -- 172.21.15.57:0/170364954 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378099cf0 con 0x55c88df6f890 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 -- 172.21.15.57:0/170364954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df6f890 msgr2=0x55c88df71c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df6f890 0x55c88df71c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3780cc100 tx=0x7fb37800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 -- 172.21.15.57:0/170364954 shutdown_connections 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df6f890 0x55c88df71c80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/170364954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df6f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 -- 172.21.15.57:0/170364954 >> 172.21.15.57:0/170364954 conn(0x55c88ded9260 msgr2=0x55c88df60450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 -- 172.21.15.57:0/170364954 shutdown_connections 2023-10-06T22:15:09.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 -- 172.21.15.57:0/170364954 wait complete. 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.724+0000 7fb39afdbb80 1 -- start start 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 1 Processor -- start 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df640c0 0x55c88df6fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88deb7e30 con 0x55c88df640c0 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88de54710 con 0x55c88df649e0 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88de543b0 con 0x55c88df6f890 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb39afdbb80 10 client.0 _ll_register_callbacks cb 0x55c88dcb1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb38539b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df640c0 0x55c88df6fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb38539b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df640c0 0x55c88df6fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37592/0 (socket says 172.21.15.57:37592) 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb38539b700 1 -- 172.21.15.57:0/2533784626 learned_addr learned my addr 172.21.15.57:0/2533784626 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.725+0000 7fb384b9a700 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb385b9c700 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb38539b700 1 -- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 msgr2=0x55c88df7acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb38539b700 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb38539b700 1 -- 172.21.15.57:0/2533784626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 msgr2=0x55c88df701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb38539b700 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb38539b700 1 -- 172.21.15.57:0/2533784626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3801a6400 con 0x55c88df640c0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb38539b700 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df640c0 0x55c88df6fc70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0128d0 tx=0x7fb37c012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcd2b7ca6930b7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb368ff9700 1 -- 172.21.15.57:0/2533784626 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c017070 con 0x55c88df640c0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c88df57f10 con 0x55c88df640c0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.726+0000 7fb368ff9700 1 -- 172.21.15.57:0/2533784626 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb37c004050 con 0x55c88df640c0 2023-10-06T22:15:09.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb368ff9700 1 -- 172.21.15.57:0/2533784626 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb37c004210 con 0x55c88df640c0 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 10 client.15639 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c88dccbc60 con 0x55c88df640c0 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb368ff9700 1 -- 172.21.15.57:0/2533784626 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c88df57f10 con 0x55c88df640c0 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb368ff9700 1 -- 172.21.15.57:0/2533784626 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb37c00f5f0 con 0x55c88df640c0 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 20 client.15639 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88df7ef60 0x55c88df81380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 4 client.15639 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.727+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c88deb0e10 con 0x55c88df7ef60 2023-10-06T22:15:09.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.728+0000 7fb384b9a700 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88df7ef60 0x55c88df81380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.728+0000 7fb384b9a700 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88df7ef60 0x55c88df81380 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8177a797275b06fe server_cookie=e5f15e9dfb33f1ac in_seq=0 out_seq=0 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.728+0000 7fb368ff9700 10 client.15639 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb368ff9700 1 -- 172.21.15.57:0/2533784626 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c88deb0e10 con 0x55c88df7ef60 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb368ff9700 10 client.15639 handle_command_reply: tid=0 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 client.15639 shutdown 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df640c0 msgr2=0x55c88df6fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df640c0 0x55c88df6fc70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0128d0 tx=0x7fb37c012be0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 shutdown_connections 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88df7ef60 0x55c88df81380 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df640c0 0x55c88df6fc70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/2533784626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df649e0 0x55c88df701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb368ff9700 0 client.15639 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.729+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 >> 172.21.15.57:0/2533784626 conn(0x55c88ded9260 msgr2=0x55c88df5ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.730+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 shutdown_connections 2023-10-06T22:15:09.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.730+0000 7fb39afdbb80 1 -- 172.21.15.57:0/2533784626 wait complete. 2023-10-06T22:15:09.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.730+0000 7fb39afdbb80 20 client.15639 trim_cache size 0 max 16384 2023-10-06T22:15:09.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.747+0000 7fb39afdbb80 1 Processor -- start 2023-10-06T22:15:09.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.747+0000 7fb39afdbb80 1 -- start start 2023-10-06T22:15:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df640c0 0x55c88df69d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88dd38920 con 0x55c88df649e0 2023-10-06T22:15:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88df7c6c0 con 0x55c88df640c0 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88df70520 con 0x55c88df6f890 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb384b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb384b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df6a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37602/0 (socket says 172.21.15.57:37602) 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb384b9a700 1 -- 172.21.15.57:0/1313973548 learned_addr learned my addr 172.21.15.57:0/1313973548 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb38539b700 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df640c0 0x55c88df69d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.748+0000 7fb385b9c700 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb384b9a700 1 -- 172.21.15.57:0/1313973548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 msgr2=0x55c88df7a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb384b9a700 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb384b9a700 1 -- 172.21.15.57:0/1313973548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df640c0 msgr2=0x55c88df69d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb384b9a700 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df640c0 0x55c88df69d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb384b9a700 1 -- 172.21.15.57:0/1313973548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c88df5c140 con 0x55c88df649e0 2023-10-06T22:15:09.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb38539b700 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df640c0 0x55c88df69d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:09.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb384b9a700 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df6a250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb38006a470 tx=0x7fb370009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2c9d3c255f3bbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb36bfff700 1 -- 172.21.15.57:0/1313973548 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370025040 con 0x55c88df649e0 2023-10-06T22:15:09.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb36bfff700 1 -- 172.21.15.57:0/1313973548 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb370011e40 con 0x55c88df649e0 2023-10-06T22:15:09.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.749+0000 7fb36bfff700 1 -- 172.21.15.57:0/1313973548 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37002b4e0 con 0x55c88df649e0 2023-10-06T22:15:09.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1313973548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 msgr2=0x55c88df6a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df6a250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb38006a470 tx=0x7fb370009260 comp rx=0 tx=0).stop 2023-10-06T22:15:09.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1313973548 shutdown_connections 2023-10-06T22:15:09.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df6f890 0x55c88df7a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df640c0 0x55c88df69d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1313973548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df6a250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1313973548 >> 172.21.15.57:0/1313973548 conn(0x55c88ded9260 msgr2=0x55c88df5ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1313973548 shutdown_connections 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1313973548 wait complete. 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 -- start start 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.750+0000 7fb39afdbb80 1 Processor -- start 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df74e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb39afdbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df6f890 0x55c88df738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88df73fe0 con 0x55c88df649e0 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88dfd2da0 con 0x55c88df6f890 2023-10-06T22:15:09.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb39afdbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c88dfd2ee0 con 0x55c88df640c0 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb39afdbb80 10 client.0 _ll_register_callbacks cb 0x7fb37c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb38539b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df74e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb384b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb38539b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df74e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59504/0 (socket says 172.21.15.57:59504) 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb38539b700 1 -- 172.21.15.57:0/1151561782 learned_addr learned my addr 172.21.15.57:0/1151561782 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.751+0000 7fb385b9c700 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df6f890 0x55c88df738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb38539b700 1 -- 172.21.15.57:0/1151561782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df6f890 msgr2=0x55c88df738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb38539b700 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df6f890 0x55c88df738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb38539b700 1 -- 172.21.15.57:0/1151561782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 msgr2=0x55c88df73190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb38539b700 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df73190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb38539b700 1 -- 172.21.15.57:0/1151561782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37002b970 con 0x55c88df640c0 2023-10-06T22:15:09.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb384b9a700 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df73190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb38539b700 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df74e70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005a80 tx=0x7fb37c00f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200433bbe7b08ad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb368ff9700 1 -- 172.21.15.57:0/1151561782 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c017030 con 0x55c88df640c0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb37809c050 con 0x55c88df640c0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.752+0000 7fb368ff9700 1 -- 172.21.15.57:0/1151561782 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb37c000e40 con 0x55c88df640c0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb368ff9700 1 -- 172.21.15.57:0/1151561782 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb37c004b20 con 0x55c88df640c0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 10 client.25657 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c88dd09250 con 0x55c88df640c0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb368ff9700 1 -- 172.21.15.57:0/1151561782 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37809c050 con 0x55c88df640c0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb368ff9700 1 -- 172.21.15.57:0/1151561782 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb37c026480 con 0x55c88df640c0 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 10 client.25657 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:09.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 20 client.25657 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88dfd4670 0x55c88dfd6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 4 client.25657 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.753+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c88deb0b30 con 0x55c88dfd4670 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb384b9a700 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88dfd4670 0x55c88dfd6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb384b9a700 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88dfd4670 0x55c88dfd6a90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1837a6c17ba15b server_cookie=b9ce8f2568f85d1f in_seq=0 out_seq=0 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb368ff9700 10 client.25657 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb368ff9700 1 -- 172.21.15.57:0/1151561782 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c88deb0b30 con 0x55c88dfd4670 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb368ff9700 10 client.25657 handle_command_reply: tid=0 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb39afdbb80 1 client.25657 shutdown 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 msgr2=0x55c88df74e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.754+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df74e70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005a80 tx=0x7fb37c00f890 comp rx=0 tx=0).stop 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 shutdown_connections 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c88dfd4670 0x55c88dfd6a90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c88df6f890 0x55c88df738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c88df640c0 0x55c88df74e70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 --2- 172.21.15.57:0/1151561782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c88df649e0 0x55c88df73190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb368ff9700 0 client.25657 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 >> 172.21.15.57:0/1151561782 conn(0x55c88ded9260 msgr2=0x55c88df69d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 shutdown_connections 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 1 -- 172.21.15.57:0/1151561782 wait complete. 2023-10-06T22:15:09.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb39afdbb80 20 client.25657 trim_cache size 0 max 16384 2023-10-06T22:15:09.763 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-06T22:15:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-06T22:15:09.763 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb36c0a1710 msgr2=0x7fb36c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb3875ff700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb36c0a1710 0x7fb36c0a3bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb370009fd0 tx=0x7fb370009990 comp rx=0 tx=0).stop 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb38019dcd0 msgr2=0x7fb38019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.755+0000 7fb3875ff700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb38019dcd0 0x7fb38019e150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb37800d010 tx=0x7fb3780079b0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 shutdown_connections 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb36c0a1710 0x7fb36c0a3bd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb38019dcd0 0x7fb38019e150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3801010b0 0x7fb380199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 --2- 172.21.15.57:0/4089389305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3801061d0 0x7fb380199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 >> 172.21.15.57:0/4089389305 conn(0x7fb3800fc950 msgr2=0x7fb3800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:09.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 shutdown_connections 2023-10-06T22:15:09.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:09.756+0000 7fb3875ff700 1 -- 172.21.15.57:0/4089389305 wait complete. 2023-10-06T22:15:09.778 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T22:15:09.779 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:15:09.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:09 smithi057 ceph-mon[98492]: scrub queued for path: / 2023-10-06T22:15:09.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:09 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:09.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:09 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:15:09.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:09 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:09.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:09 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/504708892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 msgr2=0x7fdd38101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/504708892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd38101490 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009990 tx=0x7fdd28009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/504708892 shutdown_connections 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/504708892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 0x7fdd3810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/504708892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd38101490 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/504708892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd38101a60 0x7fdd38105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/504708892 >> 172.21.15.57:0/504708892 conn(0x7fdd380fc8d0 msgr2=0x7fdd380fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/504708892 shutdown_connections 2023-10-06T22:15:10.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.004+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/504708892 wait complete. 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 Processor -- start 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 -- start start 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd380731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 0x7fdd380736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd38073c30 0x7fdd381a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381a5820 con 0x7fdd381010b0 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381a5990 con 0x7fdd381061d0 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.006+0000 7fdd3e1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381a5b50 con 0x7fdd38073c30 2023-10-06T22:15:10.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd380731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd36ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 0x7fdd380736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd380731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37624/0 (socket says 172.21.15.57:37624) 2023-10-06T22:15:10.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd36ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 0x7fdd380736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55844/0 (socket says 172.21.15.57:55844) 2023-10-06T22:15:10.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 -- 172.21.15.57:0/2478003570 learned_addr learned my addr 172.21.15.57:0/2478003570 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd37fff700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd38073c30 0x7fdd381a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 -- 172.21.15.57:0/2478003570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd38073c30 msgr2=0x7fdd381a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd38073c30 0x7fdd381a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 -- 172.21.15.57:0/2478003570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 msgr2=0x7fdd380736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 0x7fdd380736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 -- 172.21.15.57:0/2478003570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280095f0 con 0x7fdd381010b0 2023-10-06T22:15:10.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd36ffd700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 0x7fdd380736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:10.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.007+0000 7fdd377fe700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd380731b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd28005910 tx=0x7fdd28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.008+0000 7fdd34ff9700 1 -- 172.21.15.57:0/2478003570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2801d070 con 0x7fdd381010b0 2023-10-06T22:15:10.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.008+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd381a5dd0 con 0x7fdd381010b0 2023-10-06T22:15:10.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.008+0000 7fdd34ff9700 1 -- 172.21.15.57:0/2478003570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd28022470 con 0x7fdd381010b0 2023-10-06T22:15:10.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.008+0000 7fdd34ff9700 1 -- 172.21.15.57:0/2478003570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28017650 con 0x7fdd381010b0 2023-10-06T22:15:10.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.008+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd381a61a0 con 0x7fdd381010b0 2023-10-06T22:15:10.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.010+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd38067cd0 con 0x7fdd381010b0 2023-10-06T22:15:10.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.011+0000 7fdd34ff9700 1 -- 172.21.15.57:0/2478003570 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fdd280225e0 con 0x7fdd381010b0 2023-10-06T22:15:10.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.013+0000 7fdd34ff9700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd200a1760 0x7fdd200a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.013+0000 7fdd34ff9700 1 -- 172.21.15.57:0/2478003570 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fdd280ce6a0 con 0x7fdd381010b0 2023-10-06T22:15:10.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.013+0000 7fdd36ffd700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd200a1760 0x7fdd200a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.014+0000 7fdd36ffd700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd200a1760 0x7fdd200a3c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd380749a0 tx=0x7fdd2c009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.017+0000 7fdd34ff9700 1 -- 172.21.15.57:0/2478003570 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd28097970 con 0x7fdd381010b0 2023-10-06T22:15:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:09 smithi190 ceph-mon[134891]: scrub queued for path: / 2023-10-06T22:15:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:09 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:09 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:15:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:09 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:10.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:09 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:15:10.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:09 smithi156 ceph-mon[134358]: scrub queued for path: / 2023-10-06T22:15:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:09 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:09 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:15:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:09 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:10.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:09 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:15:10.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.307+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd38101b00 con 0x7fdd381010b0 2023-10-06T22:15:10.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.308+0000 7fdd34ff9700 1 -- 172.21.15.57:0/2478003570 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11369 (secure 0 0 0) 0x7fdd2809af90 con 0x7fdd381010b0 2023-10-06T22:15:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:15:10.310 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:12:50.650753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2797676784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2797676784},{"type":"v1","addr":"172.21.15.156:6835","nonce":2797676784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:15:10.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.310+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd200a1760 msgr2=0x7fdd200a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.310+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd200a1760 0x7fdd200a3c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd380749a0 tx=0x7fdd2c009260 comp rx=0 tx=0).stop 2023-10-06T22:15:10.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 msgr2=0x7fdd380731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd380731b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd28005910 tx=0x7fdd28004730 comp rx=0 tx=0).stop 2023-10-06T22:15:10.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 shutdown_connections 2023-10-06T22:15:10.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd200a1760 0x7fdd200a3c20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd38073c30 0x7fdd381a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd381010b0 0x7fdd380731b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 --2- 172.21.15.57:0/2478003570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd381061d0 0x7fdd380736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 >> 172.21.15.57:0/2478003570 conn(0x7fdd380fc8d0 msgr2=0x7fdd380fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 shutdown_connections 2023-10-06T22:15:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.311+0000 7fdd3e1ff700 1 -- 172.21.15.57:0/2478003570 wait complete. 2023-10-06T22:15:10.314 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 26 2023-10-06T22:15:10.335 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T22:15:10.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 -- 172.21.15.57:0/797286002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad00101ff0 msgr2=0x7fad001023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 --2- 172.21.15.57:0/797286002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad00101ff0 0x7fad001023d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7facf0009990 tx=0x7facf0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 -- 172.21.15.57:0/797286002 shutdown_connections 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 --2- 172.21.15.57:0/797286002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad0010b410 0x7fad0010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 --2- 172.21.15.57:0/797286002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad00101ff0 0x7fad001023d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 --2- 172.21.15.57:0/797286002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00102910 0x7fad0010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 -- 172.21.15.57:0/797286002 >> 172.21.15.57:0/797286002 conn(0x7fad000fb830 msgr2=0x7fad000fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 -- 172.21.15.57:0/797286002 shutdown_connections 2023-10-06T22:15:10.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.549+0000 7fad05fff700 1 -- 172.21.15.57:0/797286002 wait complete. 2023-10-06T22:15:10.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.550+0000 7fad05fff700 1 Processor -- start 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.550+0000 7fad05fff700 1 -- start start 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7fad05fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00101ff0 0x7fad001991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7fad05fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad00102910 0x7fad00199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7fad05fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad0010b410 0x7fad0019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7fad05fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0019e1a0 con 0x7fad00101ff0 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7fad05fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0019e310 con 0x7fad0010b410 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7fad05fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0019e4d0 con 0x7fad00102910 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7facff7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00101ff0 0x7fad001991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7facff7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00101ff0 0x7fad001991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37644/0 (socket says 172.21.15.57:37644) 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7facff7fe700 1 -- 172.21.15.57:0/2434307261 learned_addr learned my addr 172.21.15.57:0/2434307261 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7facfeffd700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad00102910 0x7fad00199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.551+0000 7facfffff700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad0010b410 0x7fad0019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facff7fe700 1 -- 172.21.15.57:0/2434307261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad00102910 msgr2=0x7fad00199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facff7fe700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad00102910 0x7fad00199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facff7fe700 1 -- 172.21.15.57:0/2434307261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad0010b410 msgr2=0x7fad0019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facff7fe700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad0010b410 0x7fad0019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facff7fe700 1 -- 172.21.15.57:0/2434307261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00095f0 con 0x7fad00101ff0 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facfeffd700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad00102910 0x7fad00199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facff7fe700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00101ff0 0x7fad001991c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7facf0005910 tx=0x7facf0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facfcff9700 1 -- 172.21.15.57:0/2434307261 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001d070 con 0x7fad00101ff0 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0019e750 con 0x7fad00101ff0 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facfcff9700 1 -- 172.21.15.57:0/2434307261 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf0022470 con 0x7fad00101ff0 2023-10-06T22:15:10.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.552+0000 7facfcff9700 1 -- 172.21.15.57:0/2434307261 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0017650 con 0x7fad00101ff0 2023-10-06T22:15:10.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.553+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad001a5ff0 con 0x7fad00101ff0 2023-10-06T22:15:10.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.556+0000 7facfcff9700 1 -- 172.21.15.57:0/2434307261 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7facf00225e0 con 0x7fad00101ff0 2023-10-06T22:15:10.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.557+0000 7facfcff9700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7face80a1710 0x7face80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.557+0000 7facfcff9700 1 -- 172.21.15.57:0/2434307261 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7facf00ce6c0 con 0x7fad00101ff0 2023-10-06T22:15:10.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.557+0000 7facfeffd700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7face80a1710 0x7face80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.558+0000 7facfeffd700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7face80a1710 0x7face80a3bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad0019a7d0 tx=0x7facf4009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.559+0000 7fad1994ab80 1 Processor -- start 2023-10-06T22:15:10.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.559+0000 7fad1994ab80 1 -- start start 2023-10-06T22:15:10.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b8094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8099e0 0x560e5b814350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b816c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7fad1994ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b772460 con 0x560e5b8099e0 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7fad1994ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b763c20 con 0x560e5b8090c0 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7fad1994ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b761540 con 0x560e5b814890 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7facfeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8099e0 0x560e5b814350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7facfeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8099e0 0x560e5b814350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37646/0 (socket says 172.21.15.57:37646) 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7facfeffd700 1 -- 172.21.15.57:0/2989516956 learned_addr learned my addr 172.21.15.57:0/2989516956 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7facff7fe700 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b8094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.560+0000 7facfffff700 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b816c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facfeffd700 1 -- 172.21.15.57:0/2989516956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 msgr2=0x560e5b816c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facfeffd700 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b816c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facfeffd700 1 -- 172.21.15.57:0/2989516956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 msgr2=0x560e5b8094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facfeffd700 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b8094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facfeffd700 1 -- 172.21.15.57:0/2989516956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e5b587de0 con 0x560e5b8099e0 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facfffff700 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b816c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facff7fe700 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b8094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7facfeffd700 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8099e0 0x560e5b814350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7facf40043c0 tx=0x7facf4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806fd879dc1d0ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7face5ffb700 1 -- 172.21.15.57:0/2989516956 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4003d10 con 0x560e5b8099e0 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7face5ffb700 1 -- 172.21.15.57:0/2989516956 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf4003e70 con 0x560e5b8099e0 2023-10-06T22:15:10.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.561+0000 7face5ffb700 1 -- 172.21.15.57:0/2989516956 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400e7d0 con 0x560e5b8099e0 2023-10-06T22:15:10.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 -- 172.21.15.57:0/2989516956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8099e0 msgr2=0x560e5b814350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8099e0 0x560e5b814350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7facf40043c0 tx=0x7facf4014000 comp rx=0 tx=0).stop 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 -- 172.21.15.57:0/2989516956 shutdown_connections 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b816c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b8094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2989516956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8099e0 0x560e5b814350 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 -- 172.21.15.57:0/2989516956 >> 172.21.15.57:0/2989516956 conn(0x560e5b77e260 msgr2=0x560e5b805450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 -- 172.21.15.57:0/2989516956 shutdown_connections 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.562+0000 7fad1994ab80 1 -- 172.21.15.57:0/2989516956 wait complete. 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.563+0000 7fad1994ab80 1 -- start start 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.563+0000 7fad1994ab80 1 Processor -- start 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8090c0 0x560e5b8115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b811b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b812070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7fad1994ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b75ce30 con 0x560e5b8090c0 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7fad1994ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b6f9710 con 0x560e5b8099e0 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7fad1994ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b6f93b0 con 0x560e5b814890 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7fad1994ab80 10 client.0 _ll_register_callbacks cb 0x560e5b556800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facff7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8090c0 0x560e5b8115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facff7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8090c0 0x560e5b8115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37652/0 (socket says 172.21.15.57:37652) 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facff7fe700 1 -- 172.21.15.57:0/2507016351 learned_addr learned my addr 172.21.15.57:0/2507016351 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facfeffd700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b811b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facfffff700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b812070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facff7fe700 1 -- 172.21.15.57:0/2507016351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 msgr2=0x560e5b812070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facff7fe700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b812070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facff7fe700 1 -- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 msgr2=0x560e5b811b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.564+0000 7facff7fe700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b811b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7facff7fe700 1 -- 172.21.15.57:0/2507016351 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad001a5ff0 con 0x560e5b8090c0 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7facfffff700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b812070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7facff7fe700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8090c0 0x560e5b8115f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7facf0097b10 tx=0x7facf009bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44db4a2a9e587d03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7facdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7facdeffd700 1 -- 172.21.15.57:0/2507016351 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00a3680 con 0x560e5b8090c0 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7facdeffd700 1 -- 172.21.15.57:0/2507016351 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7facf009bd10 con 0x560e5b8090c0 2023-10-06T22:15:10.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e5b7fcf10 con 0x560e5b8090c0 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7facdeffd700 1 -- 172.21.15.57:0/2507016351 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf009b610 con 0x560e5b8090c0 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7fad1994ab80 10 client.15663 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.565+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7facf4017070 con 0x560e5b8090c0 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7facdeffd700 1 -- 172.21.15.57:0/2507016351 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e5b7fcf10 con 0x560e5b8090c0 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7facdeffd700 1 -- 172.21.15.57:0/2507016351 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7facf0097400 con 0x560e5b8090c0 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7fad1994ab80 10 client.15663 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7fad1994ab80 10 client.15663 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7fad1994ab80 20 client.15663 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b823f90 0x560e5b8263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7fad1994ab80 4 client.15663 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:10.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e5b755e10 con 0x560e5b823f90 2023-10-06T22:15:10.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.566+0000 7facfeffd700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b823f90 0x560e5b8263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.567+0000 7facfeffd700 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b823f90 0x560e5b8263b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70c8ebc56abbf386 server_cookie=d32039af04d8a2a3 in_seq=0 out_seq=0 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.567+0000 7facdeffd700 10 client.15663 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.567+0000 7facdeffd700 1 -- 172.21.15.57:0/2507016351 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e5b755e10 con 0x560e5b823f90 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.567+0000 7facdeffd700 10 client.15663 handle_command_reply: tid=0 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 client.15663 shutdown 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8090c0 msgr2=0x560e5b8115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8090c0 0x560e5b8115f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7facf0097b10 tx=0x7facf009bb50 comp rx=0 tx=0).stop 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 shutdown_connections 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b823f90 0x560e5b8263b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b814890 0x560e5b812070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b8090c0 0x560e5b8115f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7facdeffd700 0 client.15663 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:10.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 --2- 172.21.15.57:0/2507016351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b811b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 >> 172.21.15.57:0/2507016351 conn(0x560e5b77e260 msgr2=0x560e5b804fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 shutdown_connections 2023-10-06T22:15:10.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 1 -- 172.21.15.57:0/2507016351 wait complete. 2023-10-06T22:15:10.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.568+0000 7fad1994ab80 20 client.15663 trim_cache size 0 max 16384 2023-10-06T22:15:10.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 Processor -- start 2023-10-06T22:15:10.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 -- start start 2023-10-06T22:15:10.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b811cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8099e0 0x560e5b810880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b810dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b5dd920 con 0x560e5b814890 2023-10-06T22:15:10.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b80a020 con 0x560e5b8090c0 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7fad1994ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b811300 con 0x560e5b8099e0 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7facfffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b810dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7facfffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b810dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37662/0 (socket says 172.21.15.57:37662) 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7facfffff700 1 -- 172.21.15.57:0/288537767 learned_addr learned my addr 172.21.15.57:0/288537767 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.582+0000 7facff7fe700 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b811cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facfeffd700 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8099e0 0x560e5b810880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facfffff700 1 -- 172.21.15.57:0/288537767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8099e0 msgr2=0x560e5b810880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facfffff700 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8099e0 0x560e5b810880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facfffff700 1 -- 172.21.15.57:0/288537767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 msgr2=0x560e5b811cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facfffff700 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b811cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facfffff700 1 -- 172.21.15.57:0/288537767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e5b570c60 con 0x560e5b814890 2023-10-06T22:15:10.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facff7fe700 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b811cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7facfffff700 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b810dc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7facf80075e0 tx=0x7facf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f687978ea5d6007 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7face67fc700 1 -- 172.21.15.57:0/288537767 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf800e040 con 0x560e5b814890 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.583+0000 7face67fc700 1 -- 172.21.15.57:0/288537767 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf800f040 con 0x560e5b814890 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7face67fc700 1 -- 172.21.15.57:0/288537767 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf801e9b0 con 0x560e5b814890 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 -- 172.21.15.57:0/288537767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 msgr2=0x560e5b810dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b810dc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7facf80075e0 tx=0x7facf800b040 comp rx=0 tx=0).stop 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 -- 172.21.15.57:0/288537767 shutdown_connections 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b810dc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8090c0 0x560e5b811cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 --2- 172.21.15.57:0/288537767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8099e0 0x560e5b810880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 -- 172.21.15.57:0/288537767 >> 172.21.15.57:0/288537767 conn(0x560e5b77e260 msgr2=0x560e5b8115b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 -- 172.21.15.57:0/288537767 shutdown_connections 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.584+0000 7fad1994ab80 1 -- 172.21.15.57:0/288537767 wait complete. 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.585+0000 7fad1994ab80 1 -- start start 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.585+0000 7fad1994ab80 1 Processor -- start 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.585+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8090c0 0x560e5b81b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.585+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b819460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7fad1994ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b8199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7fad1994ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b819ee0 con 0x560e5b814890 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7fad1994ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b81a050 con 0x560e5b8099e0 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7fad1994ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5b81a210 con 0x560e5b8090c0 2023-10-06T22:15:10.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7fad1994ab80 10 client.0 _ll_register_callbacks cb 0x7facf009b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facfffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b8199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8090c0 0x560e5b81b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8090c0 0x560e5b81b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59592/0 (socket says 172.21.15.57:59592) 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 -- 172.21.15.57:0/1907523360 learned_addr learned my addr 172.21.15.57:0/1907523360 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facfeffd700 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b819460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 -- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 msgr2=0x560e5b819460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b819460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 -- 172.21.15.57:0/1907523360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 msgr2=0x560e5b8199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b8199a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.586+0000 7facff7fe700 1 -- 172.21.15.57:0/1907523360 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf4017070 con 0x560e5b8090c0 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7facff7fe700 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8090c0 0x560e5b81b100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facf009b310 tx=0x7facf0005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12063cb666df4465 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7facdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:10.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7facdeffd700 1 -- 172.21.15.57:0/1907523360 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf009b150 con 0x560e5b8090c0 2023-10-06T22:15:10.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facf4003e70 con 0x560e5b8090c0 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7facdeffd700 1 -- 172.21.15.57:0/1907523360 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7facf009bd10 con 0x560e5b8090c0 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7facdeffd700 1 -- 172.21.15.57:0/1907523360 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf009b610 con 0x560e5b8090c0 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7fad1994ab80 10 client.25681 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7facf8009230 con 0x560e5b8090c0 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.587+0000 7facdeffd700 1 -- 172.21.15.57:0/1907523360 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf4003e70 con 0x560e5b8090c0 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7facdeffd700 1 -- 172.21.15.57:0/1907523360 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7facf00a3b60 con 0x560e5b8090c0 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7fad1994ab80 10 client.25681 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7fad1994ab80 10 client.25681 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7fad1994ab80 20 client.25681 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:10.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7fad1994ab80 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b878d30 0x560e5b87b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7fad1994ab80 4 client.25681 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e5b755b30 con 0x560e5b878d30 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.588+0000 7facfeffd700 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b878d30 0x560e5b87b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7facfeffd700 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b878d30 0x560e5b87b120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b354b2d22dffebd7 server_cookie=fc48fa00450a5a90 in_seq=0 out_seq=0 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7facdeffd700 10 client.25681 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7facdeffd700 1 -- 172.21.15.57:0/1907523360 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e5b755b30 con 0x560e5b878d30 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7facdeffd700 10 client.25681 handle_command_reply: tid=0 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 client.25681 shutdown 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8090c0 msgr2=0x560e5b81b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8090c0 0x560e5b81b100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facf009b310 tx=0x7facf0005c00 comp rx=0 tx=0).stop 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 shutdown_connections 2023-10-06T22:15:10.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x560e5b878d30 0x560e5b87b120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e5b814890 0x560e5b8199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e5b8090c0 0x560e5b81b100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7facdeffd700 0 client.25681 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 --2- 172.21.15.57:0/1907523360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e5b8099e0 0x560e5b819460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 >> 172.21.15.57:0/1907523360 conn(0x560e5b77e260 msgr2=0x560e5b811db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 shutdown_connections 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 1 -- 172.21.15.57:0/1907523360 wait complete. 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.589+0000 7fad1994ab80 20 client.25681 trim_cache size 0 max 16384 2023-10-06T22:15:10.596 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:10.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7face80a1710 msgr2=0x7face80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7face80a1710 0x7face80a3bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad0019a7d0 tx=0x7facf4009260 comp rx=0 tx=0).stop 2023-10-06T22:15:10.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00101ff0 msgr2=0x7fad001991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00101ff0 0x7fad001991c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7facf0005910 tx=0x7facf0005720 comp rx=0 tx=0).stop 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 shutdown_connections 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7face80a1710 0x7face80a3bd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad0010b410 0x7fad0019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad00101ff0 0x7fad001991c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 --2- 172.21.15.57:0/2434307261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad00102910 0x7fad00199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 >> 172.21.15.57:0/2434307261 conn(0x7fad000fb830 msgr2=0x7fad0010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 shutdown_connections 2023-10-06T22:15:10.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.590+0000 7fad05fff700 1 -- 172.21.15.57:0/2434307261 wait complete. 2023-10-06T22:15:10.611 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T22:15:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.826+0000 7f569c1ff700 1 -- 172.21.15.57:0/3089312964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694073b70 msgr2=0x7f5694111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.826+0000 7f569c1ff700 1 --2- 172.21.15.57:0/3089312964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694073b70 0x7f5694111910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5684009990 tx=0x7f5684009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.827+0000 7f569c1ff700 1 -- 172.21.15.57:0/3089312964 shutdown_connections 2023-10-06T22:15:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.827+0000 7f569c1ff700 1 --2- 172.21.15.57:0/3089312964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5694111e50 0x7f5694114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.827+0000 7f569c1ff700 1 --2- 172.21.15.57:0/3089312964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 0x7f5694073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.827+0000 7f569c1ff700 1 --2- 172.21.15.57:0/3089312964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694073b70 0x7f5694111910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.827+0000 7f569c1ff700 1 -- 172.21.15.57:0/3089312964 >> 172.21.15.57:0/3089312964 conn(0x7f56940fc950 msgr2=0x7f56940fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.827+0000 7f569c1ff700 1 -- 172.21.15.57:0/3089312964 shutdown_connections 2023-10-06T22:15:10.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.827+0000 7f569c1ff700 1 -- 172.21.15.57:0/3089312964 wait complete. 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.828+0000 7f569c1ff700 1 Processor -- start 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.828+0000 7f569c1ff700 1 -- start start 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.828+0000 7f569c1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 0x7f569419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.828+0000 7f569c1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5694073b70 0x7f569419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f569c1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694111e50 0x7f56941a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f569c1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56941a2640 con 0x7f5694111e50 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f569c1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56941a27b0 con 0x7f5694073b70 2023-10-06T22:15:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f569c1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56941a2970 con 0x7f5694073250 2023-10-06T22:15:10.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 0x7f569419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f569a79c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694111e50 0x7f56941a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f569979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5694073b70 0x7f569419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 0x7f569419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59606/0 (socket says 172.21.15.57:59606) 2023-10-06T22:15:10.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 -- 172.21.15.57:0/408952309 learned_addr learned my addr 172.21.15.57:0/408952309 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 -- 172.21.15.57:0/408952309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5694073b70 msgr2=0x7f569419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5694073b70 0x7f569419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 -- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694111e50 msgr2=0x7f56941a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694111e50 0x7f56941a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.829+0000 7f5699f9b700 1 -- 172.21.15.57:0/408952309 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56840095f0 con 0x7f5694073250 2023-10-06T22:15:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.830+0000 7f5699f9b700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 0x7f569419d660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9a0 tx=0x7f569000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.830+0000 7f568affd700 1 -- 172.21.15.57:0/408952309 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690009790 con 0x7f5694073250 2023-10-06T22:15:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.830+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56941a2c50 con 0x7f5694073250 2023-10-06T22:15:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.830+0000 7f568affd700 1 -- 172.21.15.57:0/408952309 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5690010430 con 0x7f5694073250 2023-10-06T22:15:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.830+0000 7f568affd700 1 -- 172.21.15.57:0/408952309 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690018680 con 0x7f5694073250 2023-10-06T22:15:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.830+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56941aa490 con 0x7f5694073250 2023-10-06T22:15:10.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.834+0000 7f568affd700 1 -- 172.21.15.57:0/408952309 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f56900105a0 con 0x7f5694073250 2023-10-06T22:15:10.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.835+0000 7f56afaabb80 1 Processor -- start 2023-10-06T22:15:10.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.835+0000 7f56afaabb80 1 -- start start 2023-10-06T22:15:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e4a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e4a9e0 0x55f978e55350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e55890 0x55f978e57c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f56afaabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978db3460 con 0x55f978e55890 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f56afaabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978da4c20 con 0x55f978e4a0c0 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f56afaabb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978da2540 con 0x55f978e4a9e0 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f569979a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e55890 0x55f978e57c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f568affd700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f56800a1710 0x7f56800a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f569979a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e55890 0x55f978e57c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37700/0 (socket says 172.21.15.57:37700) 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f569979a700 1 -- 172.21.15.57:0/1027626257 learned_addr learned my addr 172.21.15.57:0/1027626257 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f569a79c700 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e4a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f568affd700 1 -- 172.21.15.57:0/408952309 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f5690014070 con 0x7f5694073250 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.836+0000 7f5699f9b700 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e4a9e0 0x55f978e55350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569a79c700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f56800a1710 0x7f56800a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569979a700 1 -- 172.21.15.57:0/1027626257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e4a9e0 msgr2=0x55f978e55350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569979a700 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e4a9e0 0x55f978e55350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569979a700 1 -- 172.21.15.57:0/1027626257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 msgr2=0x55f978e4a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569979a700 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e4a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569979a700 1 -- 172.21.15.57:0/1027626257 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f978bc8de0 con 0x55f978e55890 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569a79c700 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e4a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569979a700 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e55890 0x55f978e57c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5684015000 tx=0x7f5684005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e99cfe8adff2ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f567ffff700 1 -- 172.21.15.57:0/1027626257 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568401d070 con 0x55f978e55890 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.837+0000 7f569a79c700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f56800a1710 0x7f56800a3bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f568c009ec0 tx=0x7f568c00b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f567ffff700 1 -- 172.21.15.57:0/1027626257 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f568401e040 con 0x55f978e55890 2023-10-06T22:15:10.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f567ffff700 1 -- 172.21.15.57:0/1027626257 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684004a60 con 0x55f978e55890 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 -- 172.21.15.57:0/1027626257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e55890 msgr2=0x55f978e57c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e55890 0x55f978e57c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5684015000 tx=0x7f5684005ea0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 -- 172.21.15.57:0/1027626257 shutdown_connections 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e55890 0x55f978e57c80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e4a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1027626257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e4a9e0 0x55f978e55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 -- 172.21.15.57:0/1027626257 >> 172.21.15.57:0/1027626257 conn(0x55f978dbf260 msgr2=0x55f978e46450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.838+0000 7f56afaabb80 1 -- 172.21.15.57:0/1027626257 shutdown_connections 2023-10-06T22:15:10.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.839+0000 7f56afaabb80 1 -- 172.21.15.57:0/1027626257 wait complete. 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.839+0000 7f56afaabb80 1 -- start start 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.839+0000 7f56afaabb80 1 Processor -- start 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e52500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e52c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f56afaabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978d9de30 con 0x55f978e4a0c0 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f56afaabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978d3a710 con 0x55f978e4a9e0 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f56afaabb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978d3a3b0 con 0x55f978e55890 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f56afaabb80 10 client.0 _ll_register_callbacks cb 0x55f978b97800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:10.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f5699f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f5699f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37706/0 (socket says 172.21.15.57:37706) 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f5699f9b700 1 -- 172.21.15.57:0/1465617198 learned_addr learned my addr 172.21.15.57:0/1465617198 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f569a79c700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e52c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.840+0000 7f569979a700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e52500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f569a79c700 1 -- 172.21.15.57:0/1465617198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 msgr2=0x55f978e52500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f569a79c700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e52500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f569a79c700 1 -- 172.21.15.57:0/1465617198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 msgr2=0x55f978e552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f569a79c700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f569a79c700 1 -- 172.21.15.57:0/1465617198 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5694114bf0 con 0x55f978e55890 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f5699f9b700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f569a79c700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e52c40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f568c018ad0 tx=0x7f568c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10b5180d7fe099e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f567d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f567d7fa700 1 -- 172.21.15.57:0/1465617198 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c00e070 con 0x55f978e55890 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f978e3df10 con 0x55f978e55890 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.841+0000 7f567d7fa700 1 -- 172.21.15.57:0/1465617198 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f568c016910 con 0x55f978e55890 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f567d7fa700 1 -- 172.21.15.57:0/1465617198 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f568c018e90 con 0x55f978e55890 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 10 client.25693 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f978bb1c60 con 0x55f978e55890 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f567d7fa700 1 -- 172.21.15.57:0/1465617198 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f978e3df10 con 0x55f978e55890 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f567d7fa700 1 -- 172.21.15.57:0/1465617198 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f568c005470 con 0x55f978e55890 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 10 client.25693 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 10 client.25693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 20 client.25693 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978e65260 0x55f978e67680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 4 client.25693 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.842+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f978d96e10 con 0x55f978e65260 2023-10-06T22:15:10.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.843+0000 7f5699f9b700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978e65260 0x55f978e67680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.843+0000 7f5699f9b700 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978e65260 0x55f978e67680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21d3c0f822f555af server_cookie=f0d2f2f88f532e7e in_seq=0 out_seq=0 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.843+0000 7f567d7fa700 10 client.25693 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f567d7fa700 1 -- 172.21.15.57:0/1465617198 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f978d96e10 con 0x55f978e65260 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f567d7fa700 10 client.25693 handle_command_reply: tid=0 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 client.25693 shutdown 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 msgr2=0x55f978e52c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e52c40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f568c018ad0 tx=0x7f568c004ad0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 shutdown_connections 2023-10-06T22:15:10.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978e65260 0x55f978e67680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e52c40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f567d7fa700 0 client.25693 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 --2- 172.21.15.57:0/1465617198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e52500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 >> 172.21.15.57:0/1465617198 conn(0x55f978dbf260 msgr2=0x55f978e46040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 shutdown_connections 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.844+0000 7f56afaabb80 1 -- 172.21.15.57:0/1465617198 wait complete. 2023-10-06T22:15:10.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.845+0000 7f56afaabb80 20 client.25693 trim_cache size 0 max 16384 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.859+0000 7f56afaabb80 1 Processor -- start 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f56afaabb80 1 -- start start 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e54e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e4d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f56afaabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978c1e920 con 0x55f978e4a0c0 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f56afaabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978e4e380 con 0x55f978e4a9e0 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f56afaabb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978e53240 con 0x55f978e55890 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f5699f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f5699f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e61570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37716/0 (socket says 172.21.15.57:37716) 2023-10-06T22:15:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f5699f9b700 1 -- 172.21.15.57:0/3860559851 learned_addr learned my addr 172.21.15.57:0/3860559851 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f569a79c700 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e4d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.860+0000 7f569979a700 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e54e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5699f9b700 1 -- 172.21.15.57:0/3860559851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 msgr2=0x55f978e4d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5699f9b700 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e4d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5699f9b700 1 -- 172.21.15.57:0/3860559851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 msgr2=0x55f978e54e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5699f9b700 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e54e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5699f9b700 1 -- 172.21.15.57:0/3860559851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f978bef250 con 0x55f978e4a0c0 2023-10-06T22:15:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5699f9b700 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e61570 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f569009a290 tx=0x7f5690096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d96052bd2056fe8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5688ff9700 1 -- 172.21.15.57:0/3860559851 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690099e40 con 0x55f978e4a0c0 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5688ff9700 1 -- 172.21.15.57:0/3860559851 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56900b0070 con 0x55f978e4a0c0 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.861+0000 7f5688ff9700 1 -- 172.21.15.57:0/3860559851 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690010880 con 0x55f978e4a0c0 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 -- 172.21.15.57:0/3860559851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 msgr2=0x55f978e61570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e61570 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f569009a290 tx=0x7f5690096040 comp rx=0 tx=0).stop 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 -- 172.21.15.57:0/3860559851 shutdown_connections 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e4d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a0c0 0x55f978e61570 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3860559851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a9e0 0x55f978e54e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 -- 172.21.15.57:0/3860559851 >> 172.21.15.57:0/3860559851 conn(0x55f978dbf260 msgr2=0x55f978e6bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 -- 172.21.15.57:0/3860559851 shutdown_connections 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 -- 172.21.15.57:0/3860559851 wait complete. 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.862+0000 7f56afaabb80 1 -- start start 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 1 Processor -- start 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a9e0 0x55f978e58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e59420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978eb9070 con 0x55f978e4a9e0 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978eb91b0 con 0x55f978e4a0c0 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f978eb9320 con 0x55f978e55890 2023-10-06T22:15:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f56afaabb80 10 client.0 _ll_register_callbacks cb 0x7f568c0184a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f569979a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a9e0 0x55f978e58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f569a79c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e59420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f569979a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a9e0 0x55f978e58ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37718/0 (socket says 172.21.15.57:37718) 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f569979a700 1 -- 172.21.15.57:0/3723703664 learned_addr learned my addr 172.21.15.57:0/3723703664 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.863+0000 7f569a79c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e59420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59638/0 (socket says 172.21.15.57:59638) 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f5699f9b700 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f569979a700 1 -- 172.21.15.57:0/3723703664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 msgr2=0x55f978e59420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f569979a700 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e59420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f569979a700 1 -- 172.21.15.57:0/3723703664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 msgr2=0x55f978e589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f569979a700 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f569979a700 1 -- 172.21.15.57:0/3723703664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56941aa490 con 0x55f978e4a9e0 2023-10-06T22:15:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f569979a700 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a9e0 0x55f978e58ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5684004570 tx=0x7f5684005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382bc546a29b188 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f567d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f567d7fa700 1 -- 172.21.15.57:0/3723703664 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568401d040 con 0x55f978e4a9e0 2023-10-06T22:15:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f568401e040 con 0x55f978e4a9e0 2023-10-06T22:15:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.864+0000 7f567d7fa700 1 -- 172.21.15.57:0/3723703664 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f56840055f0 con 0x55f978e4a9e0 2023-10-06T22:15:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.865+0000 7f567d7fa700 1 -- 172.21.15.57:0/3723703664 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5684004810 con 0x55f978e4a9e0 2023-10-06T22:15:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.865+0000 7f56afaabb80 10 client.15687 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.865+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f978bc72b0 con 0x55f978e4a9e0 2023-10-06T22:15:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.865+0000 7f567d7fa700 1 -- 172.21.15.57:0/3723703664 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f568401e040 con 0x55f978e4a9e0 2023-10-06T22:15:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f567d7fa700 1 -- 172.21.15.57:0/3723703664 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f568400fb80 con 0x55f978e4a9e0 2023-10-06T22:15:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f56afaabb80 10 client.15687 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f56afaabb80 10 client.15687 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f56afaabb80 20 client.15687 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978eba5e0 0x55f978ebca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f56afaabb80 4 client.15687 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f978d96b30 con 0x55f978eba5e0 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.866+0000 7f5699f9b700 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978eba5e0 0x55f978ebca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f5699f9b700 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978eba5e0 0x55f978ebca00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e48016c918b32d9f server_cookie=879c6a54bda25092 in_seq=0 out_seq=0 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f567d7fa700 10 client.15687 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f567d7fa700 1 -- 172.21.15.57:0/3723703664 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f978d96b30 con 0x55f978eba5e0 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f567d7fa700 10 client.15687 handle_command_reply: tid=0 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 client.15687 shutdown 2023-10-06T22:15:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a9e0 msgr2=0x55f978e58ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a9e0 0x55f978e58ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5684004570 tx=0x7f5684005270 comp rx=0 tx=0).stop 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 shutdown_connections 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55f978eba5e0 0x55f978ebca00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f978e55890 0x55f978e59420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f978e4a0c0 0x55f978e589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 --2- 172.21.15.57:0/3723703664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f978e4a9e0 0x55f978e58ee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 >> 172.21.15.57:0/3723703664 conn(0x55f978dbf260 msgr2=0x55f978e61570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f567d7fa700 0 client.15687 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 shutdown_connections 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.867+0000 7f56afaabb80 1 -- 172.21.15.57:0/3723703664 wait complete. 2023-10-06T22:15:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f56afaabb80 20 client.15687 trim_cache size 0 max 16384 2023-10-06T22:15:10.876 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f56800a1710 msgr2=0x7f56800a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f56800a1710 0x7f56800a3bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f568c009ec0 tx=0x7f568c00b040 comp rx=0 tx=0).stop 2023-10-06T22:15:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 msgr2=0x7f569419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 0x7f569419d660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9a0 tx=0x7f569000ecb0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 shutdown_connections 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f56800a1710 0x7f56800a3bd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5694111e50 0x7f56941a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5694073250 0x7f569419d660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 --2- 172.21.15.57:0/408952309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5694073b70 0x7f569419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.868+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 >> 172.21.15.57:0/408952309 conn(0x7f56940fc950 msgr2=0x7f56940fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.869+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 shutdown_connections 2023-10-06T22:15:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:10.869+0000 7f569c1ff700 1 -- 172.21.15.57:0/408952309 wait complete. 2023-10-06T22:15:10.887 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:10 smithi057 ceph-mon[98492]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-06T22:15:10.887 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:10 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2478003570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:10.889 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T22:15:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:10 smithi190 ceph-mon[134891]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-06T22:15:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:10 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2478003570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:11.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 -- 172.21.15.57:0/1756024269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102050 msgr2=0x7f2838102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 --2- 172.21.15.57:0/1756024269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102050 0x7f2838102430 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f282c009990 tx=0x7f282c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 -- 172.21.15.57:0/1756024269 shutdown_connections 2023-10-06T22:15:11.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 --2- 172.21.15.57:0/1756024269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f283810b420 0x7f283810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 --2- 172.21.15.57:0/1756024269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102050 0x7f2838102430 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 --2- 172.21.15.57:0/1756024269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2838102970 0x7f283810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 -- 172.21.15.57:0/1756024269 >> 172.21.15.57:0/1756024269 conn(0x7f28380fb830 msgr2=0x7f28380fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.099+0000 7f283edff700 1 -- 172.21.15.57:0/1756024269 shutdown_connections 2023-10-06T22:15:11.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.100+0000 7f283edff700 1 -- 172.21.15.57:0/1756024269 wait complete. 2023-10-06T22:15:11.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 Processor -- start 2023-10-06T22:15:11.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 -- start start 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2838102050 0x7f2838196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102970 0x7f28381974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f283810b420 0x7f283819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819bf80 con 0x7f2838102970 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819c0f0 con 0x7f283810b420 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f283edff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819c2b0 con 0x7f2838102050 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.101+0000 7f2837fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102970 0x7f28381974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f283810b420 0x7f283819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2838102050 0x7f2838196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f2837fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102970 0x7f28381974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37734/0 (socket says 172.21.15.57:37734) 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f283810b420 0x7f283819b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55994/0 (socket says 172.21.15.57:55994) 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f2837fff700 1 -- 172.21.15.57:0/56716692 learned_addr learned my addr 172.21.15.57:0/56716692 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283cb9b700 1 -- 172.21.15.57:0/56716692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f283810b420 msgr2=0x7f283819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283cb9b700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f283810b420 0x7f283819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283cb9b700 1 -- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102970 msgr2=0x7f28381974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283cb9b700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102970 0x7f28381974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283cb9b700 1 -- 172.21.15.57:0/56716692 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2824009520 con 0x7f2838102050 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f2837fff700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102970 0x7f28381974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:11.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.102+0000 7f283cb9b700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2838102050 0x7f2838196fa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f282c009990 tx=0x7f282c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.103+0000 7f2835ffb700 1 -- 172.21.15.57:0/56716692 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c01d070 con 0x7f2838102050 2023-10-06T22:15:11.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.103+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2830009c40 con 0x7f2838102050 2023-10-06T22:15:11.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.103+0000 7f2835ffb700 1 -- 172.21.15.57:0/56716692 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f282c021410 con 0x7f2838102050 2023-10-06T22:15:11.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.103+0000 7f2835ffb700 1 -- 172.21.15.57:0/56716692 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c017650 con 0x7f2838102050 2023-10-06T22:15:11.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.103+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c0095f0 con 0x7f2838102050 2023-10-06T22:15:11.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.108+0000 7f2835ffb700 1 -- 172.21.15.57:0/56716692 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f282c0217a0 con 0x7f2838102050 2023-10-06T22:15:11.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 Processor -- start 2023-10-06T22:15:11.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 -- start start 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699cd38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cde760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699ce1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699c38460 con 0x555699cdeca0 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699c29c20 con 0x555699cd34d0 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.109+0000 7f285275eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699c27540 con 0x555699cd3df0 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699cd38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f2835ffb700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f28280a1710 0x7f28280a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699cd38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55998/0 (socket says 172.21.15.57:55998) 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 -- 172.21.15.57:0/1804446464 learned_addr learned my addr 172.21.15.57:0/1804446464 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283cb9b700 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cde760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f2837fff700 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699ce1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f2835ffb700 1 -- 172.21.15.57:0/56716692 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f282c021e40 con 0x7f2838102050 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f28280a1710 0x7f28280a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 -- 172.21.15.57:0/1804446464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 msgr2=0x555699cde760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cde760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 -- 172.21.15.57:0/1804446464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 msgr2=0x555699ce1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699ce1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283d39c700 1 -- 172.21.15.57:0/1804446464 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555699a4dde0 con 0x555699cd34d0 2023-10-06T22:15:11.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f283cb9b700 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cde760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:11.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.110+0000 7f2837fff700 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699ce1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:11.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.111+0000 7f283d39c700 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699cd38b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2830009b50 tx=0x7f2830018640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d674e14acd15ed26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.111+0000 7f283d39c700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f28280a1710 0x7f28280a3bd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2830009000 tx=0x7f283001d000 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.111+0000 7f2822ffd700 1 -- 172.21.15.57:0/1804446464 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000e040 con 0x555699cd34d0 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.111+0000 7f2822ffd700 1 -- 172.21.15.57:0/1804446464 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f283000f040 con 0x555699cd34d0 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.111+0000 7f2822ffd700 1 -- 172.21.15.57:0/1804446464 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830007500 con 0x555699cd34d0 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 -- 172.21.15.57:0/1804446464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 msgr2=0x555699cd38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699cd38b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2830009b50 tx=0x7f2830018640 comp rx=0 tx=0).stop 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 -- 172.21.15.57:0/1804446464 shutdown_connections 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699ce1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699cd38b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 --2- 172.21.15.57:0/1804446464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cde760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 -- 172.21.15.57:0/1804446464 >> 172.21.15.57:0/1804446464 conn(0x555699c44260 msgr2=0x555699ccb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 -- 172.21.15.57:0/1804446464 shutdown_connections 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.112+0000 7f285275eb80 1 -- 172.21.15.57:0/1804446464 wait complete. 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.113+0000 7f285275eb80 1 -- start start 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.113+0000 7f285275eb80 1 Processor -- start 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f285275eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699cde6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f285275eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd3df0 0x555699cdb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f285275eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cdc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f285275eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699c22e30 con 0x555699cdeca0 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f285275eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699bbf710 con 0x555699cd3df0 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f285275eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699bbf3b0 con 0x555699cd34d0 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f285275eb80 10 client.0 _ll_register_callbacks cb 0x555699a1c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f283cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699cde6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd3df0 0x555699cdb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f283d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cdc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd3df0 0x555699cdb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:56008/0 (socket says 172.21.15.57:56008) 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f283cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699cde6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59688/0 (socket says 172.21.15.57:59688) 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 -- 172.21.15.57:0/29980002 learned_addr learned my addr 172.21.15.57:0/29980002 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 -- 172.21.15.57:0/29980002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 msgr2=0x555699cde6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699cde6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 -- 172.21.15.57:0/29980002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 msgr2=0x555699cdc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cdc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.114+0000 7f2837fff700 1 -- 172.21.15.57:0/29980002 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c0095f0 con 0x555699cd3df0 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f2837fff700 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd3df0 0x555699cdb9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f282400f670 tx=0x7f282400f980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d4d6ff458e5e8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f2817fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f2817fff700 1 -- 172.21.15.57:0/29980002 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824016070 con 0x555699cd3df0 2023-10-06T22:15:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555699cc2f10 con 0x555699cd3df0 2023-10-06T22:15:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f2817fff700 1 -- 172.21.15.57:0/29980002 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f282400e5c0 con 0x555699cd3df0 2023-10-06T22:15:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f2817fff700 1 -- 172.21.15.57:0/29980002 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f282400e7c0 con 0x555699cd3df0 2023-10-06T22:15:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f285275eb80 10 client.25703 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.115+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f2824011440 con 0x555699cd3df0 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f2817fff700 1 -- 172.21.15.57:0/29980002 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555699cc2f10 con 0x555699cd3df0 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f2817fff700 1 -- 172.21.15.57:0/29980002 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2824023940 con 0x555699cd3df0 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f285275eb80 10 client.25703 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f285275eb80 10 client.25703 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f285275eb80 20 client.25703 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f285275eb80 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699cee780 0x555699cf0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f285275eb80 4 client.25703 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555699c1be10 con 0x555699cee780 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.116+0000 7f283cb9b700 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699cee780 0x555699cf0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.117+0000 7f283cb9b700 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699cee780 0x555699cf0ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6d6f5b839e3ee1b server_cookie=f44315176f32ebfb in_seq=0 out_seq=0 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.117+0000 7f2817fff700 10 client.25703 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f2817fff700 1 -- 172.21.15.57:0/29980002 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555699c1be10 con 0x555699cee780 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f2817fff700 10 client.25703 handle_command_reply: tid=0 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 client.25703 shutdown 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd3df0 msgr2=0x555699cdb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd3df0 0x555699cdb9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f282400f670 tx=0x7f282400f980 comp rx=0 tx=0).stop 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 shutdown_connections 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699cee780 0x555699cf0ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cdc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699cde6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 --2- 172.21.15.57:0/29980002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd3df0 0x555699cdb9a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f2817fff700 0 client.25703 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 >> 172.21.15.57:0/29980002 conn(0x555699c44260 msgr2=0x555699ccafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 shutdown_connections 2023-10-06T22:15:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 1 -- 172.21.15.57:0/29980002 wait complete. 2023-10-06T22:15:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.118+0000 7f285275eb80 20 client.25703 trim_cache size 0 max 16384 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 Processor -- start 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 -- start start 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699ce5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cd6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cd7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699aa3920 con 0x555699cdeca0 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699cd3b30 con 0x555699cd34d0 2023-10-06T22:15:11.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.132+0000 7f285275eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699cd4fc0 con 0x555699cd3df0 2023-10-06T22:15:11.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f283d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cd7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f2837fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cd6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f283d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cd7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37754/0 (socket says 172.21.15.57:37754) 2023-10-06T22:15:11.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f283d39c700 1 -- 172.21.15.57:0/3348801245 learned_addr learned my addr 172.21.15.57:0/3348801245 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f283cb9b700 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699ce5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f2837fff700 1 -- 172.21.15.57:0/3348801245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 msgr2=0x555699ce5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f2837fff700 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699ce5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f2837fff700 1 -- 172.21.15.57:0/3348801245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 msgr2=0x555699cd7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f2837fff700 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cd7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f2837fff700 1 -- 172.21.15.57:0/3348801245 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283000e9b0 con 0x555699cd3df0 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.133+0000 7f2837fff700 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cd6d00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f282400f640 tx=0x7f28240092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f75f92ea69d2472a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.134+0000 7f28237fe700 1 -- 172.21.15.57:0/3348801245 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824016040 con 0x555699cd3df0 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.134+0000 7f28237fe700 1 -- 172.21.15.57:0/3348801245 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2824005570 con 0x555699cd3df0 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.134+0000 7f28237fe700 1 -- 172.21.15.57:0/3348801245 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824023a70 con 0x555699cd3df0 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 -- 172.21.15.57:0/3348801245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 msgr2=0x555699cd6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cd6d00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f282400f640 tx=0x7f28240092f0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 -- 172.21.15.57:0/3348801245 shutdown_connections 2023-10-06T22:15:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cdeca0 0x555699cd7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cd34d0 0x555699ce5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 --2- 172.21.15.57:0/3348801245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd3df0 0x555699cd6d00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 -- 172.21.15.57:0/3348801245 >> 172.21.15.57:0/3348801245 conn(0x555699c44260 msgr2=0x555699ce9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 -- 172.21.15.57:0/3348801245 shutdown_connections 2023-10-06T22:15:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 -- 172.21.15.57:0/3348801245 wait complete. 2023-10-06T22:15:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 -- start start 2023-10-06T22:15:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.135+0000 7f285275eb80 1 Processor -- start 2023-10-06T22:15:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f285275eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699ce53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f285275eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cd3df0 0x555699ce3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f285275eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cdeca0 0x555699ce3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f285275eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699ce41f0 con 0x555699cd3df0 2023-10-06T22:15:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f285275eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699ce4330 con 0x555699cdeca0 2023-10-06T22:15:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f285275eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555699ce44f0 con 0x555699cd34d0 2023-10-06T22:15:11.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f285275eb80 10 client.0 _ll_register_callbacks cb 0x7f2824011b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:11.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f2837fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cd3df0 0x555699ce3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f283cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699ce53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f2837fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cd3df0 0x555699ce3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37766/0 (socket says 172.21.15.57:37766) 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.136+0000 7f2837fff700 1 -- 172.21.15.57:0/2602132145 learned_addr learned my addr 172.21.15.57:0/2602132145 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283d39c700 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cdeca0 0x555699ce3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283cb9b700 1 -- 172.21.15.57:0/2602132145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cdeca0 msgr2=0x555699ce3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283cb9b700 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cdeca0 0x555699ce3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283cb9b700 1 -- 172.21.15.57:0/2602132145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cd3df0 msgr2=0x555699ce3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283cb9b700 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cd3df0 0x555699ce3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283cb9b700 1 -- 172.21.15.57:0/2602132145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2824011440 con 0x555699cd34d0 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283d39c700 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cdeca0 0x555699ce3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f283cb9b700 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699ce53c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f282c0cbce0 tx=0x7f282c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adda76b58be7f07c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f2817fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f2817fff700 1 -- 172.21.15.57:0/2602132145 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c09bc30 con 0x555699cd34d0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f2817fff700 1 -- 172.21.15.57:0/2602132145 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f282c09b530 con 0x555699cd34d0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f283000f040 con 0x555699cd34d0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.137+0000 7f2817fff700 1 -- 172.21.15.57:0/2602132145 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f282c09f680 con 0x555699cd34d0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 10 client.25717 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555699a36c60 con 0x555699cd34d0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f2817fff700 1 -- 172.21.15.57:0/2602132145 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f283000f040 con 0x555699cd34d0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f2817fff700 1 -- 172.21.15.57:0/2602132145 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f282c0a3430 con 0x555699cd34d0 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 10 client.25717 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 10 client.25717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 20 client.25717 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:11.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699d43680 0x555699d45aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 4 client.25717 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555699c1bb30 con 0x555699d43680 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.138+0000 7f2837fff700 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699d43680 0x555699d45aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f2837fff700 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699d43680 0x555699d45aa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59d1526813b180a server_cookie=fed7f6c8adba08a5 in_seq=0 out_seq=0 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f2817fff700 10 client.25717 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f2817fff700 1 -- 172.21.15.57:0/2602132145 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555699c1bb30 con 0x555699d43680 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f2817fff700 10 client.25717 handle_command_reply: tid=0 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 client.25717 shutdown 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 msgr2=0x555699ce53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699ce53c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f282c0cbce0 tx=0x7f282c0047c0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 shutdown_connections 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x555699d43680 0x555699d45aa0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555699cdeca0 0x555699ce3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555699cd34d0 0x555699ce53c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f2817fff700 0 client.25717 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 --2- 172.21.15.57:0/2602132145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555699cd3df0 0x555699ce3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.139+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 >> 172.21.15.57:0/2602132145 conn(0x555699c44260 msgr2=0x555699ccaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 shutdown_connections 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f285275eb80 1 -- 172.21.15.57:0/2602132145 wait complete. 2023-10-06T22:15:11.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f285275eb80 20 client.25717 trim_cache size 0 max 16384 2023-10-06T22:15:11.148 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f28280a1710 msgr2=0x7f28280a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f28280a1710 0x7f28280a3bd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2830009000 tx=0x7f283001d000 comp rx=0 tx=0).stop 2023-10-06T22:15:11.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2838102050 msgr2=0x7f2838196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2838102050 0x7f2838196fa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f282c009990 tx=0x7f282c009580 comp rx=0 tx=0).stop 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 shutdown_connections 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f28280a1710 0x7f28280a3bd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f283810b420 0x7f283819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2838102050 0x7f2838196fa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 --2- 172.21.15.57:0/56716692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2838102970 0x7f28381974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.140+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 >> 172.21.15.57:0/56716692 conn(0x7f28380fb830 msgr2=0x7f283810d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.141+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 shutdown_connections 2023-10-06T22:15:11.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.141+0000 7f283edff700 1 -- 172.21.15.57:0/56716692 wait complete. 2023-10-06T22:15:11.160 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T22:15:11.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:10 smithi156 ceph-mon[134358]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-06T22:15:11.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:10 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2478003570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:11.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.389+0000 7f8c51dff700 1 -- 172.21.15.57:0/807890287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c1010b0 msgr2=0x7f8c4c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.389+0000 7f8c51dff700 1 --2- 172.21.15.57:0/807890287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c101490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098f0 tx=0x7f8c3c009c00 comp rx=0 tx=0).stop 2023-10-06T22:15:11.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.390+0000 7f8c51dff700 1 -- 172.21.15.57:0/807890287 shutdown_connections 2023-10-06T22:15:11.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.390+0000 7f8c51dff700 1 --2- 172.21.15.57:0/807890287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1061d0 0x7f8c4c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.390+0000 7f8c51dff700 1 --2- 172.21.15.57:0/807890287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c101490 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.390+0000 7f8c51dff700 1 --2- 172.21.15.57:0/807890287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c4c101a60 0x7f8c4c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.390+0000 7f8c51dff700 1 -- 172.21.15.57:0/807890287 >> 172.21.15.57:0/807890287 conn(0x7f8c4c0fc950 msgr2=0x7f8c4c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.390+0000 7f8c51dff700 1 -- 172.21.15.57:0/807890287 shutdown_connections 2023-10-06T22:15:11.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.390+0000 7f8c51dff700 1 -- 172.21.15.57:0/807890287 wait complete. 2023-10-06T22:15:11.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.391+0000 7f8c51dff700 1 Processor -- start 2023-10-06T22:15:11.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.391+0000 7f8c51dff700 1 -- start start 2023-10-06T22:15:11.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c51dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c51dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c4c1061d0 0x7f8c4c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c51dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c19b950 0x7f8c4c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c51dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c19c3d0 con 0x7f8c4c1010b0 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c51dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c19c540 con 0x7f8c4c19b950 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c51dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c19c700 con 0x7f8c4c1061d0 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37792/0 (socket says 172.21.15.57:37792) 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/3927417428 learned_addr learned my addr 172.21.15.57:0/3927417428 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c4affd700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c4c1061d0 0x7f8c4c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.392+0000 7f8c4bfff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c19b950 0x7f8c4c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.393+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/3927417428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c4c1061d0 msgr2=0x7f8c4c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.393+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c4c1061d0 0x7f8c4c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.393+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/3927417428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c19b950 msgr2=0x7f8c4c19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.393+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c19b950 0x7f8c4c19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.393+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/3927417428 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44009d00 con 0x7f8c4c1010b0 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.393+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c196fb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009810 tx=0x7f8c3c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.393+0000 7f8c48ff9700 1 -- 172.21.15.57:0/3927417428 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c01c070 con 0x7f8c4c1010b0 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.394+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c009520 con 0x7f8c4c1010b0 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.394+0000 7f8c48ff9700 1 -- 172.21.15.57:0/3927417428 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c00bb20 con 0x7f8c4c1010b0 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.394+0000 7f8c48ff9700 1 -- 172.21.15.57:0/3927417428 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004930 con 0x7f8c4c1010b0 2023-10-06T22:15:11.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.394+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c1a3f40 con 0x7f8c4c1010b0 2023-10-06T22:15:11.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.397+0000 7f8c48ff9700 1 -- 172.21.15.57:0/3927417428 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f8c3c020070 con 0x7f8c4c1010b0 2023-10-06T22:15:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.399+0000 7f8c65701b80 1 Processor -- start 2023-10-06T22:15:11.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.399+0000 7f8c65701b80 1 -- start start 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a3d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a48350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a4ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c65701b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634279a6460 con 0x563427a3d9e0 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c65701b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427997c20 con 0x563427a3d0c0 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c65701b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427995540 con 0x563427a48890 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c48ff9700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8c340a5b20 0x7f8c340a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a48350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c48ff9700 1 -- 172.21.15.57:0/3927417428 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f8c3c0ce050 con 0x7f8c4c1010b0 2023-10-06T22:15:11.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a3d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a48350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37808/0 (socket says 172.21.15.57:37808) 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/2949037854 learned_addr learned my addr 172.21.15.57:0/2949037854 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c4bfff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8c340a5b20 0x7f8c340a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.400+0000 7f8c4affd700 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a4ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/2949037854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 msgr2=0x563427a4ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a4ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/2949037854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 msgr2=0x563427a3d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a3d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c4b7fe700 1 -- 172.21.15.57:0/2949037854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634277bbde0 con 0x563427a3d9e0 2023-10-06T22:15:11.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a48350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098c0 tx=0x7f8c3c0905f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47a74b235208df3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c4bfff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8c340a5b20 0x7f8c340a7fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009610 tx=0x7f8c4400b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c31ffb700 1 -- 172.21.15.57:0/2949037854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0a10d0 con 0x563427a3d9e0 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c31ffb700 1 -- 172.21.15.57:0/2949037854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c09b580 con 0x563427a3d9e0 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.401+0000 7f8c31ffb700 1 -- 172.21.15.57:0/2949037854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c09b0e0 con 0x563427a3d9e0 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 -- 172.21.15.57:0/2949037854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 msgr2=0x563427a48350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a48350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098c0 tx=0x7f8c3c0905f0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 -- 172.21.15.57:0/2949037854 shutdown_connections 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a4ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a3d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 --2- 172.21.15.57:0/2949037854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a48350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 -- 172.21.15.57:0/2949037854 >> 172.21.15.57:0/2949037854 conn(0x5634279b2260 msgr2=0x563427a39450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 -- 172.21.15.57:0/2949037854 shutdown_connections 2023-10-06T22:15:11.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.402+0000 7f8c65701b80 1 -- 172.21.15.57:0/2949037854 wait complete. 2023-10-06T22:15:11.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.403+0000 7f8c65701b80 1 -- start start 2023-10-06T22:15:11.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.403+0000 7f8c65701b80 1 Processor -- start 2023-10-06T22:15:11.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a48c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a53cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c65701b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427990e30 con 0x563427a3d9e0 2023-10-06T22:15:11.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c65701b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342792d710 con 0x563427a3d0c0 2023-10-06T22:15:11.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c65701b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56342792d3b0 con 0x563427a48890 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c65701b80 10 client.0 _ll_register_callbacks cb 0x56342778a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c4affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c4affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37818/0 (socket says 172.21.15.57:37818) 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c4affd700 1 -- 172.21.15.57:0/699668893 learned_addr learned my addr 172.21.15.57:0/699668893 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a48c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.404+0000 7f8c4bfff700 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a53cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c4affd700 1 -- 172.21.15.57:0/699668893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 msgr2=0x563427a53cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c4affd700 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a53cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c4affd700 1 -- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 msgr2=0x563427a48c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c4affd700 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a48c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c4affd700 1 -- 172.21.15.57:0/699668893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c1a3f40 con 0x563427a3d9e0 2023-10-06T22:15:11.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c4affd700 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a491b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8c40012840 tx=0x7f8c40012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa4bd77209565a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c2affd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c2affd700 1 -- 172.21.15.57:0/699668893 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40017070 con 0x563427a3d9e0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563427a30f10 con 0x563427a3d9e0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.405+0000 7f8c2affd700 1 -- 172.21.15.57:0/699668893 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f8c40004830 con 0x563427a3d9e0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c2affd700 1 -- 172.21.15.57:0/699668893 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c40004a30 con 0x563427a3d9e0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 10 client.15717 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5634277a4c60 con 0x563427a3d9e0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c2affd700 1 -- 172.21.15.57:0/699668893 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563427a30f10 con 0x563427a3d9e0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c2affd700 1 -- 172.21.15.57:0/699668893 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c4000f540 con 0x563427a3d9e0 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:11.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 10 client.15717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 20 client.15717 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427a57fc0 0x563427a5a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 4 client.15717 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.406+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563427989e10 con 0x563427a57fc0 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.407+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427a57fc0 0x563427a5a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.407+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427a57fc0 0x563427a5a3e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a44964610ba9503 server_cookie=4e46432914aebbbd in_seq=0 out_seq=0 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.407+0000 7f8c2affd700 10 client.15717 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c2affd700 1 -- 172.21.15.57:0/699668893 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563427989e10 con 0x563427a57fc0 2023-10-06T22:15:11.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c2affd700 10 client.15717 handle_command_reply: tid=0 2023-10-06T22:15:11.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 client.15717 shutdown 2023-10-06T22:15:11.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 msgr2=0x563427a491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a491b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8c40012840 tx=0x7f8c40012b50 comp rx=0 tx=0).stop 2023-10-06T22:15:11.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 shutdown_connections 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427a57fc0 0x563427a5a3e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427a53cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d0c0 0x563427a48c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 --2- 172.21.15.57:0/699668893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a491b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c2affd700 0 client.15717 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.408+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 >> 172.21.15.57:0/699668893 conn(0x5634279b2260 msgr2=0x563427a38fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.409+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 shutdown_connections 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.409+0000 7f8c65701b80 1 -- 172.21.15.57:0/699668893 wait complete. 2023-10-06T22:15:11.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.409+0000 7f8c65701b80 20 client.15717 trim_cache size 0 max 16384 2023-10-06T22:15:11.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.435+0000 7f8c65701b80 1 Processor -- start 2023-10-06T22:15:11.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.435+0000 7f8c65701b80 1 -- start start 2023-10-06T22:15:11.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a3d0c0 0x563427a42140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a48890 0x563427a53900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c65701b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427811920 con 0x563427a3d9e0 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c65701b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427a49520 con 0x563427a48890 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c65701b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427a47f60 con 0x563427a3d0c0 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a48890 0x563427a53900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a48890 0x563427a53900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:56056/0 (socket says 172.21.15.57:56056) 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c4affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c4bfff700 1 -- 172.21.15.57:0/1126697338 learned_addr learned my addr 172.21.15.57:0/1126697338 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.436+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a3d0c0 0x563427a42140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c4affd700 1 -- 172.21.15.57:0/1126697338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a3d0c0 msgr2=0x563427a42140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c4affd700 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a3d0c0 0x563427a42140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c4affd700 1 -- 172.21.15.57:0/1126697338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a48890 msgr2=0x563427a53900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c4affd700 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a48890 0x563427a53900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c4affd700 1 -- 172.21.15.57:0/1126697338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634277e2250 con 0x563427a3d9e0 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c4bfff700 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a48890 0x563427a53900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c4affd700 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a533c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x563427a47e80 tx=0x7f8c400096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cebaf6976e86c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c327fc700 1 -- 172.21.15.57:0/1126697338 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40017070 con 0x563427a3d9e0 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c327fc700 1 -- 172.21.15.57:0/1126697338 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c40018040 con 0x563427a3d9e0 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.437+0000 7f8c327fc700 1 -- 172.21.15.57:0/1126697338 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40005290 con 0x563427a3d9e0 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 -- 172.21.15.57:0/1126697338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 msgr2=0x563427a533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a533c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x563427a47e80 tx=0x7f8c400096d0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 -- 172.21.15.57:0/1126697338 shutdown_connections 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a48890 0x563427a53900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a3d0c0 0x563427a42140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 --2- 172.21.15.57:0/1126697338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d9e0 0x563427a533c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 -- 172.21.15.57:0/1126697338 >> 172.21.15.57:0/1126697338 conn(0x5634279b2260 msgr2=0x563427a42bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 -- 172.21.15.57:0/1126697338 shutdown_connections 2023-10-06T22:15:11.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.438+0000 7f8c65701b80 1 -- 172.21.15.57:0/1126697338 wait complete. 2023-10-06T22:15:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 -- start start 2023-10-06T22:15:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 Processor -- start 2023-10-06T22:15:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d0c0 0x563427a4baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d9e0 0x563427a4c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427aa4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427aa5350 con 0x563427a3d0c0 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427aa5490 con 0x563427a3d9e0 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563427aa5650 con 0x563427a48890 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.439+0000 7f8c65701b80 10 client.0 _ll_register_callbacks cb 0x7f8c40009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d9e0 0x563427a4c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d9e0 0x563427a4c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:56058/0 (socket says 172.21.15.57:56058) 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427aa4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 -- 172.21.15.57:0/3306530040 learned_addr learned my addr 172.21.15.57:0/3306530040 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d0c0 0x563427a4baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 -- 172.21.15.57:0/3306530040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 msgr2=0x563427aa4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427aa4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 -- 172.21.15.57:0/3306530040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d0c0 msgr2=0x563427a4baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d0c0 0x563427a4baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 -- 172.21.15.57:0/3306530040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c44008e50 con 0x563427a3d9e0 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c4affd700 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d9e0 0x563427a4c030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c40005a80 tx=0x7f8c4000f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f48cad30ad59fda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.440+0000 7f8c2affd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.441+0000 7f8c2affd700 1 -- 172.21.15.57:0/3306530040 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40017070 con 0x563427a3d9e0 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.441+0000 7f8c2affd700 1 -- 172.21.15.57:0/3306530040 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f8c4002a070 con 0x563427a3d9e0 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.441+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c3c09b580 con 0x563427a3d9e0 2023-10-06T22:15:11.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.441+0000 7f8c2affd700 1 -- 172.21.15.57:0/3306530040 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c400048f0 con 0x563427a3d9e0 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.441+0000 7f8c65701b80 10 client.25727 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.441+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5634277a4c60 con 0x563427a3d9e0 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.441+0000 7f8c2affd700 1 -- 172.21.15.57:0/3306530040 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c09b580 con 0x563427a3d9e0 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c2affd700 1 -- 172.21.15.57:0/3306530040 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c40003a70 con 0x563427a3d9e0 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c65701b80 10 client.25727 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c65701b80 10 client.25727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c65701b80 20 client.25727 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c65701b80 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427aacae0 0x563427aaef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c65701b80 4 client.25727 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563427989b30 con 0x563427aacae0 2023-10-06T22:15:11.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.442+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427aacae0 0x563427aaef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c4b7fe700 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427aacae0 0x563427aaef00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2381b346a23b8d99 server_cookie=2aa093a17df70035 in_seq=0 out_seq=0 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c2affd700 10 client.25727 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c2affd700 1 -- 172.21.15.57:0/3306530040 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563427989b30 con 0x563427aacae0 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c2affd700 10 client.25727 handle_command_reply: tid=0 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 client.25727 shutdown 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d9e0 msgr2=0x563427a4c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d9e0 0x563427a4c030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c40005a80 tx=0x7f8c4000f7d0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 shutdown_connections 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x563427aacae0 0x563427aaef00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563427a48890 0x563427aa4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563427a3d0c0 0x563427a4baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c2affd700 0 client.25727 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 --2- 172.21.15.57:0/3306530040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563427a3d9e0 0x563427a4c030 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 >> 172.21.15.57:0/3306530040 conn(0x5634279b2260 msgr2=0x563427a46090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 shutdown_connections 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.443+0000 7f8c65701b80 1 -- 172.21.15.57:0/3306530040 wait complete. 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c65701b80 20 client.25727 trim_cache size 0 max 16384 2023-10-06T22:15:11.450 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:11.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8c340a5b20 msgr2=0x7f8c340a7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8c340a5b20 0x7f8c340a7fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009610 tx=0x7f8c4400b040 comp rx=0 tx=0).stop 2023-10-06T22:15:11.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1010b0 msgr2=0x7f8c4c196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c196fb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009810 tx=0x7f8c3c00fa00 comp rx=0 tx=0).stop 2023-10-06T22:15:11.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 shutdown_connections 2023-10-06T22:15:11.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f8c340a5b20 0x7f8c340a7fe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c4c19b950 0x7f8c4c19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c4c1010b0 0x7f8c4c196fb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 --2- 172.21.15.57:0/3927417428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c4c1061d0 0x7f8c4c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 >> 172.21.15.57:0/3927417428 conn(0x7f8c4c0fc950 msgr2=0x7f8c4c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.444+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 shutdown_connections 2023-10-06T22:15:11.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.445+0000 7f8c51dff700 1 -- 172.21.15.57:0/3927417428 wait complete. 2023-10-06T22:15:11.465 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T22:15:11.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.674+0000 7f73767ff700 1 -- 172.21.15.57:0/1016009432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370073250 msgr2=0x7f7370073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.674+0000 7f73767ff700 1 --2- 172.21.15.57:0/1016009432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370073250 0x7f7370073630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7360009990 tx=0x7f7360009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.674+0000 7f73767ff700 1 -- 172.21.15.57:0/1016009432 shutdown_connections 2023-10-06T22:15:11.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.674+0000 7f73767ff700 1 --2- 172.21.15.57:0/1016009432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370111e50 0x7f7370114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.674+0000 7f73767ff700 1 --2- 172.21.15.57:0/1016009432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370073250 0x7f7370073630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.674+0000 7f73767ff700 1 --2- 172.21.15.57:0/1016009432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073b70 0x7f7370111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.675+0000 7f73767ff700 1 -- 172.21.15.57:0/1016009432 >> 172.21.15.57:0/1016009432 conn(0x7f73700fc950 msgr2=0x7f73700fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.675+0000 7f73767ff700 1 -- 172.21.15.57:0/1016009432 shutdown_connections 2023-10-06T22:15:11.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.675+0000 7f73767ff700 1 -- 172.21.15.57:0/1016009432 wait complete. 2023-10-06T22:15:11.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 Processor -- start 2023-10-06T22:15:11.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 -- start start 2023-10-06T22:15:11.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073250 0x7f737019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370073b70 0x7f737019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370111e50 0x7f73701a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701a2640 con 0x7f7370073b70 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701a27b0 con 0x7f7370111e50 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.676+0000 7f73767ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701a2970 con 0x7f7370073250 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073250 0x7f737019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f7374d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370111e50 0x7f73701a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073250 0x7f737019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59776/0 (socket says 172.21.15.57:59776) 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 -- 172.21.15.57:0/1997837212 learned_addr learned my addr 172.21.15.57:0/1997837212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736f7fe700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370073b70 0x7f737019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 -- 172.21.15.57:0/1997837212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370111e50 msgr2=0x7f73701a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370111e50 0x7f73701a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 -- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370073b70 msgr2=0x7f737019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370073b70 0x7f737019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 -- 172.21.15.57:0/1997837212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600095f0 con 0x7f7370073250 2023-10-06T22:15:11.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f7374d9c700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370111e50 0x7f73701a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:11.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736f7fe700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370073b70 0x7f737019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:11.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.677+0000 7f736ffff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073250 0x7f737019d660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7360005910 tx=0x7f7360005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.678+0000 7f736d7fa700 1 -- 172.21.15.57:0/1997837212 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736001d070 con 0x7f7370073250 2023-10-06T22:15:11.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.678+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73701a2bf0 con 0x7f7370073250 2023-10-06T22:15:11.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.678+0000 7f736d7fa700 1 -- 172.21.15.57:0/1997837212 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7360022470 con 0x7f7370073250 2023-10-06T22:15:11.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.678+0000 7f736d7fa700 1 -- 172.21.15.57:0/1997837212 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360017650 con 0x7f7370073250 2023-10-06T22:15:11.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.678+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73701aa490 con 0x7f7370073250 2023-10-06T22:15:11.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.681+0000 7f736d7fa700 1 -- 172.21.15.57:0/1997837212 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f73600225e0 con 0x7f7370073250 2023-10-06T22:15:11.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.682+0000 7f738a126b80 1 Processor -- start 2023-10-06T22:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f738a126b80 1 -- start start 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f736d7fa700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f73580a1710 0x7f73580a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f738a126b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f738a126b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f736d7fa700 1 -- 172.21.15.57:0/1997837212 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f73600ce910 con 0x7f7370073250 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f738a126b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f738a126b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b410460 con 0x555c6b4b2890 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f738a126b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b401c20 con 0x555c6b4a70c0 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f738a126b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b3ff540 con 0x555c6b4a79e0 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.683+0000 7f7374d9c700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f73580a1710 0x7f73580a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f7374d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59784/0 (socket says 172.21.15.57:59784) 2023-10-06T22:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 -- 172.21.15.57:0/4130343035 learned_addr learned my addr 172.21.15.57:0/4130343035 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736ffff700 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 -- 172.21.15.57:0/4130343035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 msgr2=0x555c6b4a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 -- 172.21.15.57:0/4130343035 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 msgr2=0x555c6b4b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 -- 172.21.15.57:0/4130343035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c6b225de0 con 0x555c6b4a79e0 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f7374d9c700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f73580a1710 0x7f73580a3bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7368009550 tx=0x7f7368009140 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736ffff700 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.684+0000 7f736f7fe700 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b2350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73640115f0 tx=0x7f7364011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afac6a1b82257fd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.685+0000 7f73567fc700 1 -- 172.21.15.57:0/4130343035 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364004230 con 0x555c6b4a79e0 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.685+0000 7f73567fc700 1 -- 172.21.15.57:0/4130343035 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7364004390 con 0x555c6b4a79e0 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.685+0000 7f73567fc700 1 -- 172.21.15.57:0/4130343035 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640197e0 con 0x555c6b4a79e0 2023-10-06T22:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.685+0000 7f738a126b80 1 -- 172.21.15.57:0/4130343035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 msgr2=0x555c6b4b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.685+0000 7f738a126b80 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b2350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73640115f0 tx=0x7f7364011900 comp rx=0 tx=0).stop 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 -- 172.21.15.57:0/4130343035 shutdown_connections 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 --2- 172.21.15.57:0/4130343035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b2350 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 -- 172.21.15.57:0/4130343035 >> 172.21.15.57:0/4130343035 conn(0x555c6b41c260 msgr2=0x555c6b4a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 -- 172.21.15.57:0/4130343035 shutdown_connections 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 -- 172.21.15.57:0/4130343035 wait complete. 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.686+0000 7f738a126b80 1 -- start start 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 1 Processor -- start 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a70c0 0x555c6b4b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4a79e0 0x555c6b4af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4b2890 0x555c6b4afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b3fae30 con 0x555c6b4a79e0 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b397710 con 0x555c6b4b2890 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b3973b0 con 0x555c6b4a70c0 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f738a126b80 10 client.0 _ll_register_callbacks cb 0x555c6b1f4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.687+0000 7f736f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4a79e0 0x555c6b4af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4a79e0 0x555c6b4af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37872/0 (socket says 172.21.15.57:37872) 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 -- 172.21.15.57:0/3061297768 learned_addr learned my addr 172.21.15.57:0/3061297768 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736ffff700 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a70c0 0x555c6b4b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f7374d9c700 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4b2890 0x555c6b4afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 -- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a70c0 msgr2=0x555c6b4b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a70c0 0x555c6b4b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 -- 172.21.15.57:0/3061297768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4b2890 msgr2=0x555c6b4afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4b2890 0x555c6b4afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 -- 172.21.15.57:0/3061297768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7364009d50 con 0x555c6b4a79e0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f736f7fe700 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4a79e0 0x555c6b4af510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f73640052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd65ceec7854390a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.688+0000 7f734f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f734f7fe700 1 -- 172.21.15.57:0/3061297768 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364003880 con 0x555c6b4a79e0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c6b49af10 con 0x555c6b4a79e0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f734f7fe700 1 -- 172.21.15.57:0/3061297768 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f73640039e0 con 0x555c6b4a79e0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f734f7fe700 1 -- 172.21.15.57:0/3061297768 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73640056f0 con 0x555c6b4a79e0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f738a126b80 10 client.15735 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555c6b20ec60 con 0x555c6b4a79e0 2023-10-06T22:15:11.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f734f7fe700 1 -- 172.21.15.57:0/3061297768 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c6b49af10 con 0x555c6b4a79e0 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.689+0000 7f734f7fe700 1 -- 172.21.15.57:0/3061297768 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7364021040 con 0x555c6b4a79e0 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.690+0000 7f738a126b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.690+0000 7f738a126b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.690+0000 7f738a126b80 20 client.15735 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.690+0000 7f738a126b80 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b4c2160 0x555c6b4c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.690+0000 7f738a126b80 4 client.15735 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.690+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c6b3f3e10 con 0x555c6b4c2160 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.690+0000 7f736ffff700 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b4c2160 0x555c6b4c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.691+0000 7f736ffff700 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b4c2160 0x555c6b4c4580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=616af964668bef13 server_cookie=7b5747c350f58fc6 in_seq=0 out_seq=0 2023-10-06T22:15:11.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.691+0000 7f734f7fe700 10 client.15735 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:11.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.691+0000 7f734f7fe700 1 -- 172.21.15.57:0/3061297768 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c6b3f3e10 con 0x555c6b4c2160 2023-10-06T22:15:11.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.691+0000 7f734f7fe700 10 client.15735 handle_command_reply: tid=0 2023-10-06T22:15:11.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.691+0000 7f738a126b80 1 client.15735 shutdown 2023-10-06T22:15:11.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.691+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4a79e0 msgr2=0x555c6b4af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.691+0000 7f738a126b80 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4a79e0 0x555c6b4af510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f73640052f0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 shutdown_connections 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b4c2160 0x555c6b4c4580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4b2890 0x555c6b4afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a70c0 0x555c6b4b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f734f7fe700 0 client.15735 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 --2- 172.21.15.57:0/3061297768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4a79e0 0x555c6b4af510 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 >> 172.21.15.57:0/3061297768 conn(0x555c6b41c260 msgr2=0x555c6b4a2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 shutdown_connections 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 1 -- 172.21.15.57:0/3061297768 wait complete. 2023-10-06T22:15:11.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.692+0000 7f738a126b80 20 client.15735 trim_cache size 0 max 16384 2023-10-06T22:15:11.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.706+0000 7f738a126b80 1 Processor -- start 2023-10-06T22:15:11.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.706+0000 7f738a126b80 1 -- start start 2023-10-06T22:15:11.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f738a126b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4aa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f738a126b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f738a126b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f738a126b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b27b920 con 0x555c6b4b2890 2023-10-06T22:15:11.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f738a126b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b4a7dc0 con 0x555c6b4a70c0 2023-10-06T22:15:11.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f738a126b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b4a8f80 con 0x555c6b4a79e0 2023-10-06T22:15:11.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f7374d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59794/0 (socket says 172.21.15.57:59794) 2023-10-06T22:15:11.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736f7fe700 1 -- 172.21.15.57:0/3846023270 learned_addr learned my addr 172.21.15.57:0/3846023270 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736ffff700 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736ffff700 1 -- 172.21.15.57:0/3846023270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 msgr2=0x555c6b4aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736ffff700 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736ffff700 1 -- 172.21.15.57:0/3846023270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 msgr2=0x555c6b4aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736ffff700 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.707+0000 7f736ffff700 1 -- 172.21.15.57:0/3846023270 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c6b24c250 con 0x555c6b4a70c0 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.708+0000 7f736ffff700 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4aa850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73600cbef0 tx=0x7f736002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97afbf23c79f138b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.708+0000 7f7356ffd700 1 -- 172.21.15.57:0/3846023270 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736009b3a0 con 0x555c6b4a70c0 2023-10-06T22:15:11.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.708+0000 7f7356ffd700 1 -- 172.21.15.57:0/3846023270 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f736009b1c0 con 0x555c6b4a70c0 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.708+0000 7f7356ffd700 1 -- 172.21.15.57:0/3846023270 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73600a3b60 con 0x555c6b4a70c0 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 -- 172.21.15.57:0/3846023270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 msgr2=0x555c6b4aa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4aa850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73600cbef0 tx=0x7f736002e040 comp rx=0 tx=0).stop 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 -- 172.21.15.57:0/3846023270 shutdown_connections 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4aa850 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 --2- 172.21.15.57:0/3846023270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 -- 172.21.15.57:0/3846023270 >> 172.21.15.57:0/3846023270 conn(0x555c6b41c260 msgr2=0x555c6b4be5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 -- 172.21.15.57:0/3846023270 shutdown_connections 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 -- 172.21.15.57:0/3846023270 wait complete. 2023-10-06T22:15:11.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 -- start start 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.709+0000 7f738a126b80 1 Processor -- start 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f738a126b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f738a126b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f738a126b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f738a126b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b515160 con 0x555c6b4b2890 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f738a126b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b5152d0 con 0x555c6b4a70c0 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f738a126b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6b515490 con 0x555c6b4a79e0 2023-10-06T22:15:11.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f738a126b80 10 client.0 _ll_register_callbacks cb 0x7f736801e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f7374d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f736f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f7374d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37892/0 (socket says 172.21.15.57:37892) 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f736ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.710+0000 7f7374d9c700 1 -- 172.21.15.57:0/2309670168 learned_addr learned my addr 172.21.15.57:0/2309670168 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f736f7fe700 1 -- 172.21.15.57:0/2309670168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 msgr2=0x555c6b4b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f736f7fe700 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f736f7fe700 1 -- 172.21.15.57:0/2309670168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 msgr2=0x555c6b4b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f736f7fe700 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f736f7fe700 1 -- 172.21.15.57:0/2309670168 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73701aa490 con 0x555c6b4a79e0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f7374d9c700 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f736f7fe700 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x555c6b4b0910 tx=0x7f7364023d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce66a1d72e7f315a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f734f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f734f7fe700 1 -- 172.21.15.57:0/2309670168 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364003a90 con 0x555c6b4a79e0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7364004390 con 0x555c6b4a79e0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f734f7fe700 1 -- 172.21.15.57:0/2309670168 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7364000e40 con 0x555c6b4a79e0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.711+0000 7f734f7fe700 1 -- 172.21.15.57:0/2309670168 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7364005360 con 0x555c6b4a79e0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 10 client.25753 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555c6b2242b0 con 0x555c6b4a79e0 2023-10-06T22:15:11.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f734f7fe700 1 -- 172.21.15.57:0/2309670168 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7364004390 con 0x555c6b4a79e0 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f734f7fe700 1 -- 172.21.15.57:0/2309670168 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7364011970 con 0x555c6b4a79e0 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 10 client.25753 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 20 client.25753 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b516760 0x555c6b518b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 4 client.25753 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.712+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c6b3f3b30 con 0x555c6b516760 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.713+0000 7f736ffff700 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b516760 0x555c6b518b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.713+0000 7f734f7fe700 10 client.25753 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.713+0000 7f736ffff700 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b516760 0x555c6b518b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6ededb481887e2 server_cookie=5cf151ff5ab8d84a in_seq=0 out_seq=0 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.713+0000 7f734f7fe700 1 -- 172.21.15.57:0/2309670168 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c6b3f3b30 con 0x555c6b516760 2023-10-06T22:15:11.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.713+0000 7f734f7fe700 10 client.25753 handle_command_reply: tid=0 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.713+0000 7f738a126b80 1 client.25753 shutdown 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 msgr2=0x555c6b4b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x555c6b4b0910 tx=0x7f7364023d40 comp rx=0 tx=0).stop 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 shutdown_connections 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555c6b516760 0x555c6b518b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555c6b4b2890 0x555c6b4b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555c6b4a70c0 0x555c6b4b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 --2- 172.21.15.57:0/2309670168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c6b4a79e0 0x555c6b4b7900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 >> 172.21.15.57:0/2309670168 conn(0x555c6b41c260 msgr2=0x555c6b4aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f734f7fe700 0 client.25753 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:11.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 shutdown_connections 2023-10-06T22:15:11.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 1 -- 172.21.15.57:0/2309670168 wait complete. 2023-10-06T22:15:11.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f738a126b80 20 client.25753 trim_cache size 0 max 16384 2023-10-06T22:15:11.721 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.714+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f73580a1710 msgr2=0x7f73580a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f73580a1710 0x7f73580a3bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7368009550 tx=0x7f7368009140 comp rx=0 tx=0).stop 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073250 msgr2=0x7f737019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073250 0x7f737019d660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7360005910 tx=0x7f7360005720 comp rx=0 tx=0).stop 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 shutdown_connections 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f73580a1710 0x7f73580a3bd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7370111e50 0x7f73701a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7370073250 0x7f737019d660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 --2- 172.21.15.57:0/1997837212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370073b70 0x7f737019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:11.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 >> 172.21.15.57:0/1997837212 conn(0x7f73700fc950 msgr2=0x7f73700fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:11.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 shutdown_connections 2023-10-06T22:15:11.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:11.715+0000 7f73767ff700 1 -- 172.21.15.57:0/1997837212 wait complete. 2023-10-06T22:15:12.738 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T22:15:13.006 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T22:15:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:12 smithi190 ceph-mon[134891]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-06T22:15:13.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:12 smithi057 ceph-mon[98492]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-06T22:15:13.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.220+0000 7fa337fff700 1 -- 172.21.15.57:0/2449264567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338106040 msgr2=0x7fa33810dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.220+0000 7fa337fff700 1 --2- 172.21.15.57:0/2449264567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338106040 0x7fa33810dbc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa33000b0a0 tx=0x7fa33000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.221+0000 7fa337fff700 1 -- 172.21.15.57:0/2449264567 shutdown_connections 2023-10-06T22:15:13.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.221+0000 7fa337fff700 1 --2- 172.21.15.57:0/2449264567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338106040 0x7fa33810dbc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.221+0000 7fa337fff700 1 --2- 172.21.15.57:0/2449264567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa338100f20 0x7fa338101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.221+0000 7fa337fff700 1 --2- 172.21.15.57:0/2449264567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3381018d0 0x7fa338105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.221+0000 7fa337fff700 1 -- 172.21.15.57:0/2449264567 >> 172.21.15.57:0/2449264567 conn(0x7fa3380fc7c0 msgr2=0x7fa3380febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.221+0000 7fa337fff700 1 -- 172.21.15.57:0/2449264567 shutdown_connections 2023-10-06T22:15:13.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.221+0000 7fa337fff700 1 -- 172.21.15.57:0/2449264567 wait complete. 2023-10-06T22:15:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 Processor -- start 2023-10-06T22:15:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 -- start start 2023-10-06T22:15:13.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338100f20 0x7fa338196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa338106040 0x7fa338197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33819b770 0x7fa33819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33819c1f0 con 0x7fa338100f20 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33819c360 con 0x7fa338106040 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa337fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33819c520 con 0x7fa33819b770 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.222+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338100f20 0x7fa338196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338100f20 0x7fa338196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57974/0 (socket says 172.21.15.57:57974) 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 -- 172.21.15.57:0/2098639139 learned_addr learned my addr 172.21.15.57:0/2098639139 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa3367fc700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa338106040 0x7fa338197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa3377fe700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33819b770 0x7fa33819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 -- 172.21.15.57:0/2098639139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33819b770 msgr2=0x7fa33819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33819b770 0x7fa33819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 -- 172.21.15.57:0/2098639139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa338106040 msgr2=0x7fa338197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa338106040 0x7fa338197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 -- 172.21.15.57:0/2098639139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa328009520 con 0x7fa338100f20 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa3367fc700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa338106040 0x7fa338197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa3377fe700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33819b770 0x7fa33819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:13.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.223+0000 7fa336ffd700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338100f20 0x7fa338196e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa32800eb30 tx=0x7fa32800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.224+0000 7fa31ffff700 1 -- 172.21.15.57:0/2098639139 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328009880 con 0x7fa338100f20 2023-10-06T22:15:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.224+0000 7fa31ffff700 1 -- 172.21.15.57:0/2098639139 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3280044e0 con 0x7fa338100f20 2023-10-06T22:15:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.224+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa330009d00 con 0x7fa338100f20 2023-10-06T22:15:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.224+0000 7fa31ffff700 1 -- 172.21.15.57:0/2098639139 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328003d80 con 0x7fa338100f20 2023-10-06T22:15:13.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.224+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3381a3dd0 con 0x7fa338100f20 2023-10-06T22:15:13.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.229+0000 7fa3511ffb80 1 Processor -- start 2023-10-06T22:15:13.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.229+0000 7fa3511ffb80 1 -- start start 2023-10-06T22:15:13.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.229+0000 7fa31ffff700 1 -- 172.21.15.57:0/2098639139 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fa328003f20 con 0x7fa338100f20 2023-10-06T22:15:13.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.229+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.229+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30e02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e30374e0 con 0x55a3e30cec40 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e3028ca0 con 0x55a3e30ce320 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e30265c0 con 0x55a3e30ddf00 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30dd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57976/0 (socket says 172.21.15.57:57976) 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa336ffd700 1 -- 172.21.15.57:0/635427166 learned_addr learned my addr 172.21.15.57:0/635427166 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3367fc700 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30e02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3377fe700 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3367fc700 1 -- 172.21.15.57:0/635427166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 msgr2=0x55a3e30ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3367fc700 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3367fc700 1 -- 172.21.15.57:0/635427166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 msgr2=0x55a3e30dd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3367fc700 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30dd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.230+0000 7fa3367fc700 1 -- 172.21.15.57:0/635427166 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3e2e4ce60 con 0x55a3e30ddf00 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.231+0000 7fa3367fc700 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30e02f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa33806a2f0 tx=0x7fa32c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c6ffdf1e78d45e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.231+0000 7fa31d7fa700 1 -- 172.21.15.57:0/635427166 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c004230 con 0x55a3e30ddf00 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.231+0000 7fa31d7fa700 1 -- 172.21.15.57:0/635427166 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa32c004cf0 con 0x55a3e30ddf00 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.231+0000 7fa31d7fa700 1 -- 172.21.15.57:0/635427166 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0056a0 con 0x55a3e30ddf00 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.231+0000 7fa31ffff700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3200a17b0 0x7fa3200a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa31ffff700 1 -- 172.21.15.57:0/2098639139 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fa328014070 con 0x7fa338100f20 2023-10-06T22:15:13.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3377fe700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3200a17b0 0x7fa3200a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 -- 172.21.15.57:0/635427166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 msgr2=0x55a3e30e02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30e02f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa33806a2f0 tx=0x7fa32c011720 comp rx=0 tx=0).stop 2023-10-06T22:15:13.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 -- 172.21.15.57:0/635427166 shutdown_connections 2023-10-06T22:15:13.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30e02f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/635427166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30dd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 -- 172.21.15.57:0/635427166 >> 172.21.15.57:0/635427166 conn(0x55a3e30432e0 msgr2=0x55a3e30ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 -- 172.21.15.57:0/635427166 shutdown_connections 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.232+0000 7fa3511ffb80 1 -- 172.21.15.57:0/635427166 wait complete. 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.233+0000 7fa3377fe700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3200a17b0 0x7fa3200a3c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa3300062a0 tx=0x7fa330006210 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.233+0000 7fa3511ffb80 1 -- start start 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.233+0000 7fa3511ffb80 1 Processor -- start 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30ce320 0x55a3e30dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30cec40 0x55a3e30daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e3021eb0 con 0x55a3e30ce320 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e2fbe790 con 0x55a3e30cec40 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e2fbe430 con 0x55a3e30ddf00 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3511ffb80 10 client.0 _ll_register_callbacks cb 0x55a3e2e1b880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30ce320 0x55a3e30dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30ce320 0x55a3e30dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57978/0 (socket says 172.21.15.57:57978) 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa336ffd700 1 -- 172.21.15.57:0/4167951462 learned_addr learned my addr 172.21.15.57:0/4167951462 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3367fc700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30cec40 0x55a3e30daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa3377fe700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.234+0000 7fa336ffd700 1 -- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 msgr2=0x55a3e30db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa336ffd700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30db240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa336ffd700 1 -- 172.21.15.57:0/4167951462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30cec40 msgr2=0x55a3e30daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa336ffd700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30cec40 0x55a3e30daba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa336ffd700 1 -- 172.21.15.57:0/4167951462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c009c90 con 0x55a3e30ce320 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa3377fe700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa336ffd700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30ce320 0x55a3e30dd920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3280a24a0 tx=0x7fa3280d0c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3a56cba14b52e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa31aff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa31aff5700 1 -- 172.21.15.57:0/4167951462 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32809aaa0 con 0x55a3e30ce320 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3e30c21e0 con 0x55a3e30ce320 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa31aff5700 1 -- 172.21.15.57:0/4167951462 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fa3280850b0 con 0x55a3e30ce320 2023-10-06T22:15:13.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa31aff5700 1 -- 172.21.15.57:0/4167951462 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa32809a410 con 0x55a3e30ce320 2023-10-06T22:15:13.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa3511ffb80 10 client.15753 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:13.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.235+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a3e2e35ce0 con 0x55a3e30ce320 2023-10-06T22:15:13.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa31aff5700 1 -- 172.21.15.57:0/4167951462 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3e30c21e0 con 0x55a3e30ce320 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa31aff5700 1 -- 172.21.15.57:0/4167951462 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa328099b30 con 0x55a3e30ce320 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa3511ffb80 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa3511ffb80 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa3511ffb80 20 client.15753 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e30ed8c0 0x55a3e30efce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa3511ffb80 4 client.15753 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3e301ae90 con 0x55a3e30ed8c0 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.236+0000 7fa3367fc700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e30ed8c0 0x55a3e30efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.237+0000 7fa3367fc700 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e30ed8c0 0x55a3e30efce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b37eacd7b733f2 server_cookie=a0f46147146e25d0 in_seq=0 out_seq=0 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.237+0000 7fa31aff5700 10 client.15753 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.237+0000 7fa31aff5700 1 -- 172.21.15.57:0/4167951462 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a3e301ae90 con 0x55a3e30ed8c0 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.237+0000 7fa31aff5700 10 client.15753 handle_command_reply: tid=0 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 client.15753 shutdown 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30ce320 msgr2=0x55a3e30dd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30ce320 0x55a3e30dd920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3280a24a0 tx=0x7fa3280d0c80 comp rx=0 tx=0).stop 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 shutdown_connections 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e30ed8c0 0x55a3e30efce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30ce320 0x55a3e30dd920 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/4167951462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30cec40 0x55a3e30daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 >> 172.21.15.57:0/4167951462 conn(0x55a3e30432e0 msgr2=0x55a3e30ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa31aff5700 0 client.15753 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 shutdown_connections 2023-10-06T22:15:13.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 1 -- 172.21.15.57:0/4167951462 wait complete. 2023-10-06T22:15:13.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.238+0000 7fa3511ffb80 20 client.15753 trim_cache size 0 max 16384 2023-10-06T22:15:13.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:12 smithi156 ceph-mon[134358]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-06T22:15:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 Processor -- start 2023-10-06T22:15:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 -- start start 2023-10-06T22:15:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e30e07b0 con 0x55a3e30cec40 2023-10-06T22:15:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e2ea29a0 con 0x55a3e30ce320 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.256+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e30ed6a0 con 0x55a3e30ddf00 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57984/0 (socket says 172.21.15.57:57984) 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 -- 172.21.15.57:0/3438301801 learned_addr learned my addr 172.21.15.57:0/3438301801 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3377fe700 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 -- 172.21.15.57:0/3438301801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 msgr2=0x55a3e30da3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30da3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 -- 172.21.15.57:0/3438301801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 msgr2=0x55a3e30d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.257+0000 7fa3367fc700 1 -- 172.21.15.57:0/3438301801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3e2e732d0 con 0x55a3e30cec40 2023-10-06T22:15:13.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.258+0000 7fa3367fc700 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d0f10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa32c011a70 tx=0x7fa32c005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0890fb12acb3b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.258+0000 7fa31dffb700 1 -- 172.21.15.57:0/3438301801 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c005020 con 0x55a3e30cec40 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.258+0000 7fa31dffb700 1 -- 172.21.15.57:0/3438301801 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa32c005ce0 con 0x55a3e30cec40 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.258+0000 7fa31dffb700 1 -- 172.21.15.57:0/3438301801 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00e6d0 con 0x55a3e30cec40 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3438301801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 msgr2=0x55a3e30d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d0f10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa32c011a70 tx=0x7fa32c005240 comp rx=0 tx=0).stop 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3438301801 shutdown_connections 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e30da3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3438301801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d0f10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3438301801 >> 172.21.15.57:0/3438301801 conn(0x55a3e30432e0 msgr2=0x55a3e30d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3438301801 shutdown_connections 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.259+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3438301801 wait complete. 2023-10-06T22:15:13.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.260+0000 7fa3511ffb80 1 -- start start 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.260+0000 7fa3511ffb80 1 Processor -- start 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3511ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e313b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e313bc70 con 0x55a3e30cec40 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e313bde0 con 0x55a3e30ce320 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3511ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3e313bfa0 con 0x55a3e30ddf00 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3511ffb80 10 client.0 _ll_register_callbacks cb 0x55a3e2e1b880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:33084/0 (socket says 172.21.15.57:33084) 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3377fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e313b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa336ffd700 1 -- 172.21.15.57:0/3467619352 learned_addr learned my addr 172.21.15.57:0/3467619352 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.261+0000 7fa3367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57988/0 (socket says 172.21.15.57:57988) 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa336ffd700 1 -- 172.21.15.57:0/3467619352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 msgr2=0x55a3e313b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa336ffd700 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e313b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa336ffd700 1 -- 172.21.15.57:0/3467619352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 msgr2=0x55a3e30d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa336ffd700 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa336ffd700 1 -- 172.21.15.57:0/3467619352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c01d040 con 0x55a3e30ce320 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa336ffd700 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d7ce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3280a29f0 tx=0x7fa328003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3971c8a6d492337f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa31aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa31aff5700 1 -- 172.21.15.57:0/3467619352 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32809a410 con 0x55a3e30ce320 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3e30ea370 con 0x55a3e30ce320 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa31aff5700 1 -- 172.21.15.57:0/3467619352 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fa3280850b0 con 0x55a3e30ce320 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa31aff5700 1 -- 172.21.15.57:0/3467619352 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa32809a180 con 0x55a3e30ce320 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa3511ffb80 10 client.25763 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.262+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a3e2e4b330 con 0x55a3e30ce320 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa31aff5700 1 -- 172.21.15.57:0/3467619352 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3e30ea370 con 0x55a3e30ce320 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa31aff5700 1 -- 172.21.15.57:0/3467619352 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa3280cb090 con 0x55a3e30ce320 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa3511ffb80 10 client.25763 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa3511ffb80 10 client.25763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:13.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa3511ffb80 20 client.25763 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e31432e0 0x55a3e3145700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa3511ffb80 4 client.25763 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7fa32c004cf0 con 0x55a3e31432e0 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.263+0000 7fa3367fc700 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e31432e0 0x55a3e3145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.264+0000 7fa3367fc700 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e31432e0 0x55a3e3145700 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84925192320b9dbb server_cookie=cec15d0e3686749d in_seq=0 out_seq=0 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.264+0000 7fa31aff5700 10 client.25763 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.265+0000 7fa31aff5700 1 -- 172.21.15.57:0/3467619352 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7fa32c004cf0 con 0x55a3e31432e0 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.265+0000 7fa31aff5700 10 client.25763 handle_command_reply: tid=0 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.265+0000 7fa3511ffb80 1 client.25763 shutdown 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.265+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 msgr2=0x55a3e30d7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d7ce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3280a29f0 tx=0x7fa328003680 comp rx=0 tx=0).stop 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 shutdown_connections 2023-10-06T22:15:13.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a3e31432e0 0x55a3e3145700 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3e30ddf00 0x55a3e313b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a3e30ce320 0x55a3e30d7ce0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 --2- 172.21.15.57:0/3467619352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a3e30cec40 0x55a3e30d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa31aff5700 0 client.25763 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 >> 172.21.15.57:0/3467619352 conn(0x55a3e30432e0 msgr2=0x55a3e30d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 shutdown_connections 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 1 -- 172.21.15.57:0/3467619352 wait complete. 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.266+0000 7fa3511ffb80 20 client.25763 trim_cache size 0 max 16384 2023-10-06T22:15:13.330 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:13.331 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-06T22:15:13.331 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "e8b71dee-88c9-4506-b742-8bab8f4d4567", 2023-10-06T22:15:13.331 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-06T22:15:13.331 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3200a17b0 msgr2=0x7fa3200a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3200a17b0 0x7fa3200a3c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa3300062a0 tx=0x7fa330006210 comp rx=0 tx=0).stop 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338100f20 msgr2=0x7fa338196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338100f20 0x7fa338196e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa32800eb30 tx=0x7fa32800ee40 comp rx=0 tx=0).stop 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 shutdown_connections 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3200a17b0 0x7fa3200a3c70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33819b770 0x7fa33819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa338100f20 0x7fa338196e40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 --2- 172.21.15.57:0/2098639139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa338106040 0x7fa338197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 >> 172.21.15.57:0/2098639139 conn(0x7fa3380fc7c0 msgr2=0x7fa3380fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 shutdown_connections 2023-10-06T22:15:13.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.267+0000 7fa337fff700 1 -- 172.21.15.57:0/2098639139 wait complete. 2023-10-06T22:15:13.336 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:15:13.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.569+0000 7fc389d9d700 1 -- 172.21.15.57:0/3792865767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 msgr2=0x7fc384068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.569+0000 7fc389d9d700 1 --2- 172.21.15.57:0/3792865767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 0x7fc384068040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc378009990 tx=0x7fc378009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.570+0000 7fc389d9d700 1 -- 172.21.15.57:0/3792865767 shutdown_connections 2023-10-06T22:15:13.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.570+0000 7fc389d9d700 1 --2- 172.21.15.57:0/3792865767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3841036f0 0x7fc384103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.570+0000 7fc389d9d700 1 --2- 172.21.15.57:0/3792865767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 0x7fc384068040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.570+0000 7fc389d9d700 1 --2- 172.21.15.57:0/3792865767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc384068580 0x7fc3840689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.570+0000 7fc389d9d700 1 -- 172.21.15.57:0/3792865767 >> 172.21.15.57:0/3792865767 conn(0x7fc3840769d0 msgr2=0x7fc384076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.570+0000 7fc389d9d700 1 -- 172.21.15.57:0/3792865767 shutdown_connections 2023-10-06T22:15:13.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.570+0000 7fc389d9d700 1 -- 172.21.15.57:0/3792865767 wait complete. 2023-10-06T22:15:13.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 Processor -- start 2023-10-06T22:15:13.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 -- start start 2023-10-06T22:15:13.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc384068580 0x7fc38419d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841036f0 0x7fc38419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 0x7fc3841a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3841a2240 con 0x7fc384105ca0 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3841a23b0 con 0x7fc3841036f0 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.571+0000 7fc389d9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3841a2570 con 0x7fc384068580 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc383fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841036f0 0x7fc38419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc388d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc384068580 0x7fc38419d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc383fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841036f0 0x7fc38419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:33094/0 (socket says 172.21.15.57:33094) 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc388d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc384068580 0x7fc38419d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59134/0 (socket says 172.21.15.57:59134) 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc38959c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 0x7fc3841a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc388d9b700 1 -- 172.21.15.57:0/1250838353 learned_addr learned my addr 172.21.15.57:0/1250838353 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc38959c700 1 -- 172.21.15.57:0/1250838353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc384068580 msgr2=0x7fc38419d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc38959c700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc384068580 0x7fc38419d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc38959c700 1 -- 172.21.15.57:0/1250838353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841036f0 msgr2=0x7fc38419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc38959c700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841036f0 0x7fc38419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc38959c700 1 -- 172.21.15.57:0/1250838353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3780095f0 con 0x7fc384105ca0 2023-10-06T22:15:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc388d9b700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc384068580 0x7fc38419d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc383fff700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841036f0 0x7fc38419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.572+0000 7fc38959c700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 0x7fc3841a1c00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00efc0 tx=0x7fc37c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.573+0000 7fc381ffb700 1 -- 172.21.15.57:0/1250838353 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c018070 con 0x7fc384105ca0 2023-10-06T22:15:13.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.573+0000 7fc381ffb700 1 -- 172.21.15.57:0/1250838353 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc37c00f040 con 0x7fc384105ca0 2023-10-06T22:15:13.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.573+0000 7fc381ffb700 1 -- 172.21.15.57:0/1250838353 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00cd40 con 0x7fc384105ca0 2023-10-06T22:15:13.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.573+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3841a27f0 con 0x7fc384105ca0 2023-10-06T22:15:13.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.573+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3841aa090 con 0x7fc384105ca0 2023-10-06T22:15:13.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.578+0000 7fc39e85cb80 1 Processor -- start 2023-10-06T22:15:13.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.578+0000 7fc381ffb700 1 -- 172.21.15.57:0/1250838353 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fc37c003a00 con 0x7fc384105ca0 2023-10-06T22:15:13.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.578+0000 7fc39e85cb80 1 -- start start 2023-10-06T22:15:13.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.578+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68366680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.578+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68366bc0 0x55fe68371530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68371a70 0x55fe68373e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe682cf460 con 0x55fe68371a70 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe682c0c20 con 0x55fe68366bc0 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe682be540 con 0x55fe683662a0 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68366bc0 0x55fe68371530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68366bc0 0x55fe68371530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:33104/0 (socket says 172.21.15.57:33104) 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 -- 172.21.15.57:0/359858694 learned_addr learned my addr 172.21.15.57:0/359858694 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc383fff700 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68366680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc383fff700 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68371a70 0x55fe68373e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 -- 172.21.15.57:0/359858694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 msgr2=0x55fe68366680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68366680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 -- 172.21.15.57:0/359858694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68371a70 msgr2=0x55fe68373e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68371a70 0x55fe68373e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.579+0000 7fc388d9b700 1 -- 172.21.15.57:0/359858694 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe680e4de0 con 0x55fe68366bc0 2023-10-06T22:15:13.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.580+0000 7fc381ffb700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc3740a17b0 0x7fc3740a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.580+0000 7fc388d9b700 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68366bc0 0x55fe68371530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc378005f30 tx=0x7fc378005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc70e9ca4055100 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.580+0000 7fc381ffb700 1 -- 172.21.15.57:0/1250838353 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc37c0ccc40 con 0x7fc384105ca0 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.580+0000 7fc383fff700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc3740a17b0 0x7fc3740a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.580+0000 7fc36effd700 1 -- 172.21.15.57:0/359858694 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378027070 con 0x55fe68366bc0 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.580+0000 7fc36effd700 1 -- 172.21.15.57:0/359858694 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc37801e040 con 0x55fe68366bc0 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.580+0000 7fc36effd700 1 -- 172.21.15.57:0/359858694 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37801d050 con 0x55fe68366bc0 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 -- 172.21.15.57:0/359858694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68366bc0 msgr2=0x55fe68371530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc383fff700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc3740a17b0 0x7fc3740a3c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc370009fd0 tx=0x7fc370009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68366bc0 0x55fe68371530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc378005f30 tx=0x7fc378005c00 comp rx=0 tx=0).stop 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 -- 172.21.15.57:0/359858694 shutdown_connections 2023-10-06T22:15:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68371a70 0x55fe68373e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68366680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/359858694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68366bc0 0x55fe68371530 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 -- 172.21.15.57:0/359858694 >> 172.21.15.57:0/359858694 conn(0x55fe682db260 msgr2=0x55fe68362630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 -- 172.21.15.57:0/359858694 shutdown_connections 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.581+0000 7fc39e85cb80 1 -- 172.21.15.57:0/359858694 wait complete. 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.582+0000 7fc39e85cb80 1 -- start start 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.582+0000 7fc39e85cb80 1 Processor -- start 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.582+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.582+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.582+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.582+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe682b9e30 con 0x55fe68366bc0 2023-10-06T22:15:13.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe68256710 con 0x55fe683662a0 2023-10-06T22:15:13.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe682563b0 con 0x55fe68371a70 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc39e85cb80 10 client.0 _ll_register_callbacks cb 0x55fe680b3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58030/0 (socket says 172.21.15.57:58030) 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc38959c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 -- 172.21.15.57:0/1913240297 learned_addr learned my addr 172.21.15.57:0/1913240297 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc388d9b700 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 -- 172.21.15.57:0/1913240297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 msgr2=0x55fe6836ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 -- 172.21.15.57:0/1913240297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 msgr2=0x55fe68371520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.583+0000 7fc383fff700 1 -- 172.21.15.57:0/1913240297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc384114c90 con 0x55fe68366bc0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc383fff700 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836e730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc3700097c0 tx=0x7fc3700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c35e5f19c406114 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc363fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc363fff700 1 -- 172.21.15.57:0/1913240297 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370027070 con 0x55fe68366bc0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe6835a160 con 0x55fe68366bc0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc363fff700 1 -- 172.21.15.57:0/1913240297 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc37002b410 con 0x55fe68366bc0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc363fff700 1 -- 172.21.15.57:0/1913240297 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc370016070 con 0x55fe68366bc0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc39e85cb80 10 client.15771 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.584+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fe680cdc60 con 0x55fe68366bc0 2023-10-06T22:15:13.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc363fff700 1 -- 172.21.15.57:0/1913240297 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc370039020 con 0x55fe68366bc0 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc363fff700 1 -- 172.21.15.57:0/1913240297 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc3700359c0 con 0x55fe68366bc0 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc39e85cb80 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc39e85cb80 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc39e85cb80 20 client.15771 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683814f0 0x55fe68383910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc39e85cb80 4 client.15771 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe682b2e10 con 0x55fe683814f0 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.585+0000 7fc388d9b700 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683814f0 0x55fe68383910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.586+0000 7fc388d9b700 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683814f0 0x55fe68383910 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eeec35c1eff3ccf server_cookie=313f5e8ea9b4f681 in_seq=0 out_seq=0 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.586+0000 7fc363fff700 10 client.15771 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc363fff700 1 -- 172.21.15.57:0/1913240297 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe682b2e10 con 0x55fe683814f0 2023-10-06T22:15:13.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc363fff700 10 client.15771 handle_command_reply: tid=0 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 client.15771 shutdown 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 msgr2=0x55fe6836e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836e730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc3700097c0 tx=0x7fc3700047c0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 shutdown_connections 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683814f0 0x55fe68383910 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/1913240297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836e730 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 >> 172.21.15.57:0/1913240297 conn(0x55fe682db260 msgr2=0x55fe683621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc363fff700 0 client.15771 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 shutdown_connections 2023-10-06T22:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 1 -- 172.21.15.57:0/1913240297 wait complete. 2023-10-06T22:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.587+0000 7fc39e85cb80 20 client.15771 trim_cache size 0 max 16384 2023-10-06T22:15:13.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 Processor -- start 2023-10-06T22:15:13.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 -- start start 2023-10-06T22:15:13.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6813a920 con 0x55fe68366bc0 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe683632e0 con 0x55fe683662a0 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6836f470 con 0x55fe68371a70 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc383fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc388d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc383fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58032/0 (socket says 172.21.15.57:58032) 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc383fff700 1 -- 172.21.15.57:0/2611741176 learned_addr learned my addr 172.21.15.57:0/2611741176 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.606+0000 7fc38959c700 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc383fff700 1 -- 172.21.15.57:0/2611741176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 msgr2=0x55fe6836bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc383fff700 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc383fff700 1 -- 172.21.15.57:0/2611741176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 msgr2=0x55fe68371050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc383fff700 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc383fff700 1 -- 172.21.15.57:0/2611741176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe6810b250 con 0x55fe68366bc0 2023-10-06T22:15:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc38959c700 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc388d9b700 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:13.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc383fff700 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836ba00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc370035da0 tx=0x7fc3700113b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4651e050b975577f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc36f7fe700 1 -- 172.21.15.57:0/2611741176 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370027030 con 0x55fe68366bc0 2023-10-06T22:15:13.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc36f7fe700 1 -- 172.21.15.57:0/2611741176 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc370039070 con 0x55fe68366bc0 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.607+0000 7fc36f7fe700 1 -- 172.21.15.57:0/2611741176 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37002bb40 con 0x55fe68366bc0 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 -- 172.21.15.57:0/2611741176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 msgr2=0x55fe6836ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836ba00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc370035da0 tx=0x7fc3700113b0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 -- 172.21.15.57:0/2611741176 shutdown_connections 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe68371a70 0x55fe6836bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe683662a0 0x55fe68371050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/2611741176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe6836ba00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 -- 172.21.15.57:0/2611741176 >> 172.21.15.57:0/2611741176 conn(0x55fe682db260 msgr2=0x55fe68369a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 -- 172.21.15.57:0/2611741176 shutdown_connections 2023-10-06T22:15:13.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.608+0000 7fc39e85cb80 1 -- 172.21.15.57:0/2611741176 wait complete. 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.609+0000 7fc39e85cb80 1 -- start start 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.609+0000 7fc39e85cb80 1 Processor -- start 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.609+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68377fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.609+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe68378510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc39e85cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68371a70 0x55fe683765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe68376cb0 con 0x55fe68366bc0 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe68376e20 con 0x55fe68371a70 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc39e85cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe68376fe0 con 0x55fe683662a0 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc39e85cb80 10 client.0 _ll_register_callbacks cb 0x7fc370010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68377fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc383fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe68378510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68377fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59176/0 (socket says 172.21.15.57:59176) 2023-10-06T22:15:13.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 -- 172.21.15.57:0/484175449 learned_addr learned my addr 172.21.15.57:0/484175449 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc38959c700 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68371a70 0x55fe683765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 -- 172.21.15.57:0/484175449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68371a70 msgr2=0x55fe683765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68371a70 0x55fe683765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 -- 172.21.15.57:0/484175449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 msgr2=0x55fe68378510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe68378510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc388d9b700 1 -- 172.21.15.57:0/484175449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe680e4de0 con 0x55fe683662a0 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.610+0000 7fc38959c700 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68371a70 0x55fe683765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc388d9b700 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68377fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc3780095c0 tx=0x7fc37800bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e72215a940f4421 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc363fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc363fff700 1 -- 172.21.15.57:0/484175449 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378032070 con 0x55fe683662a0 2023-10-06T22:15:13.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc37801e040 con 0x55fe683662a0 2023-10-06T22:15:13.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc363fff700 1 -- 172.21.15.57:0/484175449 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc378023730 con 0x55fe683662a0 2023-10-06T22:15:13.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc363fff700 1 -- 172.21.15.57:0/484175449 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc37801d040 con 0x55fe683662a0 2023-10-06T22:15:13.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc39e85cb80 10 client.25789 fetch_fsmap learned FSMap version 26 2023-10-06T22:15:13.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.611+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc3841aa090 con 0x55fe683662a0 2023-10-06T22:15:13.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc363fff700 1 -- 172.21.15.57:0/484175449 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc37801e040 con 0x55fe683662a0 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc363fff700 1 -- 172.21.15.57:0/484175449 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc37800fae0 con 0x55fe683662a0 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc39e85cb80 10 client.25789 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc39e85cb80 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc39e85cb80 20 client.25789 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683d5fa0 0x55fe683d8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc39e85cb80 4 client.25789 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe682b2b30 con 0x55fe683d5fa0 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.612+0000 7fc383fff700 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683d5fa0 0x55fe683d8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc383fff700 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683d5fa0 0x55fe683d8390 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a8cf07aebeb246 server_cookie=200a43a803972aba in_seq=0 out_seq=0 2023-10-06T22:15:13.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc363fff700 10 client.25789 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc363fff700 1 -- 172.21.15.57:0/484175449 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fe682b2b30 con 0x55fe683d5fa0 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc363fff700 10 client.25789 handle_command_reply: tid=0 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 client.25789 shutdown 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 msgr2=0x55fe68377fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68377fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc3780095c0 tx=0x7fc37800bc70 comp rx=0 tx=0).stop 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 shutdown_connections 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fe683d5fa0 0x55fe683d8390 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fe68371a70 0x55fe683765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc363fff700 0 client.25789 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe683662a0 0x55fe68377fd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 --2- 172.21.15.57:0/484175449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe68366bc0 0x55fe68378510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.613+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 >> 172.21.15.57:0/484175449 conn(0x55fe682db260 msgr2=0x55fe68361880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 shutdown_connections 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc39e85cb80 1 -- 172.21.15.57:0/484175449 wait complete. 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc39e85cb80 20 client.25789 trim_cache size 0 max 16384 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stdout: "e8b71dee-88c9-4506-b742-8bab8f4d4567": { 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:15:13.621 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "e8b71dee-88c9-4506-b742-8bab8f4d4567", 2023-10-06T22:15:13.622 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:15:13.622 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:15:13.622 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:15:13.622 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:13.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc3740a17b0 msgr2=0x7fc3740a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc3740a17b0 0x7fc3740a3c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc370009fd0 tx=0x7fc370009260 comp rx=0 tx=0).stop 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 msgr2=0x7fc3841a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 0x7fc3841a1c00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00efc0 tx=0x7fc37c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 shutdown_connections 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc3740a17b0 0x7fc3740a3c70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc384105ca0 0x7fc3841a1c00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc384068580 0x7fc38419d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.614+0000 7fc389d9d700 1 --2- 172.21.15.57:0/1250838353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3841036f0 0x7fc38419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.615+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 >> 172.21.15.57:0/1250838353 conn(0x7fc3840769d0 msgr2=0x7fc3840fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.615+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 shutdown_connections 2023-10-06T22:15:13.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:13.615+0000 7fc389d9d700 1 -- 172.21.15.57:0/1250838353 wait complete. 2023-10-06T22:15:13.635 INFO:tasks.cephfs.filesystem:scrub status for tag:e8b71dee-88c9-4506-b742-8bab8f4d4567 - {'path': '/', 'tag': 'e8b71dee-88c9-4506-b742-8bab8f4d4567', 'options': 'recursive,force'} 2023-10-06T22:15:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:13 smithi190 ceph-mon[134891]: scrub queued for path: / 2023-10-06T22:15:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:13 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:13 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:15:14.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:13 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:13 smithi057 ceph-mon[98492]: scrub queued for path: / 2023-10-06T22:15:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:13 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:13 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:15:14.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:13 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:14.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:13 smithi156 ceph-mon[134358]: scrub queued for path: / 2023-10-06T22:15:14.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:13 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:14.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:13 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:15:14.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:13 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:14 smithi190 ceph-mon[134891]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-10-06T22:15:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:14 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:15:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:14 smithi057 ceph-mon[98492]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-10-06T22:15:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:14 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:15:15.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:14 smithi156 ceph-mon[134358]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-10-06T22:15:15.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:14 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:15:15.802 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:15 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Service RestartSec=10s expired, scheduling restart. 2023-10-06T22:15:15.802 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:15 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Scheduled restart job, restart counter is at 1. 2023-10-06T22:15:15.803 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:15 smithi156 systemd[1]: Stopped Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:15:15.803 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:15 smithi156 systemd[1]: Starting Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:15:16.097 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:15 smithi156 podman[174932]: 2023-10-06T22:15:16.098 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:16 smithi156 bash[174932]: 2954cf7ca7d86cfb3691fdee6f5cfc3388c9e4a3eeaec27fb7564143ef9467e7 2023-10-06T22:15:16.098 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:16 smithi156 ceph-mds[174972]: -- 172.21.15.156:0/3007633183 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e8488691e0 con 0x55e84883ec00 2023-10-06T22:15:16.098 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: starting mds.e at 2023-10-06T22:15:16.503 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:15:16 smithi156 systemd[1]: Started Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:15:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:16 smithi057 ceph-mon[98492]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2023-10-06T22:15:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:16 smithi057 ceph-mon[98492]: daemon mds.e restarted 2023-10-06T22:15:17.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:16 smithi156 ceph-mon[134358]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2023-10-06T22:15:17.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:16 smithi156 ceph-mon[134358]: daemon mds.e restarted 2023-10-06T22:15:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:16 smithi190 ceph-mon[134891]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2023-10-06T22:15:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:16 smithi190 ceph-mon[134891]: daemon mds.e restarted 2023-10-06T22:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 -- 172.21.15.57:0/976329471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc0682d0 msgr2=0x7f07bc068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 --2- 172.21.15.57:0/976329471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc0682d0 0x7f07bc068750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009990 tx=0x7f07ac009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 -- 172.21.15.57:0/976329471 shutdown_connections 2023-10-06T22:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 --2- 172.21.15.57:0/976329471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc068c90 0x7f07bc10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 --2- 172.21.15.57:0/976329471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc106f60 0x7f07bc107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 --2- 172.21.15.57:0/976329471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc0682d0 0x7f07bc068750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 -- 172.21.15.57:0/976329471 >> 172.21.15.57:0/976329471 conn(0x7f07bc076b70 msgr2=0x7f07bc076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 -- 172.21.15.57:0/976329471 shutdown_connections 2023-10-06T22:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.827+0000 7f07c41ff700 1 -- 172.21.15.57:0/976329471 wait complete. 2023-10-06T22:15:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.829+0000 7f07c41ff700 1 Processor -- start 2023-10-06T22:15:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.829+0000 7f07c41ff700 1 -- start start 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c41ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc0682d0 0x7f07bc196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c41ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc068c90 0x7f07bc1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c41ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc106f60 0x7f07bc19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c41ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc19c0e0 con 0x7f07bc106f60 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c41ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc19c250 con 0x7f07bc068c90 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c41ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc19c410 con 0x7f07bc0682d0 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c1f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc0682d0 0x7f07bc196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c279c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc106f60 0x7f07bc19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c179a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc068c90 0x7f07bc1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c1f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc0682d0 0x7f07bc196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:59194/0 (socket says 172.21.15.57:59194) 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c179a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc068c90 0x7f07bc1974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:33122/0 (socket says 172.21.15.57:33122) 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c279c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc106f60 0x7f07bc19baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58064/0 (socket says 172.21.15.57:58064) 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.830+0000 7f07c1f9b700 1 -- 172.21.15.57:0/2511227687 learned_addr learned my addr 172.21.15.57:0/2511227687 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c1f9b700 1 -- 172.21.15.57:0/2511227687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc068c90 msgr2=0x7f07bc1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc068c90 0x7f07bc1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c1f9b700 1 -- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc106f60 msgr2=0x7f07bc19baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc106f60 0x7f07bc19baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c1f9b700 1 -- 172.21.15.57:0/2511227687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4009c40 con 0x7f07bc0682d0 2023-10-06T22:15:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c179a700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc068c90 0x7f07bc1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc0682d0 0x7f07bc196f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9a0 tx=0x7f07b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07b2ffd700 1 -- 172.21.15.57:0/2511227687 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8009790 con 0x7f07bc0682d0 2023-10-06T22:15:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07b2ffd700 1 -- 172.21.15.57:0/2511227687 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b8010430 con 0x7f07bc0682d0 2023-10-06T22:15:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac0095f0 con 0x7f07bc0682d0 2023-10-06T22:15:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.831+0000 7f07b2ffd700 1 -- 172.21.15.57:0/2511227687 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8018680 con 0x7f07bc0682d0 2023-10-06T22:15:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.832+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc100150 con 0x7f07bc0682d0 2023-10-06T22:15:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.836+0000 7f07d7ae2b80 1 Processor -- start 2023-10-06T22:15:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.836+0000 7f07d7ae2b80 1 -- start start 2023-10-06T22:15:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77afc00 0x55e1b77affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77b0520 0x55e1b77bae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b7719400 con 0x55e1b77afc00 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b770abc0 con 0x55e1b77b0520 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b77084e0 con 0x55e1b77bb3d0 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77afc00 0x55e1b77affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77afc00 0x55e1b77affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58080/0 (socket says 172.21.15.57:58080) 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 -- 172.21.15.57:0/105877608 learned_addr learned my addr 172.21.15.57:0/105877608 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c179a700 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 -- 172.21.15.57:0/105877608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 msgr2=0x55e1b77bd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77bd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 -- 172.21.15.57:0/105877608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77b0520 msgr2=0x55e1b77bae90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:15:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77b0520 0x55e1b77bae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c279c700 1 -- 172.21.15.57:0/105877608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b752ed80 con 0x55e1b77afc00 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.837+0000 7f07c179a700 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77bd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.838+0000 7f07b2ffd700 1 -- 172.21.15.57:0/2511227687 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f07b80105a0 con 0x7f07bc0682d0 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.838+0000 7f07c279c700 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77afc00 0x55e1b77affe0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e760 tx=0x7f07b4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4a465699774a42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.838+0000 7f07a7fff700 1 -- 172.21.15.57:0/105877608 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b400e040 con 0x55e1b77afc00 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.838+0000 7f07a7fff700 1 -- 172.21.15.57:0/105877608 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b400f040 con 0x55e1b77afc00 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.838+0000 7f07a7fff700 1 -- 172.21.15.57:0/105877608 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4007700 con 0x55e1b77afc00 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/105877608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77afc00 msgr2=0x55e1b77affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77afc00 0x55e1b77affe0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e760 tx=0x7f07b4009920 comp rx=0 tx=0).stop 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/105877608 shutdown_connections 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77bd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77afc00 0x55e1b77affe0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/105877608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77b0520 0x55e1b77bae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/105877608 >> 172.21.15.57:0/105877608 conn(0x55e1b7725200 msgr2=0x55e1b77ac080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/105877608 shutdown_connections 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/105877608 wait complete. 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07b2ffd700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07a80a16a0 0x7f07a80a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.839+0000 7f07b2ffd700 1 -- 172.21.15.57:0/2511227687 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f07b8014070 con 0x7f07bc0682d0 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.840+0000 7f07c179a700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07a80a16a0 0x7f07a80a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.840+0000 7f07d7ae2b80 1 -- start start 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.840+0000 7f07d7ae2b80 1 Processor -- start 2023-10-06T22:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.840+0000 7f07c179a700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07a80a16a0 0x7f07a80a3b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07bc198710 tx=0x7f07ac01e040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77afc00 0x55e1b77bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b7703dd0 con 0x55e1b77b0520 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b76a06b0 con 0x55e1b77bb3d0 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b76a0350 con 0x55e1b77afc00 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07d7ae2b80 10 client.0 _ll_register_callbacks cb 0x55e1b74fd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07c179a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07c179a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58096/0 (socket says 172.21.15.57:58096) 2023-10-06T22:15:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07c1f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77afc00 0x55e1b77bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.841+0000 7f07c179a700 1 -- 172.21.15.57:0/1181172296 learned_addr learned my addr 172.21.15.57:0/1181172296 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c279c700 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c1f9b700 1 -- 172.21.15.57:0/1181172296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77bb3d0 msgr2=0x55e1b77b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c1f9b700 1 -- 172.21.15.57:0/1181172296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 msgr2=0x55e1b77b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c1f9b700 1 -- 172.21.15.57:0/1181172296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b400e410 con 0x55e1b77afc00 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c179a700 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77afc00 0x55e1b77bae20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f07b80caac0 tx=0x7f07b809a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb793f39dc9c15d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07a57fa700 1 -- 172.21.15.57:0/1181172296 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b809a280 con 0x55e1b77afc00 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1b77a3d50 con 0x55e1b77afc00 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.842+0000 7f07a57fa700 1 -- 172.21.15.57:0/1181172296 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f07b809e3d0 con 0x55e1b77afc00 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07a57fa700 1 -- 172.21.15.57:0/1181172296 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b80a2610 con 0x55e1b77afc00 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 10 client.25804 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e1b7517c00 con 0x55e1b77afc00 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07a57fa700 1 -- 172.21.15.57:0/1181172296 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1b77a3d50 con 0x55e1b77afc00 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07a57fa700 1 -- 172.21.15.57:0/1181172296 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07b807d020 con 0x55e1b77afc00 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 10 client.25804 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 20 client.25804 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b77cadd0 0x55e1b77cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 4 client.25804 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.843+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1b76fcdb0 con 0x55e1b77cadd0 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.844+0000 7f07c179a700 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b77cadd0 0x55e1b77cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.844+0000 7f07c179a700 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b77cadd0 0x55e1b77cd1f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ac26cc69a07e4 server_cookie=a42d92ebc75db02b in_seq=0 out_seq=0 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.844+0000 7f07a57fa700 10 client.25804 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07a57fa700 1 -- 172.21.15.57:0/1181172296 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e1b76fcdb0 con 0x55e1b77cadd0 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07a57fa700 10 client.25804 handle_command_reply: tid=0 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 client.25804 shutdown 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77afc00 msgr2=0x55e1b77bae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77afc00 0x55e1b77bae20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f07b80caac0 tx=0x7f07b809a7c0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 shutdown_connections 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b77cadd0 0x55e1b77cd1f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77afc00 0x55e1b77bae20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07a57fa700 0 client.25804 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1181172296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 >> 172.21.15.57:0/1181172296 conn(0x55e1b7725200 msgr2=0x55e1b77abc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 shutdown_connections 2023-10-06T22:15:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.845+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1181172296 wait complete. 2023-10-06T22:15:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.846+0000 7f07d7ae2b80 20 client.25804 trim_cache size 0 max 16384 2023-10-06T22:15:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.861+0000 7f07d7ae2b80 1 Processor -- start 2023-10-06T22:15:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.861+0000 7f07d7ae2b80 1 -- start start 2023-10-06T22:15:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77ba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b75848c0 con 0x55e1b77b0520 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b77acdc0 con 0x55e1b77afc00 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b77b8dc0 con 0x55e1b77bb3d0 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07c179a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07c179a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77c1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58100/0 (socket says 172.21.15.57:58100) 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07c1f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07c179a700 1 -- 172.21.15.57:0/3668874135 learned_addr learned my addr 172.21.15.57:0/3668874135 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.862+0000 7f07c279c700 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07c179a700 1 -- 172.21.15.57:0/3668874135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 msgr2=0x55e1b77ba950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07c179a700 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77ba950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07c179a700 1 -- 172.21.15.57:0/3668874135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 msgr2=0x55e1b77b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07c179a700 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07c179a700 1 -- 172.21.15.57:0/3668874135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b75551f0 con 0x55e1b77b0520 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07c179a700 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77c1f10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0098e0 tx=0x7f07ac026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1761c47ac9ce1631 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07b0ff9700 1 -- 172.21.15.57:0/3668874135 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac021050 con 0x55e1b77b0520 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07b0ff9700 1 -- 172.21.15.57:0/3668874135 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07ac022040 con 0x55e1b77b0520 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.863+0000 7f07b0ff9700 1 -- 172.21.15.57:0/3668874135 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac004740 con 0x55e1b77b0520 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/3668874135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 msgr2=0x55e1b77c1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77c1f10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0098e0 tx=0x7f07ac026000 comp rx=0 tx=0).stop 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/3668874135 shutdown_connections 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77ba950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/3668874135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77c1f10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/3668874135 >> 172.21.15.57:0/3668874135 conn(0x55e1b7725200 msgr2=0x55e1b77b7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/3668874135 shutdown_connections 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.864+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/3668874135 wait complete. 2023-10-06T22:15:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 -- start start 2023-10-06T22:15:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 Processor -- start 2023-10-06T22:15:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b77b4de0 con 0x55e1b77b0520 2023-10-06T22:15:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b77b4f50 con 0x55e1b77afc00 2023-10-06T22:15:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.865+0000 7f07d7ae2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b77b5110 con 0x55e1b77bb3d0 2023-10-06T22:15:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07d7ae2b80 10 client.0 _ll_register_callbacks cb 0x55e1b74fd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c179a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:33146/0 (socket says 172.21.15.57:33146) 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 -- 172.21.15.57:0/1675066620 learned_addr learned my addr 172.21.15.57:0/1675066620 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c279c700 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 -- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 msgr2=0x55e1b77b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 -- 172.21.15.57:0/1675066620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 msgr2=0x55e1b77b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 -- 172.21.15.57:0/1675066620 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1b7517c00 con 0x55e1b77afc00 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c179a700 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07c1f9b700 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b5a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07b80985d0 tx=0x7f07b8099e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2fc67502610240 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.866+0000 7f07a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07a57fa700 1 -- 172.21.15.57:0/1675066620 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b809e3d0 con 0x55e1b77afc00 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07b400f040 con 0x55e1b77afc00 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07a57fa700 1 -- 172.21.15.57:0/1675066620 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f07b807b050 con 0x55e1b77afc00 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07a57fa700 1 -- 172.21.15.57:0/1675066620 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b80b82c0 con 0x55e1b77afc00 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 10 client.25799 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f07bc10e5c0 con 0x55e1b77afc00 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07a57fa700 1 -- 172.21.15.57:0/1675066620 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b400f040 con 0x55e1b77afc00 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07a57fa700 1 -- 172.21.15.57:0/1675066620 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07b807d040 con 0x55e1b77afc00 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 20 client.25799 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b781fc30 0x55e1b7822050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 4 client.25799 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.867+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e1b76fcad0 con 0x55e1b781fc30 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.868+0000 7f07c179a700 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b781fc30 0x55e1b7822050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.868+0000 7f07a57fa700 10 client.25799 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.868+0000 7f07c179a700 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b781fc30 0x55e1b7822050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24250c176822711 server_cookie=1e9cbc128d8c778c in_seq=0 out_seq=0 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.872+0000 7f07a57fa700 1 -- 172.21.15.57:0/1675066620 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e1b76fcad0 con 0x55e1b781fc30 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.872+0000 7f07a57fa700 10 client.25799 handle_command_reply: tid=0 2023-10-06T22:15:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.872+0000 7f07d7ae2b80 1 client.25799 shutdown 2023-10-06T22:15:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 msgr2=0x55e1b77b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b5a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07b80985d0 tx=0x7f07b8099e70 comp rx=0 tx=0).stop 2023-10-06T22:15:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 shutdown_connections 2023-10-06T22:15:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b781fc30 0x55e1b7822050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b77bb3d0 0x55e1b77b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b77afc00 0x55e1b77b5a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07a57fa700 0 client.25799 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 --2- 172.21.15.57:0/1675066620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b77b0520 0x55e1b77b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 >> 172.21.15.57:0/1675066620 conn(0x55e1b7725200 msgr2=0x55e1b77abae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 shutdown_connections 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 1 -- 172.21.15.57:0/1675066620 wait complete. 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.873+0000 7f07d7ae2b80 20 client.25799 trim_cache size 0 max 16384 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:15:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:15:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:15:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:15:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07a80a16a0 msgr2=0x7f07a80a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07a80a16a0 0x7f07a80a3b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07bc198710 tx=0x7f07ac01e040 comp rx=0 tx=0).stop 2023-10-06T22:15:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc0682d0 msgr2=0x7f07bc196f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc0682d0 0x7f07bc196f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9a0 tx=0x7f07b800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 shutdown_connections 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07a80a16a0 0x7f07a80a3b60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07bc106f60 0x7f07bc19baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07bc0682d0 0x7f07bc196f60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 --2- 172.21.15.57:0/2511227687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07bc068c90 0x7f07bc1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 >> 172.21.15.57:0/2511227687 conn(0x7f07bc076b70 msgr2=0x7f07bc0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 shutdown_connections 2023-10-06T22:15:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:17.874+0000 7f07c41ff700 1 -- 172.21.15.57:0/2511227687 wait complete. 2023-10-06T22:15:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:17 smithi057 ceph-mon[98492]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T22:15:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:17 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6834/1662104434,v1:172.21.15.156:6835/1662104434] up:boot 2023-10-06T22:15:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:17 smithi057 ceph-mon[98492]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-10-06T22:15:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:17 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:15:18.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:17 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:15:18.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:17 smithi156 ceph-mon[134358]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T22:15:18.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:17 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6834/1662104434,v1:172.21.15.156:6835/1662104434] up:boot 2023-10-06T22:15:18.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:17 smithi156 ceph-mon[134358]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-10-06T22:15:18.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:17 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:15:18.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:17 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:15:18.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:17 smithi190 ceph-mon[134891]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T22:15:18.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:17 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6834/1662104434,v1:172.21.15.156:6835/1662104434] up:boot 2023-10-06T22:15:18.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:17 smithi190 ceph-mon[134891]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-10-06T22:15:18.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:17 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:15:18.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:17 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:15:19.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:18 smithi057 ceph-mon[98492]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:19.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:18 smithi156 ceph-mon[134358]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:19.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:18 smithi190 ceph-mon[134891]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:20 smithi057 ceph-mon[98492]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:21.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:20 smithi156 ceph-mon[134358]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:21.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:20 smithi190 ceph-mon[134891]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:23.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:22 smithi057 ceph-mon[98492]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:23.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:22 smithi156 ceph-mon[134358]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:23.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:22 smithi190 ceph-mon[134891]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 10 op/s 2023-10-06T22:15:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:24 smithi057 ceph-mon[98492]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-10-06T22:15:25.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:24 smithi156 ceph-mon[134358]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-10-06T22:15:25.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:24 smithi190 ceph-mon[134891]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-10-06T22:15:27.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:26 smithi057 ceph-mon[98492]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-10-06T22:15:27.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:26 smithi156 ceph-mon[134358]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-10-06T22:15:27.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:26 smithi190 ceph-mon[134891]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-10-06T22:15:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:28 smithi057 ceph-mon[98492]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-06T22:15:29.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:28 smithi156 ceph-mon[134358]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-06T22:15:29.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:28 smithi190 ceph-mon[134891]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-06T22:15:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.111+0000 7f98363ff700 1 -- 172.21.15.57:0/1457474115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9830106f80 msgr2=0x7f9830107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.111+0000 7f98363ff700 1 --2- 172.21.15.57:0/1457474115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9830106f80 0x7f9830107360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f98200099c0 tx=0x7f9820009cd0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.112+0000 7f98363ff700 1 -- 172.21.15.57:0/1457474115 shutdown_connections 2023-10-06T22:15:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.112+0000 7f98363ff700 1 --2- 172.21.15.57:0/1457474115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9830068c90 0x7f983010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.112+0000 7f98363ff700 1 --2- 172.21.15.57:0/1457474115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9830106f80 0x7f9830107360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.112+0000 7f98363ff700 1 --2- 172.21.15.57:0/1457474115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98300682d0 0x7f9830068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.112+0000 7f98363ff700 1 -- 172.21.15.57:0/1457474115 >> 172.21.15.57:0/1457474115 conn(0x7f9830076b70 msgr2=0x7f9830076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.112+0000 7f98363ff700 1 -- 172.21.15.57:0/1457474115 shutdown_connections 2023-10-06T22:15:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.112+0000 7f98363ff700 1 -- 172.21.15.57:0/1457474115 wait complete. 2023-10-06T22:15:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.113+0000 7f98363ff700 1 Processor -- start 2023-10-06T22:15:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.114+0000 7f98363ff700 1 -- start start 2023-10-06T22:15:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.114+0000 7f98363ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98300682d0 0x7f983010b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.114+0000 7f98363ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9830068c90 0x7f983010bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.114+0000 7f98363ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9830106f80 0x7f983019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.114+0000 7f98363ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983019b110 con 0x7f98300682d0 2023-10-06T22:15:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.114+0000 7f98363ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983019b280 con 0x7f9830068c90 2023-10-06T22:15:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.114+0000 7f98363ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983019b440 con 0x7f9830106f80 2023-10-06T22:15:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98300682d0 0x7f983010b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98300682d0 0x7f983010b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43824/0 (socket says 172.21.15.57:43824) 2023-10-06T22:15:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 -- 172.21.15.57:0/4222508315 learned_addr learned my addr 172.21.15.57:0/4222508315 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982f7fe700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9830068c90 0x7f983010bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f983499c700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9830106f80 0x7f983019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 -- 172.21.15.57:0/4222508315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9830106f80 msgr2=0x7f983019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9830106f80 0x7f983019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 -- 172.21.15.57:0/4222508315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9830068c90 msgr2=0x7f983010bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9830068c90 0x7f983010bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 -- 172.21.15.57:0/4222508315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98200095f0 con 0x7f98300682d0 2023-10-06T22:15:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f983499c700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9830106f80 0x7f983019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.115+0000 7f982ffff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98300682d0 0x7f983010b960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9820009990 tx=0x7f982000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.116+0000 7f982d7fa700 1 -- 172.21.15.57:0/4222508315 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982001d070 con 0x7f98300682d0 2023-10-06T22:15:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.116+0000 7f982d7fa700 1 -- 172.21.15.57:0/4222508315 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98200054f0 con 0x7f98300682d0 2023-10-06T22:15:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.116+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983019b6c0 con 0x7f98300682d0 2023-10-06T22:15:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.116+0000 7f982d7fa700 1 -- 172.21.15.57:0/4222508315 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820004ae0 con 0x7f98300682d0 2023-10-06T22:15:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.116+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983019ddc0 con 0x7f98300682d0 2023-10-06T22:15:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.122+0000 7f982d7fa700 1 -- 172.21.15.57:0/4222508315 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f982000fb80 con 0x7f98300682d0 2023-10-06T22:15:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.122+0000 7f9849c79b80 1 Processor -- start 2023-10-06T22:15:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.123+0000 7f9849c79b80 1 -- start start 2023-10-06T22:15:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.123+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc4169dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.123+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.123+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.123+0000 7f9849c79b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41607400 con 0x55dc4169e520 2023-10-06T22:15:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.123+0000 7f9849c79b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc415f8bc0 con 0x55dc416a93d0 2023-10-06T22:15:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.123+0000 7f9849c79b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc415f64e0 con 0x55dc4169dc00 2023-10-06T22:15:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.124+0000 7f982ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.124+0000 7f982ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43838/0 (socket says 172.21.15.57:43838) 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.124+0000 7f982ffff700 1 -- 172.21.15.57:0/611127921 learned_addr learned my addr 172.21.15.57:0/611127921 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.124+0000 7f982f7fe700 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.124+0000 7f983499c700 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc4169dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f983499c700 1 -- 172.21.15.57:0/611127921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 msgr2=0x55dc416ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f982d7fa700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f98180a1810 0x7f98180a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f983499c700 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f983499c700 1 -- 172.21.15.57:0/611127921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 msgr2=0x55dc416a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f983499c700 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f983499c700 1 -- 172.21.15.57:0/611127921 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc4141cd80 con 0x55dc4169dc00 2023-10-06T22:15:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f982d7fa700 1 -- 172.21.15.57:0/4222508315 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f98200ce120 con 0x7f98300682d0 2023-10-06T22:15:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f983499c700 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc4169dfe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98280084c0 tx=0x7f98280087d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6788eadafdc3a6c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.125+0000 7f982f7fe700 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.126+0000 7f983499c700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f98180a1810 0x7f98180a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.126+0000 7f98167fc700 1 -- 172.21.15.57:0/611127921 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280105c0 con 0x55dc4169dc00 2023-10-06T22:15:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.126+0000 7f98167fc700 1 -- 172.21.15.57:0/611127921 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9828010040 con 0x55dc4169dc00 2023-10-06T22:15:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.126+0000 7f98167fc700 1 -- 172.21.15.57:0/611127921 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828014b40 con 0x55dc4169dc00 2023-10-06T22:15:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 -- 172.21.15.57:0/611127921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 msgr2=0x55dc4169dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f983499c700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f98180a1810 0x7f98180a3cd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9828014fd0 tx=0x7f9828014ca0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc4169dfe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98280084c0 tx=0x7f98280087d0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 -- 172.21.15.57:0/611127921 shutdown_connections 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc4169dfe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 --2- 172.21.15.57:0/611127921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 -- 172.21.15.57:0/611127921 >> 172.21.15.57:0/611127921 conn(0x55dc41613200 msgr2=0x55dc4169a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 -- 172.21.15.57:0/611127921 shutdown_connections 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.127+0000 7f9849c79b80 1 -- 172.21.15.57:0/611127921 wait complete. 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.128+0000 7f9849c79b80 1 -- start start 2023-10-06T22:15:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.128+0000 7f9849c79b80 1 Processor -- start 2023-10-06T22:15:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.129+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169dc00 0x55dc416a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.129+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169e520 0x55dc416a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.129+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.129+0000 7f9849c79b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc415f1dd0 con 0x55dc4169dc00 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.129+0000 7f9849c79b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4158e6b0 con 0x55dc416a93d0 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.129+0000 7f9849c79b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4158e350 con 0x55dc4169e520 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.129+0000 7f9849c79b80 10 client.0 _ll_register_callbacks cb 0x55dc413eb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416a6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59852/0 (socket says 172.21.15.57:59852) 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 -- 172.21.15.57:0/152710367 learned_addr learned my addr 172.21.15.57:0/152710367 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f982ffff700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169dc00 0x55dc416a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f982f7fe700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169e520 0x55dc416a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 -- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169e520 msgr2=0x55dc416a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169e520 0x55dc416a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 -- 172.21.15.57:0/152710367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169dc00 msgr2=0x55dc416a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169dc00 0x55dc416a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 -- 172.21.15.57:0/152710367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98240098e0 con 0x55dc416a93d0 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f982f7fe700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169e520 0x55dc416a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.130+0000 7f983499c700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416a6c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9828011040 tx=0x7f9828013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91879216f0f46078 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.131+0000 7f980f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.131+0000 7f980f7fe700 1 -- 172.21.15.57:0/152710367 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280107d0 con 0x55dc416a93d0 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.131+0000 7f980f7fe700 1 -- 172.21.15.57:0/152710367 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9828027040 con 0x55dc416a93d0 2023-10-06T22:15:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.131+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc41691d50 con 0x55dc416a93d0 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.131+0000 7f980f7fe700 1 -- 172.21.15.57:0/152710367 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9828013530 con 0x55dc416a93d0 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.131+0000 7f9849c79b80 10 client.25811 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.131+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f9828007d00 con 0x55dc416a93d0 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.132+0000 7f980f7fe700 1 -- 172.21.15.57:0/152710367 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc41691d50 con 0x55dc416a93d0 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.132+0000 7f980f7fe700 1 -- 172.21.15.57:0/152710367 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f982802b600 con 0x55dc416a93d0 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.132+0000 7f9849c79b80 10 client.25811 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.133+0000 7f9849c79b80 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.133+0000 7f9849c79b80 20 client.25811 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.133+0000 7f9849c79b80 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc416b8f60 0x55dc416bb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.133+0000 7f9849c79b80 4 client.25811 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.133+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc415eadb0 con 0x55dc416b8f60 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.133+0000 7f982ffff700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc416b8f60 0x55dc416bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.134+0000 7f982ffff700 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc416b8f60 0x55dc416bb380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d91fa89ff70c61 server_cookie=1a4a8100b98d29e9 in_seq=0 out_seq=0 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.134+0000 7f980f7fe700 10 client.25811 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.134+0000 7f980f7fe700 1 -- 172.21.15.57:0/152710367 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc415eadb0 con 0x55dc416b8f60 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.134+0000 7f980f7fe700 10 client.25811 handle_command_reply: tid=0 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.134+0000 7f9849c79b80 1 client.25811 shutdown 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 msgr2=0x55dc416a6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416a6c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9828011040 tx=0x7f9828013000 comp rx=0 tx=0).stop 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 shutdown_connections 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc416b8f60 0x55dc416bb380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc416a6c00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169dc00 0x55dc416a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 --2- 172.21.15.57:0/152710367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169e520 0x55dc416a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 >> 172.21.15.57:0/152710367 conn(0x55dc41613200 msgr2=0x55dc41699c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f980f7fe700 0 client.25811 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 shutdown_connections 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.135+0000 7f9849c79b80 1 -- 172.21.15.57:0/152710367 wait complete. 2023-10-06T22:15:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.136+0000 7f9849c79b80 20 client.25811 trim_cache size 0 max 16384 2023-10-06T22:15:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.165+0000 7f9849c79b80 1 Processor -- start 2023-10-06T22:15:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.165+0000 7f9849c79b80 1 -- start start 2023-10-06T22:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc4169dc00 0x55dc416a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc416a93d0 0x55dc416a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f9849c79b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc414728c0 con 0x55dc4169e520 2023-10-06T22:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f9849c79b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc416a7760 con 0x55dc4169dc00 2023-10-06T22:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f9849c79b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41696860 con 0x55dc416a93d0 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f982ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc4169dc00 0x55dc416a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f982ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc4169dc00 0x55dc416a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59864/0 (socket says 172.21.15.57:59864) 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.166+0000 7f982ffff700 1 -- 172.21.15.57:0/2703915722 learned_addr learned my addr 172.21.15.57:0/2703915722 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f983499c700 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc416a93d0 0x55dc416a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f982f7fe700 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f983499c700 1 -- 172.21.15.57:0/2703915722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc4169dc00 msgr2=0x55dc416a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f983499c700 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc4169dc00 0x55dc416a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f983499c700 1 -- 172.21.15.57:0/2703915722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 msgr2=0x55dc416aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f983499c700 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f983499c700 1 -- 172.21.15.57:0/2703915722 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc41405c00 con 0x55dc416a93d0 2023-10-06T22:15:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.167+0000 7f983499c700 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc416a93d0 0x55dc416a33f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f982802ba10 tx=0x7f98280073c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d128bea051a8cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.168+0000 7f9816ffd700 1 -- 172.21.15.57:0/2703915722 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828013530 con 0x55dc416a93d0 2023-10-06T22:15:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.168+0000 7f9816ffd700 1 -- 172.21.15.57:0/2703915722 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9828013cd0 con 0x55dc416a93d0 2023-10-06T22:15:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.168+0000 7f9816ffd700 1 -- 172.21.15.57:0/2703915722 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828010a90 con 0x55dc416a93d0 2023-10-06T22:15:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.168+0000 7f9849c79b80 1 -- 172.21.15.57:0/2703915722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc416a93d0 msgr2=0x55dc416a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.168+0000 7f9849c79b80 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc416a93d0 0x55dc416a33f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f982802ba10 tx=0x7f98280073c0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 -- 172.21.15.57:0/2703915722 shutdown_connections 2023-10-06T22:15:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc416a93d0 0x55dc416a33f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc4169dc00 0x55dc416a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 --2- 172.21.15.57:0/2703915722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc416aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 -- 172.21.15.57:0/2703915722 >> 172.21.15.57:0/2703915722 conn(0x55dc41613200 msgr2=0x55dc416a1470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 -- 172.21.15.57:0/2703915722 shutdown_connections 2023-10-06T22:15:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 -- 172.21.15.57:0/2703915722 wait complete. 2023-10-06T22:15:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.169+0000 7f9849c79b80 1 -- start start 2023-10-06T22:15:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 1 Processor -- start 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc41700f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc41701450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc41705d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc417062b0 con 0x55dc4169e520 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc417063f0 con 0x55dc416a93d0 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc417065b0 con 0x55dc4169dc00 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.170+0000 7f9849c79b80 10 client.0 _ll_register_callbacks cb 0x7f98200d19c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc41700f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc41700f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57094/0 (socket says 172.21.15.57:57094) 2023-10-06T22:15:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 -- 172.21.15.57:0/1570994637 learned_addr learned my addr 172.21.15.57:0/1570994637 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982f7fe700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc41701450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f983499c700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc41705d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:30 smithi057 ceph-mon[98492]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T22:15:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 -- 172.21.15.57:0/1570994637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 msgr2=0x55dc41705d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc41705d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 -- 172.21.15.57:0/1570994637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 msgr2=0x55dc41701450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc41701450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f982ffff700 1 -- 172.21.15.57:0/1570994637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9828007d00 con 0x55dc4169dc00 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.171+0000 7f983499c700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc41705d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.172+0000 7f982ffff700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc41700f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f982009aee0 tx=0x7f98200d1af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24997e0a1fde35bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.172+0000 7f980f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.172+0000 7f980f7fe700 1 -- 172.21.15.57:0/1570994637 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982009a8b0 con 0x55dc4169dc00 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.172+0000 7f980f7fe700 1 -- 172.21.15.57:0/1570994637 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9820005660 con 0x55dc4169dc00 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.172+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9828010040 con 0x55dc4169dc00 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.173+0000 7f980f7fe700 1 -- 172.21.15.57:0/1570994637 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982009b650 con 0x55dc4169dc00 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.173+0000 7f9849c79b80 10 client.25822 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.173+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dc414431f0 con 0x55dc4169dc00 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.173+0000 7f980f7fe700 1 -- 172.21.15.57:0/1570994637 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98200b7080 con 0x55dc4169dc00 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.174+0000 7f980f7fe700 1 -- 172.21.15.57:0/1570994637 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f982009b240 con 0x55dc4169dc00 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.174+0000 7f9849c79b80 10 client.25822 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.174+0000 7f9849c79b80 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.174+0000 7f9849c79b80 20 client.25822 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.174+0000 7f9849c79b80 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc4170da90 0x55dc4170feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.174+0000 7f9849c79b80 4 client.25822 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:15:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.174+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc415eaad0 con 0x55dc4170da90 2023-10-06T22:15:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.175+0000 7f982f7fe700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc4170da90 0x55dc4170feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.175+0000 7f982f7fe700 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc4170da90 0x55dc4170feb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f5c43b847556bd7 server_cookie=73c477087fb580f3 in_seq=0 out_seq=0 2023-10-06T22:15:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.175+0000 7f980f7fe700 10 client.25822 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f980f7fe700 1 -- 172.21.15.57:0/1570994637 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc415eaad0 con 0x55dc4170da90 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f980f7fe700 10 client.25822 handle_command_reply: tid=0 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 client.25822 shutdown 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 msgr2=0x55dc41700f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc41700f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f982009aee0 tx=0x7f98200d1af0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 shutdown_connections 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55dc4170da90 0x55dc4170feb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc416a93d0 0x55dc41705d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc4169dc00 0x55dc41700f10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 --2- 172.21.15.57:0/1570994637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dc4169e520 0x55dc41701450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 >> 172.21.15.57:0/1570994637 conn(0x55dc41613200 msgr2=0x55dc416a7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.184+0000 7f980f7fe700 0 client.25822 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.185+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 shutdown_connections 2023-10-06T22:15:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.185+0000 7f9849c79b80 1 -- 172.21.15.57:0/1570994637 wait complete. 2023-10-06T22:15:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.185+0000 7f9849c79b80 20 client.25822 trim_cache size 0 max 16384 2023-10-06T22:15:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:15:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:15:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:15:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:15:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f98180a1810 msgr2=0x7f98180a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f98180a1810 0x7f98180a3cd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9828014fd0 tx=0x7f9828014ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98300682d0 msgr2=0x7f983010b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98300682d0 0x7f983010b960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9820009990 tx=0x7f982000f690 comp rx=0 tx=0).stop 2023-10-06T22:15:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 shutdown_connections 2023-10-06T22:15:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f98180a1810 0x7f98180a3cd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9830106f80 0x7f983019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98300682d0 0x7f983010b960 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 --2- 172.21.15.57:0/4222508315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9830068c90 0x7f983010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 >> 172.21.15.57:0/4222508315 conn(0x7f9830076b70 msgr2=0x7f9830100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 shutdown_connections 2023-10-06T22:15:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:31.187+0000 7f98363ff700 1 -- 172.21.15.57:0/4222508315 wait complete. 2023-10-06T22:15:31.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:30 smithi156 ceph-mon[134358]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T22:15:31.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:30 smithi190 ceph-mon[134891]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T22:15:33.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:32 smithi057 ceph-mon[98492]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T22:15:33.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:32 smithi156 ceph-mon[134358]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T22:15:33.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:32 smithi190 ceph-mon[134891]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2023-10-06T22:15:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:34 smithi057 ceph-mon[98492]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2023-10-06T22:15:35.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:34 smithi156 ceph-mon[134358]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2023-10-06T22:15:35.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:34 smithi190 ceph-mon[134891]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2023-10-06T22:15:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:36 smithi057 ceph-mon[98492]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-10-06T22:15:37.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:36 smithi156 ceph-mon[134358]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-10-06T22:15:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:36 smithi190 ceph-mon[134891]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-10-06T22:15:39.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:38 smithi057 ceph-mon[98492]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-10-06T22:15:39.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:38 smithi156 ceph-mon[134358]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-10-06T22:15:39.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:38 smithi190 ceph-mon[134891]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-10-06T22:15:41.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:40 smithi057 ceph-mon[98492]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-10-06T22:15:41.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:40 smithi156 ceph-mon[134358]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-10-06T22:15:41.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:40 smithi190 ceph-mon[134891]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-10-06T22:15:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:42 smithi057 ceph-mon[98492]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-10-06T22:15:43.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:42 smithi156 ceph-mon[134358]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-10-06T22:15:43.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:42 smithi190 ceph-mon[134891]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-10-06T22:15:43.636 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:15:43.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 -- 172.21.15.57:0/707090457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 msgr2=0x7fc10810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 --2- 172.21.15.57:0/707090457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 0x7fc10810adf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009990 tx=0x7fc0f8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 -- 172.21.15.57:0/707090457 shutdown_connections 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 --2- 172.21.15.57:0/707090457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10810b410 0x7fc10810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 --2- 172.21.15.57:0/707090457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc108101ff0 0x7fc1081023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 --2- 172.21.15.57:0/707090457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 0x7fc10810adf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 -- 172.21.15.57:0/707090457 >> 172.21.15.57:0/707090457 conn(0x7fc1080fb830 msgr2=0x7fc1080fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 -- 172.21.15.57:0/707090457 shutdown_connections 2023-10-06T22:15:43.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.948+0000 7fc1107ff700 1 -- 172.21.15.57:0/707090457 wait complete. 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 Processor -- start 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 -- start start 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc108101ff0 0x7fc10819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 0x7fc10819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc10810b410 0x7fc108194c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc108195320 con 0x7fc10810b410 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc108195490 con 0x7fc108102910 2023-10-06T22:15:43.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.950+0000 7fc1107ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc108195650 con 0x7fc108101ff0 2023-10-06T22:15:43.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.951+0000 7fc10ed9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc10810b410 0x7fc108194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.951+0000 7fc10ed9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc10810b410 0x7fc108194c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33716/0 (socket says 172.21.15.57:33716) 2023-10-06T22:15:43.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.951+0000 7fc10ed9c700 1 -- 172.21.15.57:0/2681001170 learned_addr learned my addr 172.21.15.57:0/2681001170 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:43.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.951+0000 7fc10e59b700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc108101ff0 0x7fc10819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.951+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 0x7fc10819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc10ed9c700 1 -- 172.21.15.57:0/2681001170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc108101ff0 msgr2=0x7fc10819bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc108101ff0 0x7fc10819bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc10ed9c700 1 -- 172.21.15.57:0/2681001170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 msgr2=0x7fc10819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 0x7fc10819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc10ed9c700 1 -- 172.21.15.57:0/2681001170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100009c40 con 0x7fc10810b410 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc10810b410 0x7fc108194c40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc10000c370 tx=0x7fc10000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:43.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc0ff7fe700 1 -- 172.21.15.57:0/2681001170 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10000e050 con 0x7fc10810b410 2023-10-06T22:15:43.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.952+0000 7fc0ff7fe700 1 -- 172.21.15.57:0/2681001170 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10000f040 con 0x7fc10810b410 2023-10-06T22:15:43.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.953+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f80095f0 con 0x7fc10810b410 2023-10-06T22:15:43.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.953+0000 7fc0ff7fe700 1 -- 172.21.15.57:0/2681001170 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100013660 con 0x7fc10810b410 2023-10-06T22:15:43.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.953+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1081997c0 con 0x7fc10810b410 2023-10-06T22:15:43.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.959+0000 7fc0ff7fe700 1 -- 172.21.15.57:0/2681001170 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fc1000090d0 con 0x7fc10810b410 2023-10-06T22:15:43.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.959+0000 7fc1240c7b80 1 Processor -- start 2023-10-06T22:15:43.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.959+0000 7fc1240c7b80 1 -- start start 2023-10-06T22:15:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.959+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a0c0 0x564874e3a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 0x564874e45350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e47c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874da3460 con 0x564874e3a0c0 2023-10-06T22:15:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874d94c20 con 0x564874e45890 2023-10-06T22:15:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874d92540 con 0x564874e3a9e0 2023-10-06T22:15:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc10dd9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a0c0 0x564874e3a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc10dd9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a0c0 0x564874e3a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33732/0 (socket says 172.21.15.57:33732) 2023-10-06T22:15:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc10dd9a700 1 -- 172.21.15.57:0/637795578 learned_addr learned my addr 172.21.15.57:0/637795578 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:43.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.960+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e47c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.961+0000 7fc10dd9a700 1 -- 172.21.15.57:0/637795578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 msgr2=0x564874e45350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:43.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.961+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 0x564874e45350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.961+0000 7fc10dd9a700 1 -- 172.21.15.57:0/637795578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a0c0 msgr2=0x564874e3a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:43.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.961+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a0c0 0x564874e3a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.961+0000 7fc10dd9a700 1 -- 172.21.15.57:0/637795578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564874bb8de0 con 0x564874e45890 2023-10-06T22:15:43.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.961+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e47c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009990 tx=0x7fc0f8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2ce659a8de79d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.962+0000 7fc0fcff9700 1 -- 172.21.15.57:0/637795578 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8005240 con 0x564874e45890 2023-10-06T22:15:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.962+0000 7fc0ff7fe700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc0f40a1790 0x7fc0f40a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.962+0000 7fc0fcff9700 1 -- 172.21.15.57:0/637795578 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f800be80 con 0x564874e45890 2023-10-06T22:15:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.962+0000 7fc0fcff9700 1 -- 172.21.15.57:0/637795578 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8004ce0 con 0x564874e45890 2023-10-06T22:15:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.962+0000 7fc0ff7fe700 1 -- 172.21.15.57:0/2681001170 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc1000cd080 con 0x7fc10810b410 2023-10-06T22:15:43.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc10e59b700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc0f40a1790 0x7fc0f40a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc1240c7b80 1 -- 172.21.15.57:0/637795578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 msgr2=0x564874e47c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:43.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e47c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009990 tx=0x7fc0f8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc1240c7b80 1 -- 172.21.15.57:0/637795578 shutdown_connections 2023-10-06T22:15:43.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e47c80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a0c0 0x564874e3a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/637795578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 0x564874e45350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.963+0000 7fc1240c7b80 1 -- 172.21.15.57:0/637795578 >> 172.21.15.57:0/637795578 conn(0x564874daf260 msgr2=0x564874e36450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.964+0000 7fc1240c7b80 1 -- 172.21.15.57:0/637795578 shutdown_connections 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.964+0000 7fc1240c7b80 1 -- 172.21.15.57:0/637795578 wait complete. 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.964+0000 7fc10e59b700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc0f40a1790 0x7fc0f40a3c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc10400ddb0 tx=0x7fc10400d670 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.964+0000 7fc1240c7b80 1 -- start start 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 1 Processor -- start 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e45350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 0x564874e425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e45890 0x564874e42d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874d8de30 con 0x564874e45890 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874d2a710 con 0x564874e3a0c0 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874d2a3b0 con 0x564874e3a9e0 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc1240c7b80 10 client.0 _ll_register_callbacks cb 0x564874b87800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.965+0000 7fc10e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e45350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e45350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57718/0 (socket says 172.21.15.57:57718) 2023-10-06T22:15:43.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10e59b700 1 -- 172.21.15.57:0/3073544825 learned_addr learned my addr 172.21.15.57:0/3073544825 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:43.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e45890 0x564874e42d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10ed9c700 1 -- 172.21.15.57:0/3073544825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 msgr2=0x564874e425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:43.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 0x564874e425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10ed9c700 1 -- 172.21.15.57:0/3073544825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 msgr2=0x564874e45350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:43.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e45350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10ed9c700 1 -- 172.21.15.57:0/3073544825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10810e590 con 0x564874e45890 2023-10-06T22:15:43.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10e59b700 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e45350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:43.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.966+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e45890 0x564874e42d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc1000ca660 tx=0x7fc10009a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7d3b7d5da519a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:43.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.967+0000 7fc0f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:43.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.967+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/3073544825 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1000b1070 con 0x564874e45890 2023-10-06T22:15:43.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.967+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/3073544825 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc10009a520 con 0x564874e45890 2023-10-06T22:15:43.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.967+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564874e2df10 con 0x564874e45890 2023-10-06T22:15:43.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.967+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/3073544825 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc100099e20 con 0x564874e45890 2023-10-06T22:15:43.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.967+0000 7fc1240c7b80 10 client.15825 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.967+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564874ba1c60 con 0x564874e45890 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/3073544825 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10007c020 con 0x564874e45890 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/3073544825 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc1000ca940 con 0x564874e45890 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc1240c7b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc1240c7b80 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc1240c7b80 20 client.15825 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874e55280 0x564874e576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc1240c7b80 4 client.15825 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.968+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564874d86e10 con 0x564874e55280 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.969+0000 7fc10e59b700 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874e55280 0x564874e576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.969+0000 7fc0f1ffb700 10 client.15825 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.969+0000 7fc10e59b700 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874e55280 0x564874e576a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8eb20d5c5cade9 server_cookie=31dac4d0dbf48dda in_seq=0 out_seq=0 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/3073544825 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564874d86e10 con 0x564874e55280 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc0f1ffb700 10 client.15825 handle_command_reply: tid=0 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 client.15825 shutdown 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e45890 msgr2=0x564874e42d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e45890 0x564874e42d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc1000ca660 tx=0x7fc10009a360 comp rx=0 tx=0).stop 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 shutdown_connections 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874e55280 0x564874e576a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e45890 0x564874e42d10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e45350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/3073544825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a9e0 0x564874e425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 >> 172.21.15.57:0/3073544825 conn(0x564874daf260 msgr2=0x564874e35fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:43.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.970+0000 7fc0f1ffb700 0 client.15825 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:43.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.971+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 shutdown_connections 2023-10-06T22:15:43.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.971+0000 7fc1240c7b80 1 -- 172.21.15.57:0/3073544825 wait complete. 2023-10-06T22:15:43.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.971+0000 7fc1240c7b80 20 client.15825 trim_cache size 0 max 16384 2023-10-06T22:15:43.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.989+0000 7fc1240c7b80 1 Processor -- start 2023-10-06T22:15:43.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.989+0000 7fc1240c7b80 1 -- start start 2023-10-06T22:15:43.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a0c0 0x564874e42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e3d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e44e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874c0e920 con 0x564874e3a9e0 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874e51c30 con 0x564874e45890 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874e3e480 con 0x564874e3a0c0 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc10e59b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a0c0 0x564874e42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc10e59b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a0c0 0x564874e42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58550/0 (socket says 172.21.15.57:58550) 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc10e59b700 1 -- 172.21.15.57:0/181488680 learned_addr learned my addr 172.21.15.57:0/181488680 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e44e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.990+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e3d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:43.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.991+0000 7fc10dd9a700 1 -- 172.21.15.57:0/181488680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a0c0 msgr2=0x564874e42580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:43.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.991+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a0c0 0x564874e42580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.991+0000 7fc10dd9a700 1 -- 172.21.15.57:0/181488680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 msgr2=0x564874e44e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:43.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.991+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e44e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.991+0000 7fc10dd9a700 1 -- 172.21.15.57:0/181488680 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564874bdf250 con 0x564874e3a9e0 2023-10-06T22:15:43.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.991+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e44e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:43.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.991+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e3d030 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8006010 tx=0x7fc0f80098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6edaed1412d2a03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:43.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.992+0000 7fc0fd7fa700 1 -- 172.21.15.57:0/181488680 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f802e070 con 0x564874e3a9e0 2023-10-06T22:15:43.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.992+0000 7fc0fd7fa700 1 -- 172.21.15.57:0/181488680 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f8005530 con 0x564874e3a9e0 2023-10-06T22:15:43.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.992+0000 7fc0fd7fa700 1 -- 172.21.15.57:0/181488680 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8004a20 con 0x564874e3a9e0 2023-10-06T22:15:43.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.992+0000 7fc1240c7b80 1 -- 172.21.15.57:0/181488680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 msgr2=0x564874e3d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:43.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.992+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e3d030 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8006010 tx=0x7fc0f80098c0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 -- 172.21.15.57:0/181488680 shutdown_connections 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e45890 0x564874e44e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e3a0c0 0x564874e42580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/181488680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e3d030 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 -- 172.21.15.57:0/181488680 >> 172.21.15.57:0/181488680 conn(0x564874daf260 msgr2=0x564874e41e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 -- 172.21.15.57:0/181488680 shutdown_connections 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 -- 172.21.15.57:0/181488680 wait complete. 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.993+0000 7fc1240c7b80 1 -- start start 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.994+0000 7fc1240c7b80 1 Processor -- start 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.994+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e4a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.994+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e4aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.994+0000 7fc1240c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e45890 0x564874ea80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:43.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.994+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874e4b200 con 0x564874e3a9e0 2023-10-06T22:15:43.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.994+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874ea8630 con 0x564874e3a0c0 2023-10-06T22:15:43.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.994+0000 7fc1240c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564874ea87a0 con 0x564874e45890 2023-10-06T22:15:43.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc1240c7b80 10 client.0 _ll_register_callbacks cb 0x7fc100096250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:44.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e4a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e4a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57748/0 (socket says 172.21.15.57:57748) 2023-10-06T22:15:44.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 -- 172.21.15.57:0/556918360 learned_addr learned my addr 172.21.15.57:0/556918360 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e4aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10ed9c700 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e45890 0x564874ea80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 -- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e45890 msgr2=0x564874ea80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e45890 0x564874ea80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 -- 172.21.15.57:0/556918360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 msgr2=0x564874e4aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e4aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.995+0000 7fc10e59b700 1 -- 172.21.15.57:0/556918360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564874bb8de0 con 0x564874e3a0c0 2023-10-06T22:15:44.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.996+0000 7fc10e59b700 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e4a550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc104028670 tx=0x7fc104014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5614b2c479779932 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.996+0000 7fc0f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.996+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/556918360 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104020030 con 0x564874e3a0c0 2023-10-06T22:15:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.996+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0f800be80 con 0x564874e3a0c0 2023-10-06T22:15:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.996+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/556918360 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc104028cc0 con 0x564874e3a0c0 2023-10-06T22:15:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/556918360 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc104028e80 con 0x564874e3a0c0 2023-10-06T22:15:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 10 client.25832 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:44.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564874bb72b0 con 0x564874e3a0c0 2023-10-06T22:15:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/556918360 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f800be80 con 0x564874e3a0c0 2023-10-06T22:15:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/556918360 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc104005580 con 0x564874e3a0c0 2023-10-06T22:15:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 20 client.25832 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:44.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874ea9eb0 0x564874eac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 4 client.25832 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:15:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.997+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564874d86b30 con 0x564874ea9eb0 2023-10-06T22:15:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.998+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874ea9eb0 0x564874eac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.998+0000 7fc10dd9a700 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874ea9eb0 0x564874eac2d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea9c14d9affed60 server_cookie=d6778bb0ab0d60a5 in_seq=0 out_seq=0 2023-10-06T22:15:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.998+0000 7fc0f1ffb700 10 client.25832 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc0f1ffb700 1 -- 172.21.15.57:0/556918360 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564874d86b30 con 0x564874ea9eb0 2023-10-06T22:15:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc0f1ffb700 10 client.25832 handle_command_reply: tid=0 2023-10-06T22:15:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 client.25832 shutdown 2023-10-06T22:15:44.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 msgr2=0x564874e4a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e4a550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc104028670 tx=0x7fc104014000 comp rx=0 tx=0).stop 2023-10-06T22:15:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 shutdown_connections 2023-10-06T22:15:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564874ea9eb0 0x564874eac2d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564874e45890 0x564874ea80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc0f1ffb700 0 client.25832 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:44.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564874e3a0c0 0x564874e4a550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 --2- 172.21.15.57:0/556918360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564874e3a9e0 0x564874e4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 >> 172.21.15.57:0/556918360 conn(0x564874daf260 msgr2=0x564874e35ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 shutdown_connections 2023-10-06T22:15:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:43.999+0000 7fc1240c7b80 1 -- 172.21.15.57:0/556918360 wait complete. 2023-10-06T22:15:44.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.000+0000 7fc1240c7b80 20 client.25832 trim_cache size 0 max 16384 2023-10-06T22:15:44.011 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:44.011 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-06T22:15:44.011 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-06T22:15:44.011 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.000+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc0f40a1790 msgr2=0x7fc0f40a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.000+0000 7fc1107ff700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc0f40a1790 0x7fc0f40a3c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc10400ddb0 tx=0x7fc10400d670 comp rx=0 tx=0).stop 2023-10-06T22:15:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.000+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc10810b410 msgr2=0x7fc108194c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.000+0000 7fc1107ff700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc10810b410 0x7fc108194c40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc10000c370 tx=0x7fc10000c680 comp rx=0 tx=0).stop 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.001+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 shutdown_connections 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.001+0000 7fc1107ff700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc0f40a1790 0x7fc0f40a3c50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.001+0000 7fc1107ff700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc10810b410 0x7fc108194c40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.001+0000 7fc1107ff700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc108101ff0 0x7fc10819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.001+0000 7fc1107ff700 1 --2- 172.21.15.57:0/2681001170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc108102910 0x7fc10819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.001+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 >> 172.21.15.57:0/2681001170 conn(0x7fc1080fb830 msgr2=0x7fc1080fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.002+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 shutdown_connections 2023-10-06T22:15:44.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.002+0000 7fc1107ff700 1 -- 172.21.15.57:0/2681001170 wait complete. 2023-10-06T22:15:44.033 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T22:15:44.033 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:15:44.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 -- 172.21.15.57:0/3829396942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f378410b410 msgr2=0x7f378410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 --2- 172.21.15.57:0/3829396942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f378410b410 0x7f378410d8d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f377c009f90 tx=0x7f377c00b330 comp rx=0 tx=0).stop 2023-10-06T22:15:44.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 -- 172.21.15.57:0/3829396942 shutdown_connections 2023-10-06T22:15:44.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 --2- 172.21.15.57:0/3829396942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f378410b410 0x7f378410d8d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 --2- 172.21.15.57:0/3829396942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3784101ff0 0x7f37841023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 --2- 172.21.15.57:0/3829396942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 0x7f378410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 -- 172.21.15.57:0/3829396942 >> 172.21.15.57:0/3829396942 conn(0x7f37840fb7f0 msgr2=0x7f37840fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 -- 172.21.15.57:0/3829396942 shutdown_connections 2023-10-06T22:15:44.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.251+0000 7f3789bff700 1 -- 172.21.15.57:0/3829396942 wait complete. 2023-10-06T22:15:44.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 Processor -- start 2023-10-06T22:15:44.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 -- start start 2023-10-06T22:15:44.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3784101ff0 0x7f37840ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 0x7f37841001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378410b410 0x7f3784071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37841009f0 con 0x7f3784102910 2023-10-06T22:15:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37840717c0 con 0x7f3784101ff0 2023-10-06T22:15:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.253+0000 7f3789bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784071980 con 0x7f378410b410 2023-10-06T22:15:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 0x7f37841001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 0x7f37841001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33776/0 (socket says 172.21.15.57:33776) 2023-10-06T22:15:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 -- 172.21.15.57:0/226175349 learned_addr learned my addr 172.21.15.57:0/226175349 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f37837fe700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3784101ff0 0x7f37840ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3783fff700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378410b410 0x7f3784071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 -- 172.21.15.57:0/226175349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378410b410 msgr2=0x7f3784071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378410b410 0x7f3784071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 -- 172.21.15.57:0/226175349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3784101ff0 msgr2=0x7f37840ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3784101ff0 0x7f37840ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f3782ffd700 1 -- 172.21.15.57:0/226175349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c009c40 con 0x7f3784102910 2023-10-06T22:15:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.254+0000 7f37837fe700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3784101ff0 0x7f37840ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.255+0000 7f3782ffd700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 0x7f37841001a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f377800e9f0 tx=0x7f377800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.255+0000 7f3780ff9700 1 -- 172.21.15.57:0/226175349 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37780096e0 con 0x7f3784102910 2023-10-06T22:15:44.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.255+0000 7f3780ff9700 1 -- 172.21.15.57:0/226175349 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f377800cbd0 con 0x7f3784102910 2023-10-06T22:15:44.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.255+0000 7f3780ff9700 1 -- 172.21.15.57:0/226175349 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3778018680 con 0x7f3784102910 2023-10-06T22:15:44.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.256+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3784071c60 con 0x7f3784102910 2023-10-06T22:15:44.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.256+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37840720f0 con 0x7f3784102910 2023-10-06T22:15:44.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.258+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3784067cd0 con 0x7f3784102910 2023-10-06T22:15:44.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.263+0000 7f3780ff9700 1 -- 172.21.15.57:0/226175349 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f377800cd40 con 0x7f3784102910 2023-10-06T22:15:44.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.265+0000 7f3780ff9700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f376c0a1860 0x7f376c0a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.265+0000 7f3780ff9700 1 -- 172.21.15.57:0/226175349 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f3778014070 con 0x7f3784102910 2023-10-06T22:15:44.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.266+0000 7f37837fe700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f376c0a1860 0x7f376c0a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.266+0000 7f3780ff9700 1 -- 172.21.15.57:0/226175349 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3778097110 con 0x7f3784102910 2023-10-06T22:15:44.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.267+0000 7f37837fe700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f376c0a1860 0x7f376c0a3d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3774005fd0 tx=0x7f3774009310 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 -- 172.21.15.57:0/2083062707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83dc10d6a0 msgr2=0x7f83dc10fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 --2- 172.21.15.57:0/2083062707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83dc10d6a0 0x7f83dc10fb10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83d400b0a0 tx=0x7f83d400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 -- 172.21.15.57:0/2083062707 shutdown_connections 2023-10-06T22:15:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 --2- 172.21.15.57:0/2083062707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83dc10d6a0 0x7f83dc10fb10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 --2- 172.21.15.57:0/2083062707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83dc0732e0 0x7f83dc0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 --2- 172.21.15.57:0/2083062707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc073c00 0x7f83dc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 -- 172.21.15.57:0/2083062707 >> 172.21.15.57:0/2083062707 conn(0x7f83dc0fc930 msgr2=0x7f83dc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 -- 172.21.15.57:0/2083062707 shutdown_connections 2023-10-06T22:15:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.437+0000 7f83e35ff700 1 -- 172.21.15.57:0/2083062707 wait complete. 2023-10-06T22:15:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 Processor -- start 2023-10-06T22:15:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 -- start start 2023-10-06T22:15:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc0732e0 0x7f83dc072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83dc073c00 0x7f83dc072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83dc10d6a0 0x7f83dc06b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc06bc60 con 0x7f83dc0732e0 2023-10-06T22:15:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc06bdd0 con 0x7f83dc10d6a0 2023-10-06T22:15:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.439+0000 7f83e35ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc06bf90 con 0x7f83dc073c00 2023-10-06T22:15:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.440+0000 7f83e139b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc0732e0 0x7f83dc072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.440+0000 7f83e139b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc0732e0 0x7f83dc072540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33806/0 (socket says 172.21.15.57:33806) 2023-10-06T22:15:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.440+0000 7f83e139b700 1 -- 172.21.15.57:0/3357664898 learned_addr learned my addr 172.21.15.57:0/3357664898 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.441+0000 7f83e1b9c700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83dc10d6a0 0x7f83dc06b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.441+0000 7f83e139b700 1 -- 172.21.15.57:0/3357664898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83dc073c00 msgr2=0x7f83dc072ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:15:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.441+0000 7f83e139b700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83dc073c00 0x7f83dc072ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.441+0000 7f83e139b700 1 -- 172.21.15.57:0/3357664898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83dc10d6a0 msgr2=0x7f83dc06b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.441+0000 7f83e139b700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83dc10d6a0 0x7f83dc06b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.441+0000 7f83e139b700 1 -- 172.21.15.57:0/3357664898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4009d00 con 0x7f83dc0732e0 2023-10-06T22:15:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.441+0000 7f83e139b700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc0732e0 0x7f83dc072540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9a0 tx=0x7f83d800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.442+0000 7f83d27fc700 1 -- 172.21.15.57:0/3357664898 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d800cbd0 con 0x7f83dc0732e0 2023-10-06T22:15:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.442+0000 7f83d27fc700 1 -- 172.21.15.57:0/3357664898 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d800cd30 con 0x7f83dc0732e0 2023-10-06T22:15:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.442+0000 7f83d27fc700 1 -- 172.21.15.57:0/3357664898 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8018710 con 0x7f83dc0732e0 2023-10-06T22:15:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.442+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc0700d0 con 0x7f83dc0732e0 2023-10-06T22:15:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.443+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc070590 con 0x7f83dc0732e0 2023-10-06T22:15:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.449+0000 7f83d27fc700 1 -- 172.21.15.57:0/3357664898 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f83d8010430 con 0x7f83dc0732e0 2023-10-06T22:15:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.449+0000 7f83f6e3fb80 1 Processor -- start 2023-10-06T22:15:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.450+0000 7f83f6e3fb80 1 -- start start 2023-10-06T22:15:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.450+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.450+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9111520 0x55cae91246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.450+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9124be0 0x55cae9126fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.450+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae907a400 con 0x55cae9124be0 2023-10-06T22:15:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.450+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae906bbc0 con 0x55cae9111520 2023-10-06T22:15:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.450+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae90694e0 con 0x55cae9110c00 2023-10-06T22:15:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.451+0000 7f83e139b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9111520 0x55cae91246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.451+0000 7f83e139b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9111520 0x55cae91246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57780/0 (socket says 172.21.15.57:57780) 2023-10-06T22:15:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.451+0000 7f83e139b700 1 -- 172.21.15.57:0/2199428560 learned_addr learned my addr 172.21.15.57:0/2199428560 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9124be0 0x55cae9126fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83e1b9c700 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83e139b700 1 -- 172.21.15.57:0/2199428560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 msgr2=0x55cae9110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83e139b700 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9110fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83e139b700 1 -- 172.21.15.57:0/2199428560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9124be0 msgr2=0x55cae9126fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83e139b700 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9124be0 0x55cae9126fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83d27fc700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f83c80a18e0 0x7f83c80a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.452+0000 7f83e139b700 1 -- 172.21.15.57:0/2199428560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cae8e8fd80 con 0x55cae9111520 2023-10-06T22:15:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.453+0000 7f83e139b700 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9111520 0x55cae91246a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83d80cadb0 tx=0x7f83d8096910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d89bea50660f13d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.453+0000 7f83e1b9c700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f83c80a18e0 0x7f83c80a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.453+0000 7f83d27fc700 1 -- 172.21.15.57:0/3357664898 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f83d8014070 con 0x7f83dc0732e0 2023-10-06T22:15:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.453+0000 7f83c77fe700 1 -- 172.21.15.57:0/2199428560 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d809a3a0 con 0x55cae9111520 2023-10-06T22:15:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.453+0000 7f83c77fe700 1 -- 172.21.15.57:0/2199428560 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d809a110 con 0x55cae9111520 2023-10-06T22:15:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.453+0000 7f83c77fe700 1 -- 172.21.15.57:0/2199428560 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d809fa60 con 0x55cae9111520 2023-10-06T22:15:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.454+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/2199428560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9111520 msgr2=0x55cae91246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.454+0000 7f83e1b9c700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f83c80a18e0 0x7f83c80a3da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83d400b070 tx=0x7f83d400b740 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.454+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9111520 0x55cae91246a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83d80cadb0 tx=0x7f83d8096910 comp rx=0 tx=0).stop 2023-10-06T22:15:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.455+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/2199428560 shutdown_connections 2023-10-06T22:15:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.455+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9124be0 0x55cae9126fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.455+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.455+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/2199428560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9111520 0x55cae91246a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.455+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/2199428560 >> 172.21.15.57:0/2199428560 conn(0x55cae9086200 msgr2=0x55cae910d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.455+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/2199428560 shutdown_connections 2023-10-06T22:15:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.455+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/2199428560 wait complete. 2023-10-06T22:15:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.456+0000 7f83f6e3fb80 1 -- start start 2023-10-06T22:15:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.456+0000 7f83f6e3fb80 1 Processor -- start 2023-10-06T22:15:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9124660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae91218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae9121fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae9064dd0 con 0x55cae9111520 2023-10-06T22:15:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae90016b0 con 0x55cae9124be0 2023-10-06T22:15:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae9001350 con 0x55cae9110c00 2023-10-06T22:15:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83f6e3fb80 10 client.0 _ll_register_callbacks cb 0x55cae8e5e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.457+0000 7f83e139b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9124660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e139b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9124660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58630/0 (socket says 172.21.15.57:58630) 2023-10-06T22:15:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e139b700 1 -- 172.21.15.57:0/311993866 learned_addr learned my addr 172.21.15.57:0/311993866 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae91218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e1b9c700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae9121fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e0b9a700 1 -- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 msgr2=0x55cae9124660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9124660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e0b9a700 1 -- 172.21.15.57:0/311993866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 msgr2=0x55cae9121fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae9121fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.458+0000 7f83e0b9a700 1 -- 172.21.15.57:0/311993866 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc070590 con 0x55cae9111520 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.459+0000 7f83e139b700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9124660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.459+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae91218e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00dd10 tx=0x7f83cc00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a15c347ad3e09e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.459+0000 7f83c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.459+0000 7f83c4ff9700 1 -- 172.21.15.57:0/311993866 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc004a30 con 0x55cae9111520 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.459+0000 7f83c4ff9700 1 -- 172.21.15.57:0/311993866 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f83cc00de60 con 0x55cae9111520 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.460+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cae9104d50 con 0x55cae9111520 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.460+0000 7f83c4ff9700 1 -- 172.21.15.57:0/311993866 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83cc00fe60 con 0x55cae9111520 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.460+0000 7f83f6e3fb80 10 client.15849 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.460+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cae8e78c00 con 0x55cae9111520 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.461+0000 7f83c4ff9700 1 -- 172.21.15.57:0/311993866 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cae9104d50 con 0x55cae9111520 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.461+0000 7f83c4ff9700 1 -- 172.21.15.57:0/311993866 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f83cc011550 con 0x55cae9111520 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.461+0000 7f83f6e3fb80 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.461+0000 7f83f6e3fb80 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.461+0000 7f83f6e3fb80 20 client.15849 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.462+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9134790 0x55cae9136b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.462+0000 7f83f6e3fb80 4 client.15849 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.462+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cae905ddb0 con 0x55cae9134790 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.462+0000 7f83e139b700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9134790 0x55cae9136b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.463+0000 7f83e139b700 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9134790 0x55cae9136b80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e957c69d6047c8 server_cookie=1e5b1e94e6fd76dc in_seq=0 out_seq=0 2023-10-06T22:15:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.463+0000 7f83c4ff9700 10 client.15849 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.463+0000 7f83c4ff9700 1 -- 172.21.15.57:0/311993866 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cae905ddb0 con 0x55cae9134790 2023-10-06T22:15:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.463+0000 7f83c4ff9700 10 client.15849 handle_command_reply: tid=0 2023-10-06T22:15:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.463+0000 7f83f6e3fb80 1 client.15849 shutdown 2023-10-06T22:15:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 msgr2=0x55cae91218e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae91218e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00dd10 tx=0x7f83cc00f640 comp rx=0 tx=0).stop 2023-10-06T22:15:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 shutdown_connections 2023-10-06T22:15:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9134790 0x55cae9136b80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae9121fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9124660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/311993866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae91218e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 >> 172.21.15.57:0/311993866 conn(0x55cae9086200 msgr2=0x55cae910cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83c4ff9700 0 client.15849 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 shutdown_connections 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.464+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/311993866 wait complete. 2023-10-06T22:15:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.465+0000 7f83f6e3fb80 20 client.15849 trim_cache size 0 max 16384 2023-10-06T22:15:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.498+0000 7f83f6e3fb80 1 Processor -- start 2023-10-06T22:15:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.498+0000 7f83f6e3fb80 1 -- start start 2023-10-06T22:15:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.499+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9110c00 0x55cae912b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.499+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9111520 0x55cae9124290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.499+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.499+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae8ee58c0 con 0x55cae9110c00 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.499+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae912bc70 con 0x55cae9124be0 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.499+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae910be30 con 0x55cae9111520 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9110c00 0x55cae912b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9110c00 0x55cae912b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33826/0 (socket says 172.21.15.57:33826) 2023-10-06T22:15:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 -- 172.21.15.57:0/3906132826 learned_addr learned my addr 172.21.15.57:0/3906132826 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e1b9c700 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 -- 172.21.15.57:0/3906132826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9111520 msgr2=0x55cae9124290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:15:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9111520 0x55cae9124290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 -- 172.21.15.57:0/3906132826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 msgr2=0x55cae912fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.500+0000 7f83e139b700 1 -- 172.21.15.57:0/3906132826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cae8eb61f0 con 0x55cae9110c00 2023-10-06T22:15:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.501+0000 7f83e139b700 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9110c00 0x55cae912b730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f83d800f010 tx=0x7f83d8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171cd0d66d327add server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.501+0000 7f83c7fff700 1 -- 172.21.15.57:0/3906132826 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d809a3a0 con 0x55cae9110c00 2023-10-06T22:15:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.501+0000 7f83c7fff700 1 -- 172.21.15.57:0/3906132826 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d8099f30 con 0x55cae9110c00 2023-10-06T22:15:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.501+0000 7f83c7fff700 1 -- 172.21.15.57:0/3906132826 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80a29f0 con 0x55cae9110c00 2023-10-06T22:15:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/3906132826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9110c00 msgr2=0x55cae912b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9110c00 0x55cae912b730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f83d800f010 tx=0x7f83d8010d50 comp rx=0 tx=0).stop 2023-10-06T22:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/3906132826 shutdown_connections 2023-10-06T22:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9110c00 0x55cae912b730 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/3906132826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9111520 0x55cae9124290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/3906132826 >> 172.21.15.57:0/3906132826 conn(0x55cae9086200 msgr2=0x55cae91367a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.502+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/3906132826 shutdown_connections 2023-10-06T22:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.503+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/3906132826 wait complete. 2023-10-06T22:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.503+0000 7f83f6e3fb80 1 -- start start 2023-10-06T22:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.503+0000 7f83f6e3fb80 1 Processor -- start 2023-10-06T22:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.504+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9129710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.504+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae9129c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.504+0000 7f83f6e3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.504+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae91875f0 con 0x55cae9111520 2023-10-06T22:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.504+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae9187760 con 0x55cae9124be0 2023-10-06T22:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.504+0000 7f83f6e3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cae9187920 con 0x55cae9110c00 2023-10-06T22:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.504+0000 7f83f6e3fb80 10 client.0 _ll_register_callbacks cb 0x7f83cc00d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.505+0000 7f83e139b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9129710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.505+0000 7f83e0b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae9129c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.505+0000 7f83e0b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae9129c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33838/0 (socket says 172.21.15.57:33838) 2023-10-06T22:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.505+0000 7f83e0b9a700 1 -- 172.21.15.57:0/412844790 learned_addr learned my addr 172.21.15.57:0/412844790 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.505+0000 7f83e1b9c700 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.506+0000 7f83e0b9a700 1 -- 172.21.15.57:0/412844790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 msgr2=0x55cae9129710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.506+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9129710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.506+0000 7f83e0b9a700 1 -- 172.21.15.57:0/412844790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 msgr2=0x55cae912a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.506+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.506+0000 7f83e0b9a700 1 -- 172.21.15.57:0/412844790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cae8e8fd80 con 0x55cae9111520 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.506+0000 7f83e0b9a700 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae9129c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f83cc011980 tx=0x7f83cc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96fd6edb5526424 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.506+0000 7f83c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.507+0000 7f83c4ff9700 1 -- 172.21.15.57:0/412844790 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc005450 con 0x55cae9111520 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.507+0000 7f83c4ff9700 1 -- 172.21.15.57:0/412844790 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f83cc00fe60 con 0x55cae9111520 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.507+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83d809a110 con 0x55cae9111520 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.507+0000 7f83c4ff9700 1 -- 172.21.15.57:0/412844790 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83cc00de60 con 0x55cae9111520 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.507+0000 7f83f6e3fb80 10 client.15855 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.507+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cae8e8e250 con 0x55cae9111520 2023-10-06T22:15:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83c4ff9700 1 -- 172.21.15.57:0/412844790 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d809a110 con 0x55cae9111520 2023-10-06T22:15:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83c4ff9700 1 -- 172.21.15.57:0/412844790 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f83cc024430 con 0x55cae9111520 2023-10-06T22:15:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83f6e3fb80 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83f6e3fb80 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83f6e3fb80 20 client.15855 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9189020 0x55cae918b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83f6e3fb80 4 client.15855 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:15:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.508+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cae905dad0 con 0x55cae9189020 2023-10-06T22:15:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.509+0000 7f83e139b700 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9189020 0x55cae918b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.509+0000 7f83e139b700 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9189020 0x55cae918b440 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee541455ead2896 server_cookie=b797c4e7bba46c93 in_seq=0 out_seq=0 2023-10-06T22:15:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.509+0000 7f83c4ff9700 10 client.15855 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.513+0000 7f83c4ff9700 1 -- 172.21.15.57:0/412844790 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cae905dad0 con 0x55cae9189020 2023-10-06T22:15:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.513+0000 7f83c4ff9700 10 client.15855 handle_command_reply: tid=0 2023-10-06T22:15:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.514+0000 7f83f6e3fb80 1 client.15855 shutdown 2023-10-06T22:15:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.514+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 msgr2=0x55cae9129c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.514+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae9129c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f83cc011980 tx=0x7f83cc005970 comp rx=0 tx=0).stop 2023-10-06T22:15:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 shutdown_connections 2023-10-06T22:15:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cae9189020 0x55cae918b440 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cae9124be0 0x55cae912a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cae9110c00 0x55cae9129710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 --2- 172.21.15.57:0/412844790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cae9111520 0x55cae9129c50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 >> 172.21.15.57:0/412844790 conn(0x55cae9086200 msgr2=0x55cae911cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83c4ff9700 0 client.15855 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 shutdown_connections 2023-10-06T22:15:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.515+0000 7f83f6e3fb80 1 -- 172.21.15.57:0/412844790 wait complete. 2023-10-06T22:15:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.516+0000 7f83f6e3fb80 20 client.15855 trim_cache size 0 max 16384 2023-10-06T22:15:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:15:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:15:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:15:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:15:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.516+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f83c80a18e0 msgr2=0x7f83c80a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.516+0000 7f83e35ff700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f83c80a18e0 0x7f83c80a3da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83d400b070 tx=0x7f83d400b740 comp rx=0 tx=0).stop 2023-10-06T22:15:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.516+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc0732e0 msgr2=0x7f83dc072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.516+0000 7f83e35ff700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc0732e0 0x7f83dc072540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9a0 tx=0x7f83d800ed60 comp rx=0 tx=0).stop 2023-10-06T22:15:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 shutdown_connections 2023-10-06T22:15:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f83c80a18e0 0x7f83c80a3da0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83dc10d6a0 0x7f83dc06b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83dc0732e0 0x7f83dc072540 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 --2- 172.21.15.57:0/3357664898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83dc073c00 0x7f83dc072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 >> 172.21.15.57:0/3357664898 conn(0x7f83dc0fc930 msgr2=0x7f83dc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 shutdown_connections 2023-10-06T22:15:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:44.517+0000 7f83e35ff700 1 -- 172.21.15.57:0/3357664898 wait complete. 2023-10-06T22:15:44.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.588+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3784101190 con 0x7f3784102910 2023-10-06T22:15:44.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.589+0000 7f3780ff9700 1 -- 172.21.15.57:0/226175349 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f377809a730 con 0x7f3784102910 2023-10-06T22:15:44.591 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:15:44.592 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:15:16.820723+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25784":{"gid":25784,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/1662104434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1662104434},{"type":"v1","addr":"172.21.15.156:6835","nonce":1662104434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:15:44.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.592+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f376c0a1860 msgr2=0x7f376c0a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.592+0000 7f3789bff700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f376c0a1860 0x7f376c0a3d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3774005fd0 tx=0x7f3774009310 comp rx=0 tx=0).stop 2023-10-06T22:15:44.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.592+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 msgr2=0x7f37841001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.592+0000 7f3789bff700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 0x7f37841001a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f377800e9f0 tx=0x7f377800edb0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 shutdown_connections 2023-10-06T22:15:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f376c0a1860 0x7f376c0a3d20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378410b410 0x7f3784071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3784101ff0 0x7f37840ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 --2- 172.21.15.57:0/226175349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3784102910 0x7f37841001a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 >> 172.21.15.57:0/226175349 conn(0x7f37840fb7f0 msgr2=0x7f3784102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 shutdown_connections 2023-10-06T22:15:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.593+0000 7f3789bff700 1 -- 172.21.15.57:0/226175349 wait complete. 2023-10-06T22:15:44.598 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 28 2023-10-06T22:15:44.623 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T22:15:44.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 -- 172.21.15.57:0/475008044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c10d530 msgr2=0x7f921c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 --2- 172.21.15.57:0/475008044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c10d530 0x7f921c10f940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f921400b0a0 tx=0x7f921400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 -- 172.21.15.57:0/475008044 shutdown_connections 2023-10-06T22:15:44.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 --2- 172.21.15.57:0/475008044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c10d530 0x7f921c10f940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 --2- 172.21.15.57:0/475008044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c101ba0 0x7f921c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 --2- 172.21.15.57:0/475008044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921c068580 0x7f921c068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 -- 172.21.15.57:0/475008044 >> 172.21.15.57:0/475008044 conn(0x7f921c0769d0 msgr2=0x7f921c076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 -- 172.21.15.57:0/475008044 shutdown_connections 2023-10-06T22:15:44.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.836+0000 7f9220d9d700 1 -- 172.21.15.57:0/475008044 wait complete. 2023-10-06T22:15:44.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.837+0000 7f9220d9d700 1 Processor -- start 2023-10-06T22:15:44.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.838+0000 7f9220d9d700 1 -- start start 2023-10-06T22:15:44.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.838+0000 7f9220d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c068580 0x7f921c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.838+0000 7f9220d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c101ba0 0x7f921c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.838+0000 7f9220d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921c10d530 0x7f921c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.838+0000 7f9220d9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c19e130 con 0x7f921c10d530 2023-10-06T22:15:44.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.838+0000 7f9220d9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c19e2a0 con 0x7f921c101ba0 2023-10-06T22:15:44.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.838+0000 7f9220d9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c19e460 con 0x7f921c068580 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c101ba0 0x7f921c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c068580 0x7f921c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c101ba0 0x7f921c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57836/0 (socket says 172.21.15.57:57836) 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921affd700 1 -- 172.21.15.57:0/2759309583 learned_addr learned my addr 172.21.15.57:0/2759309583 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921bfff700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921c10d530 0x7f921c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921b7fe700 1 -- 172.21.15.57:0/2759309583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c101ba0 msgr2=0x7f921c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921b7fe700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c101ba0 0x7f921c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921b7fe700 1 -- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921c10d530 msgr2=0x7f921c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921b7fe700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921c10d530 0x7f921c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921b7fe700 1 -- 172.21.15.57:0/2759309583 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9214009d00 con 0x7f921c068580 2023-10-06T22:15:44.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.839+0000 7f921bfff700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921c10d530 0x7f921c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:44.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.840+0000 7f921b7fe700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c068580 0x7f921c199150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f920c00e9f0 tx=0x7f920c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.840+0000 7f9218ff9700 1 -- 172.21.15.57:0/2759309583 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0096d0 con 0x7f921c068580 2023-10-06T22:15:44.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.840+0000 7f9218ff9700 1 -- 172.21.15.57:0/2759309583 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920c00cbd0 con 0x7f921c068580 2023-10-06T22:15:44.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.840+0000 7f9218ff9700 1 -- 172.21.15.57:0/2759309583 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c018680 con 0x7f921c068580 2023-10-06T22:15:44.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.840+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c19e740 con 0x7f921c068580 2023-10-06T22:15:44.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.840+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c1a5f80 con 0x7f921c068580 2023-10-06T22:15:44.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.844+0000 7f9218ff9700 1 -- 172.21.15.57:0/2759309583 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f920c00cd40 con 0x7f921c068580 2023-10-06T22:15:44.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.845+0000 7f9218ff9700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f92040a1830 0x7f92040a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.845+0000 7f9218ff9700 1 -- 172.21.15.57:0/2759309583 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f920c014070 con 0x7f921c068580 2023-10-06T22:15:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.846+0000 7f921affd700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f92040a1830 0x7f92040a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.846+0000 7f921affd700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f92040a1830 0x7f92040a3cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f921c19a760 tx=0x7f9210009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.847+0000 7f9235872b80 1 Processor -- start 2023-10-06T22:15:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.847+0000 7f9235872b80 1 -- start start 2023-10-06T22:15:44.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.847+0000 7f9235872b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9097e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.847+0000 7f9235872b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90989530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f9235872b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 0x559c9098be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f9235872b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c908e7460 con 0x559c90989a70 2023-10-06T22:15:44.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f9235872b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c908d8c20 con 0x559c9097e2a0 2023-10-06T22:15:44.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f9235872b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c908d6540 con 0x559c9097ebc0 2023-10-06T22:15:44.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90989530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9097e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9097e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57852/0 (socket says 172.21.15.57:57852) 2023-10-06T22:15:44.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90989530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58680/0 (socket says 172.21.15.57:58680) 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921b7fe700 1 -- 172.21.15.57:0/3804760498 learned_addr learned my addr 172.21.15.57:0/3804760498 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921bfff700 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 0x559c9098be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921affd700 1 -- 172.21.15.57:0/3804760498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 msgr2=0x559c9097e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921affd700 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9097e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921affd700 1 -- 172.21.15.57:0/3804760498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 msgr2=0x559c9098be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921affd700 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 0x559c9098be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921affd700 1 -- 172.21.15.57:0/3804760498 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c906fcde0 con 0x559c9097ebc0 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.848+0000 7f921b7fe700 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9097e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.849+0000 7f921affd700 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90989530 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9210011ab0 tx=0x7f9210014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9072cc43ffb8375e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.849+0000 7f9201ffb700 1 -- 172.21.15.57:0/3804760498 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000ee20 con 0x559c9097ebc0 2023-10-06T22:15:44.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.849+0000 7f9201ffb700 1 -- 172.21.15.57:0/3804760498 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9210021070 con 0x559c9097ebc0 2023-10-06T22:15:44.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.849+0000 7f9201ffb700 1 -- 172.21.15.57:0/3804760498 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921001d830 con 0x559c9097ebc0 2023-10-06T22:15:44.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 -- 172.21.15.57:0/3804760498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 msgr2=0x559c90989530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90989530 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9210011ab0 tx=0x7f9210014000 comp rx=0 tx=0).stop 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 -- 172.21.15.57:0/3804760498 shutdown_connections 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 0x559c9098be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9097e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 --2- 172.21.15.57:0/3804760498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90989530 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 -- 172.21.15.57:0/3804760498 >> 172.21.15.57:0/3804760498 conn(0x559c908f3260 msgr2=0x559c9097a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 -- 172.21.15.57:0/3804760498 shutdown_connections 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.850+0000 7f9235872b80 1 -- 172.21.15.57:0/3804760498 wait complete. 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.851+0000 7f9235872b80 1 -- start start 2023-10-06T22:15:44.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 1 Processor -- start 2023-10-06T22:15:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097e2a0 0x559c90989490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097ebc0 0x559c90986710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c908d1e30 con 0x559c9097ebc0 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9086e710 con 0x559c90989a70 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9086e3b0 con 0x559c9097e2a0 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.852+0000 7f9235872b80 10 client.0 _ll_register_callbacks cb 0x559c906cb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097e2a0 0x559c90989490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097e2a0 0x559c90989490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58686/0 (socket says 172.21.15.57:58686) 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 -- 172.21.15.57:0/303229734 learned_addr learned my addr 172.21.15.57:0/303229734 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921affd700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097ebc0 0x559c90986710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921bfff700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 -- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 msgr2=0x559c90986db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 -- 172.21.15.57:0/303229734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097ebc0 msgr2=0x559c90986710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097ebc0 0x559c90986710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921b7fe700 1 -- 172.21.15.57:0/303229734 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c1a5f80 con 0x559c9097e2a0 2023-10-06T22:15:44.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.853+0000 7f921bfff700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f921b7fe700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097e2a0 0x559c90989490 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f920c099f30 tx=0x7f920c09a770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c920c3ffe7c806cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f91faffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f91faffd700 1 -- 172.21.15.57:0/303229734 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0a2330 con 0x559c9097e2a0 2023-10-06T22:15:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c90972160 con 0x559c9097e2a0 2023-10-06T22:15:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f91faffd700 1 -- 172.21.15.57:0/303229734 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f920c09a230 con 0x559c9097e2a0 2023-10-06T22:15:44.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f91faffd700 1 -- 172.21.15.57:0/303229734 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f920c09e380 con 0x559c9097e2a0 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f9235872b80 10 client.25858 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f9210017070 con 0x559c9097e2a0 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.854+0000 7f91faffd700 1 -- 172.21.15.57:0/303229734 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c90972160 con 0x559c9097e2a0 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f91faffd700 1 -- 172.21.15.57:0/303229734 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f920c0cade0 con 0x559c9097e2a0 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f9235872b80 10 client.25858 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f9235872b80 10 client.25858 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f9235872b80 20 client.25858 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f9235872b80 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c90999530 0x559c9099b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f9235872b80 4 client.25858 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c908cae10 con 0x559c90999530 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.855+0000 7f921affd700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c90999530 0x559c9099b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.856+0000 7f921affd700 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c90999530 0x559c9099b920 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56f253d022f109e8 server_cookie=1ac2b79e603f8c6b in_seq=0 out_seq=0 2023-10-06T22:15:44.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.856+0000 7f91faffd700 10 client.25858 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f91faffd700 1 -- 172.21.15.57:0/303229734 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c908cae10 con 0x559c90999530 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f91faffd700 10 client.25858 handle_command_reply: tid=0 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 client.25858 shutdown 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097e2a0 msgr2=0x559c90989490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097e2a0 0x559c90989490 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f920c099f30 tx=0x7f920c09a770 comp rx=0 tx=0).stop 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 shutdown_connections 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c90999530 0x559c9099b920 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097e2a0 0x559c90989490 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 --2- 172.21.15.57:0/303229734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097ebc0 0x559c90986710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 >> 172.21.15.57:0/303229734 conn(0x559c908f3260 msgr2=0x559c9097a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f91faffd700 0 client.25858 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:44.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 shutdown_connections 2023-10-06T22:15:44.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.857+0000 7f9235872b80 1 -- 172.21.15.57:0/303229734 wait complete. 2023-10-06T22:15:44.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.858+0000 7f9235872b80 20 client.25858 trim_cache size 0 max 16384 2023-10-06T22:15:44.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 Processor -- start 2023-10-06T22:15:44.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 -- start start 2023-10-06T22:15:44.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097e2a0 0x559c90989030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90985ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c90752920 con 0x559c9097e2a0 2023-10-06T22:15:44.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c90974720 con 0x559c90989a70 2023-10-06T22:15:44.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.872+0000 7f9235872b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c90971ca0 con 0x559c9097ebc0 2023-10-06T22:15:44.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097e2a0 0x559c90989030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097e2a0 0x559c90989030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33876/0 (socket says 172.21.15.57:33876) 2023-10-06T22:15:44.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921b7fe700 1 -- 172.21.15.57:0/44175316 learned_addr learned my addr 172.21.15.57:0/44175316 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921affd700 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90985ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921bfff700 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921affd700 1 -- 172.21.15.57:0/44175316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 msgr2=0x559c90986430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921affd700 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921affd700 1 -- 172.21.15.57:0/44175316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097e2a0 msgr2=0x559c90989030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921affd700 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097e2a0 0x559c90989030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921affd700 1 -- 172.21.15.57:0/44175316 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c906e5c60 con 0x559c9097ebc0 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.873+0000 7f921b7fe700 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097e2a0 0x559c90989030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.874+0000 7f921affd700 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90985ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f921001d450 tx=0x7f9210005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b41f0fc7f6338e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.874+0000 7f92027fc700 1 -- 172.21.15.57:0/44175316 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000ec20 con 0x559c9097ebc0 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.874+0000 7f92027fc700 1 -- 172.21.15.57:0/44175316 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f921001cdd0 con 0x559c9097ebc0 2023-10-06T22:15:44.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.874+0000 7f92027fc700 1 -- 172.21.15.57:0/44175316 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210004710 con 0x559c9097ebc0 2023-10-06T22:15:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 -- 172.21.15.57:0/44175316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 msgr2=0x559c90985ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90985ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f921001d450 tx=0x7f9210005ce0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 -- 172.21.15.57:0/44175316 shutdown_connections 2023-10-06T22:15:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c90989a70 0x559c90986430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c9097e2a0 0x559c90989030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 --2- 172.21.15.57:0/44175316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c90985ef0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 -- 172.21.15.57:0/44175316 >> 172.21.15.57:0/44175316 conn(0x559c908f3260 msgr2=0x559c90981a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 -- 172.21.15.57:0/44175316 shutdown_connections 2023-10-06T22:15:44.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 -- 172.21.15.57:0/44175316 wait complete. 2023-10-06T22:15:44.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.875+0000 7f9235872b80 1 -- start start 2023-10-06T22:15:44.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 1 Processor -- start 2023-10-06T22:15:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9098f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c9098fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 0x559c909ec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c90990660 con 0x559c90989a70 2023-10-06T22:15:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c909ec840 con 0x559c9097e2a0 2023-10-06T22:15:44.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c909ec980 con 0x559c9097ebc0 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.876+0000 7f9235872b80 10 client.0 _ll_register_callbacks cb 0x7f920c099c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c9098fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c9098fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58706/0 (socket says 172.21.15.57:58706) 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 -- 172.21.15.57:0/265519356 learned_addr learned my addr 172.21.15.57:0/265519356 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921b7fe700 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9098f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 -- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 msgr2=0x559c9098f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9098f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 -- 172.21.15.57:0/265519356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 msgr2=0x559c909ec300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 0x559c909ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 -- 172.21.15.57:0/265519356 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9210017070 con 0x559c9097ebc0 2023-10-06T22:15:44.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921b7fe700 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9098f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f921affd700 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c9098fd20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f921000e480 tx=0x7f921000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=373c29d3c0cc4f3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.877+0000 7f91faffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.878+0000 7f91faffd700 1 -- 172.21.15.57:0/265519356 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210005440 con 0x559c9097ebc0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.878+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9210021070 con 0x559c9097ebc0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.878+0000 7f91faffd700 1 -- 172.21.15.57:0/265519356 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f92100055a0 con 0x559c9097ebc0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.878+0000 7f91faffd700 1 -- 172.21.15.57:0/265519356 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9210033a90 con 0x559c9097ebc0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.878+0000 7f9235872b80 10 client.25864 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.878+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f921002a040 con 0x559c9097ebc0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.878+0000 7f91faffd700 1 -- 172.21.15.57:0/265519356 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f921004a080 con 0x559c9097ebc0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f91faffd700 1 -- 172.21.15.57:0/265519356 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f921001c750 con 0x559c9097ebc0 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f9235872b80 10 client.25864 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:44.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f9235872b80 10 client.25864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:15:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f9235872b80 20 client.25864 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f9235872b80 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c909edd80 0x559c909f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f9235872b80 4 client.25864 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c908cab30 con 0x559c909edd80 2023-10-06T22:15:44.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.879+0000 7f921b7fe700 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c909edd80 0x559c909f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:44.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f921b7fe700 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c909edd80 0x559c909f01a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58ad5615d52985eb server_cookie=394dea026f45aa4d in_seq=0 out_seq=0 2023-10-06T22:15:44.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f91faffd700 10 client.25864 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:44.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f91faffd700 1 -- 172.21.15.57:0/265519356 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c908cab30 con 0x559c909edd80 2023-10-06T22:15:44.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f91faffd700 10 client.25864 handle_command_reply: tid=0 2023-10-06T22:15:44.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 client.25864 shutdown 2023-10-06T22:15:44.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 msgr2=0x559c9098fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c9098fd20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f921000e480 tx=0x7f921000ec90 comp rx=0 tx=0).stop 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 shutdown_connections 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x559c909edd80 0x559c909f01a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559c90989a70 0x559c909ec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c9097e2a0 0x559c9098f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f91faffd700 0 client.25864 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 --2- 172.21.15.57:0/265519356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c9097ebc0 0x559c9098fd20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 >> 172.21.15.57:0/265519356 conn(0x559c908f3260 msgr2=0x559c90979880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 shutdown_connections 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.880+0000 7f9235872b80 1 -- 172.21.15.57:0/265519356 wait complete. 2023-10-06T22:15:44.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9235872b80 20 client.25864 trim_cache size 0 max 16384 2023-10-06T22:15:44.887 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:44.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f92040a1830 msgr2=0x7f92040a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f92040a1830 0x7f92040a3cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f921c19a760 tx=0x7f9210009260 comp rx=0 tx=0).stop 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c068580 msgr2=0x7f921c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c068580 0x7f921c199150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f920c00e9f0 tx=0x7f920c00ed00 comp rx=0 tx=0).stop 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 shutdown_connections 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f92040a1830 0x7f92040a3cf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921c10d530 0x7f921c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f921c068580 0x7f921c199150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 --2- 172.21.15.57:0/2759309583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f921c101ba0 0x7f921c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.881+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 >> 172.21.15.57:0/2759309583 conn(0x7f921c0769d0 msgr2=0x7f921c10f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.882+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 shutdown_connections 2023-10-06T22:15:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:44.882+0000 7f9220d9d700 1 -- 172.21.15.57:0/2759309583 wait complete. 2023-10-06T22:15:44.910 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T22:15:45.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 -- 172.21.15.57:0/3029945981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a5c101ff0 msgr2=0x7f0a5c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 --2- 172.21.15.57:0/3029945981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a5c101ff0 0x7f0a5c1023d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009990 tx=0x7f0a44009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 -- 172.21.15.57:0/3029945981 shutdown_connections 2023-10-06T22:15:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 --2- 172.21.15.57:0/3029945981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c10b410 0x7f0a5c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 --2- 172.21.15.57:0/3029945981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a5c101ff0 0x7f0a5c1023d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 --2- 172.21.15.57:0/3029945981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a5c102910 0x7f0a5c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 -- 172.21.15.57:0/3029945981 >> 172.21.15.57:0/3029945981 conn(0x7f0a5c0fb830 msgr2=0x7f0a5c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 -- 172.21.15.57:0/3029945981 shutdown_connections 2023-10-06T22:15:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.130+0000 7f0a615ff700 1 -- 172.21.15.57:0/3029945981 wait complete. 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.131+0000 7f0a615ff700 1 Processor -- start 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.131+0000 7f0a615ff700 1 -- start start 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a615ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a5c101ff0 0x7f0a5c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a615ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c102910 0x7f0a5c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a615ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a5c10b410 0x7f0a5c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a615ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c19e230 con 0x7f0a5c101ff0 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a615ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c19e3a0 con 0x7f0a5c10b410 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a615ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c19e560 con 0x7f0a5c102910 2023-10-06T22:15:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c102910 0x7f0a5c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c102910 0x7f0a5c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58738/0 (socket says 172.21.15.57:58738) 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1970990067 learned_addr learned my addr 172.21.15.57:0/1970990067 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.132+0000 7f0a5b7fe700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a5c10b410 0x7f0a5c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1970990067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a5c10b410 msgr2=0x7f0a5c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a5c10b410 0x7f0a5c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a5c101ff0 msgr2=0x7f0a5c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a5c101ff0 0x7f0a5c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1970990067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440095f0 con 0x7f0a5c102910 2023-10-06T22:15:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c102910 0x7f0a5c199790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00e9f0 tx=0x7f0a4c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a53fff700 1 -- 172.21.15.57:0/1970990067 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00c9c0 con 0x7f0a5c102910 2023-10-06T22:15:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.133+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c19e840 con 0x7f0a5c102910 2023-10-06T22:15:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.134+0000 7f0a53fff700 1 -- 172.21.15.57:0/1970990067 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a4c010430 con 0x7f0a5c102910 2023-10-06T22:15:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.134+0000 7f0a53fff700 1 -- 172.21.15.57:0/1970990067 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c018680 con 0x7f0a5c102910 2023-10-06T22:15:45.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.134+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c1a6080 con 0x7f0a5c102910 2023-10-06T22:15:45.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.138+0000 7f0a53fff700 1 -- 172.21.15.57:0/1970990067 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f0a4c010860 con 0x7f0a5c102910 2023-10-06T22:15:45.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.139+0000 7f0a53fff700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0a480a1790 0x7f0a480a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.139+0000 7f0a53fff700 1 -- 172.21.15.57:0/1970990067 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0a4c014070 con 0x7f0a5c102910 2023-10-06T22:15:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.140+0000 7f0a5affd700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0a480a1790 0x7f0a480a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.140+0000 7f0a74e4db80 1 Processor -- start 2023-10-06T22:15:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a74e4db80 1 -- start start 2023-10-06T22:15:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a5affd700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0a480a1790 0x7f0a480a3c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b2e0 tx=0x7f0a44005fd0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c174a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c22350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c24c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14b80460 con 0x557c14c22890 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14b71c20 con 0x557c14c179e0 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.141+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14b6f540 con 0x557c14c170c0 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c174a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58750/0 (socket says 172.21.15.57:58750) 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 -- 172.21.15.57:0/3297558196 learned_addr learned my addr 172.21.15.57:0/3297558196 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5b7fe700 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c24c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c22350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 -- 172.21.15.57:0/3297558196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 msgr2=0x557c14c22350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c22350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 -- 172.21.15.57:0/3297558196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 msgr2=0x557c14c24c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c24c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5affd700 1 -- 172.21.15.57:0/3297558196 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c14995de0 con 0x557c14c170c0 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.142+0000 7f0a5b7fe700 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c24c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.143+0000 7f0a5affd700 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c174a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a44005790 tx=0x7f0a4401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c84f6c0e8ded62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.143+0000 7f0a517fa700 1 -- 172.21.15.57:0/3297558196 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4401d070 con 0x557c14c170c0 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.143+0000 7f0a517fa700 1 -- 172.21.15.57:0/3297558196 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a4400be70 con 0x557c14c170c0 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.143+0000 7f0a517fa700 1 -- 172.21.15.57:0/3297558196 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4402fc90 con 0x557c14c170c0 2023-10-06T22:15:45.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 -- 172.21.15.57:0/3297558196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 msgr2=0x557c14c174a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c174a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a44005790 tx=0x7f0a4401d000 comp rx=0 tx=0).stop 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 -- 172.21.15.57:0/3297558196 shutdown_connections 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c24c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c174a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/3297558196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c22350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 -- 172.21.15.57:0/3297558196 >> 172.21.15.57:0/3297558196 conn(0x557c14b8c260 msgr2=0x557c14c13450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 -- 172.21.15.57:0/3297558196 shutdown_connections 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.144+0000 7f0a74e4db80 1 -- 172.21.15.57:0/3297558196 wait complete. 2023-10-06T22:15:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.145+0000 7f0a74e4db80 1 -- start start 2023-10-06T22:15:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.145+0000 7f0a74e4db80 1 Processor -- start 2023-10-06T22:15:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c22340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c1f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c1fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14b6ae30 con 0x557c14c22890 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14b07710 con 0x557c14c179e0 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14b073b0 con 0x557c14c170c0 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a74e4db80 10 client.0 _ll_register_callbacks cb 0x557c14964800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c1f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c1f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57912/0 (socket says 172.21.15.57:57912) 2023-10-06T22:15:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.147+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1742409598 learned_addr learned my addr 172.21.15.57:0/1742409598 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1742409598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 msgr2=0x557c14c22340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c22340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1742409598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 msgr2=0x557c14c1fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c1fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/1742409598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c10e590 con 0x557c14c179e0 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c1f550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00c5b0 tx=0x7f0a4c0107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab31778bccad2bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a427fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a427fc700 1 -- 172.21.15.57:0/1742409598 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0a23c0 con 0x557c14c179e0 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c14c0af10 con 0x557c14c179e0 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a427fc700 1 -- 172.21.15.57:0/1742409598 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0a4c09a2c0 con 0x557c14c179e0 2023-10-06T22:15:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a427fc700 1 -- 172.21.15.57:0/1742409598 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a4c09e410 con 0x557c14c179e0 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a74e4db80 10 client.25874 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.148+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c1497ec60 con 0x557c14c179e0 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a427fc700 1 -- 172.21.15.57:0/1742409598 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c14c0af10 con 0x557c14c179e0 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a427fc700 1 -- 172.21.15.57:0/1742409598 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a4c0b8320 con 0x557c14c179e0 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a74e4db80 10 client.25874 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a74e4db80 10 client.25874 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a74e4db80 20 client.25874 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c322f0 0x557c14c34710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a74e4db80 4 client.25874 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c14b63e10 con 0x557c14c322f0 2023-10-06T22:15:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.149+0000 7f0a5affd700 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c322f0 0x557c14c34710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.150+0000 7f0a427fc700 10 client.25874 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.150+0000 7f0a5affd700 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c322f0 0x557c14c34710 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=953a67b305f4f75b server_cookie=76692d178c7bd0ff in_seq=0 out_seq=0 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a427fc700 1 -- 172.21.15.57:0/1742409598 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c14b63e10 con 0x557c14c322f0 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a427fc700 10 client.25874 handle_command_reply: tid=0 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 client.25874 shutdown 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 msgr2=0x557c14c1f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c1f550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00c5b0 tx=0x7f0a4c0107d0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 shutdown_connections 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c322f0 0x557c14c34710 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c1fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c22340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/1742409598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c1f550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 >> 172.21.15.57:0/1742409598 conn(0x557c14b8c260 msgr2=0x557c14c12fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.151+0000 7f0a427fc700 0 client.25874 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.152+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 shutdown_connections 2023-10-06T22:15:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.152+0000 7f0a74e4db80 1 -- 172.21.15.57:0/1742409598 wait complete. 2023-10-06T22:15:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.152+0000 7f0a74e4db80 20 client.25874 trim_cache size 0 max 16384 2023-10-06T22:15:45.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.167+0000 7f0a74e4db80 1 Processor -- start 2023-10-06T22:15:45.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.167+0000 7f0a74e4db80 1 -- start start 2023-10-06T22:15:45.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c19fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c21e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c149eb920 con 0x557c14c22890 2023-10-06T22:15:45.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14c0d520 con 0x557c14c179e0 2023-10-06T22:15:45.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14c1b400 con 0x557c14c170c0 2023-10-06T22:15:45.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c19fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c19fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57928/0 (socket says 172.21.15.57:57928) 2023-10-06T22:15:45.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.168+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/389333809 learned_addr learned my addr 172.21.15.57:0/389333809 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/389333809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 msgr2=0x557c14c1f500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/389333809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c1f500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/389333809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 msgr2=0x557c14c21e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/389333809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c21e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a5a7fc700 1 -- 172.21.15.57:0/389333809 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c149bc250 con 0x557c14c179e0 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/389333809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c19fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c098610 tx=0x7f0a4c099d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6340065c753dd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a51ffb700 1 -- 172.21.15.57:0/389333809 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0cd3a0 con 0x557c14c179e0 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a51ffb700 1 -- 172.21.15.57:0/389333809 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a4c09a2c0 con 0x557c14c179e0 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.169+0000 7f0a51ffb700 1 -- 172.21.15.57:0/389333809 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0cade0 con 0x557c14c179e0 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 -- 172.21.15.57:0/389333809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 msgr2=0x557c14c19fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/389333809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c19fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c098610 tx=0x7f0a4c099d50 comp rx=0 tx=0).stop 2023-10-06T22:15:45.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 -- 172.21.15.57:0/389333809 shutdown_connections 2023-10-06T22:15:45.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/389333809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c21e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/389333809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c170c0 0x557c14c1f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/389333809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c179e0 0x557c14c19fc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 -- 172.21.15.57:0/389333809 >> 172.21.15.57:0/389333809 conn(0x557c14b8c260 msgr2=0x557c14c1edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 -- 172.21.15.57:0/389333809 shutdown_connections 2023-10-06T22:15:45.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.170+0000 7f0a74e4db80 1 -- 172.21.15.57:0/389333809 wait complete. 2023-10-06T22:15:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:44 smithi057 ceph-mon[98492]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-10-06T22:15:45.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:44 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/226175349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:45.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.171+0000 7f0a74e4db80 1 -- start start 2023-10-06T22:15:45.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.171+0000 7f0a74e4db80 1 Processor -- start 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c170c0 0x557c14c290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c179e0 0x557c14c27400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a74e4db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c27940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14c27e80 con 0x557c14c22890 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14c27ff0 con 0x557c14c170c0 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a74e4db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c14c281b0 con 0x557c14c179e0 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a74e4db80 10 client.0 _ll_register_callbacks cb 0x557c14964800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a5affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c170c0 0x557c14c290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a5affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c170c0 0x557c14c290a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57930/0 (socket says 172.21.15.57:57930) 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a5affd700 1 -- 172.21.15.57:0/838413337 learned_addr learned my addr 172.21.15.57:0/838413337 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a5b7fe700 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c27940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.172+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c179e0 0x557c14c27400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.173+0000 7f0a5affd700 1 -- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c179e0 msgr2=0x557c14c27400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.173+0000 7f0a5affd700 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c179e0 0x557c14c27400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.173+0000 7f0a5affd700 1 -- 172.21.15.57:0/838413337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 msgr2=0x557c14c27940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.173+0000 7f0a5affd700 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c27940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.173+0000 7f0a5affd700 1 -- 172.21.15.57:0/838413337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c1497ec60 con 0x557c14c170c0 2023-10-06T22:15:45.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.173+0000 7f0a5affd700 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c170c0 0x557c14c290a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098e0 tx=0x7f0a4402f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e6340aeb30191a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.173+0000 7f0a427fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:45.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.174+0000 7f0a427fc700 1 -- 172.21.15.57:0/838413337 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4403c070 con 0x557c14c170c0 2023-10-06T22:15:45.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.174+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a4400be70 con 0x557c14c170c0 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.174+0000 7f0a427fc700 1 -- 172.21.15.57:0/838413337 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0a440362c0 con 0x557c14c170c0 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.174+0000 7f0a427fc700 1 -- 172.21.15.57:0/838413337 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a44034040 con 0x557c14c170c0 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.174+0000 7f0a74e4db80 10 client.25880 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.174+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c149942b0 con 0x557c14c170c0 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a427fc700 1 -- 172.21.15.57:0/838413337 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a44045020 con 0x557c14c170c0 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a427fc700 1 -- 172.21.15.57:0/838413337 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a4403f030 con 0x557c14c170c0 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a74e4db80 10 client.25880 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a74e4db80 10 client.25880 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a74e4db80 20 client.25880 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c8f8a0 0x557c14c91c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a74e4db80 4 client.25880 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c14b63b30 con 0x557c14c8f8a0 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.175+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c8f8a0 0x557c14c91c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.176+0000 7f0a5a7fc700 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c8f8a0 0x557c14c91c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d2715524d37bdae server_cookie=e0837f1bb01188e1 in_seq=0 out_seq=0 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.176+0000 7f0a427fc700 10 client.25880 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.176+0000 7f0a427fc700 1 -- 172.21.15.57:0/838413337 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c14b63b30 con 0x557c14c8f8a0 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.176+0000 7f0a427fc700 10 client.25880 handle_command_reply: tid=0 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.176+0000 7f0a74e4db80 1 client.25880 shutdown 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c170c0 msgr2=0x557c14c290a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c170c0 0x557c14c290a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098e0 tx=0x7f0a4402f890 comp rx=0 tx=0).stop 2023-10-06T22:15:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 shutdown_connections 2023-10-06T22:15:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x557c14c8f8a0 0x557c14c91c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557c14c22890 0x557c14c27940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c14c170c0 0x557c14c290a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 --2- 172.21.15.57:0/838413337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557c14c179e0 0x557c14c27400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 >> 172.21.15.57:0/838413337 conn(0x557c14b8c260 msgr2=0x557c14c12ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a427fc700 0 client.25880 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:15:45.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 shutdown_connections 2023-10-06T22:15:45.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 1 -- 172.21.15.57:0/838413337 wait complete. 2023-10-06T22:15:45.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.177+0000 7f0a74e4db80 20 client.25880 trim_cache size 0 max 16384 2023-10-06T22:15:45.182 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0a480a1790 msgr2=0x7f0a480a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0a480a1790 0x7f0a480a3c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b2e0 tx=0x7f0a44005fd0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c102910 msgr2=0x7f0a5c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c102910 0x7f0a5c199790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00e9f0 tx=0x7f0a4c00ed00 comp rx=0 tx=0).stop 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 shutdown_connections 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0a480a1790 0x7f0a480a3c50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a5c10b410 0x7f0a5c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a5c101ff0 0x7f0a5c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 --2- 172.21.15.57:0/1970990067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a5c102910 0x7f0a5c199790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.178+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 >> 172.21.15.57:0/1970990067 conn(0x7f0a5c0fb830 msgr2=0x7f0a5c10d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.179+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 shutdown_connections 2023-10-06T22:15:45.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.179+0000 7f0a615ff700 1 -- 172.21.15.57:0/1970990067 wait complete. 2023-10-06T22:15:45.201 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T22:15:45.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:44 smithi156 ceph-mon[134358]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-10-06T22:15:45.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:44 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/226175349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:45.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:44 smithi190 ceph-mon[134891]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-10-06T22:15:45.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:44 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/226175349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:15:45.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 -- 172.21.15.57:0/864198564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8900ffe30 msgr2=0x7fa8901002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 --2- 172.21.15.57:0/864198564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8900ffe30 0x7fa8901002b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa880009990 tx=0x7fa880009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 -- 172.21.15.57:0/864198564 shutdown_connections 2023-10-06T22:15:45.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 --2- 172.21.15.57:0/864198564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8901007f0 0x7fa89010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 --2- 172.21.15.57:0/864198564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa890105c70 0x7fa890106050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 --2- 172.21.15.57:0/864198564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8900ffe30 0x7fa8901002b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 -- 172.21.15.57:0/864198564 >> 172.21.15.57:0/864198564 conn(0x7fa8900769d0 msgr2=0x7fa890076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 -- 172.21.15.57:0/864198564 shutdown_connections 2023-10-06T22:15:45.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.425+0000 7fa89599d700 1 -- 172.21.15.57:0/864198564 wait complete. 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.427+0000 7fa89599d700 1 Processor -- start 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.427+0000 7fa89599d700 1 -- start start 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89599d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8900ffe30 0x7fa890197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89599d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8901007f0 0x7fa890197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89599d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa890105c70 0x7fa8901a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89599d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901a3070 con 0x7fa8900ffe30 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89599d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901a31b0 con 0x7fa890105c70 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89599d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901a32f0 con 0x7fa8901007f0 2023-10-06T22:15:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa887fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8901007f0 0x7fa890197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa887fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8901007f0 0x7fa890197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58792/0 (socket says 172.21.15.57:58792) 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa887fff700 1 -- 172.21.15.57:0/3417198696 learned_addr learned my addr 172.21.15.57:0/3417198696 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89519c700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa890105c70 0x7fa8901a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.428+0000 7fa89499b700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8900ffe30 0x7fa890197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa887fff700 1 -- 172.21.15.57:0/3417198696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa890105c70 msgr2=0x7fa8901a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa887fff700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa890105c70 0x7fa8901a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa887fff700 1 -- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8900ffe30 msgr2=0x7fa890197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa887fff700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8900ffe30 0x7fa890197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa887fff700 1 -- 172.21.15.57:0/3417198696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800095f0 con 0x7fa8901007f0 2023-10-06T22:15:45.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa887fff700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8901007f0 0x7fa890197ff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa88000b650 tx=0x7fa8800057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa885ffb700 1 -- 172.21.15.57:0/3417198696 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88001d070 con 0x7fa8901007f0 2023-10-06T22:15:45.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8901a34f0 con 0x7fa8901007f0 2023-10-06T22:15:45.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.429+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8901a3910 con 0x7fa8901007f0 2023-10-06T22:15:45.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.431+0000 7fa885ffb700 1 -- 172.21.15.57:0/3417198696 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa880022470 con 0x7fa8901007f0 2023-10-06T22:15:45.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.431+0000 7fa885ffb700 1 -- 172.21.15.57:0/3417198696 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880017650 con 0x7fa8901007f0 2023-10-06T22:15:45.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.434+0000 7fa8aa565b80 1 Processor -- start 2023-10-06T22:15:45.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.435+0000 7fa8aa565b80 1 -- start start 2023-10-06T22:15:45.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.435+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.435+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4295bf0 0x55f7f4296070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.435+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f429c900 0x55f7f429ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.435+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f41fa460 con 0x55f7f4297240 2023-10-06T22:15:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.435+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f41ebc20 con 0x55f7f4295bf0 2023-10-06T22:15:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.435+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f41e9540 con 0x55f7f429c900 2023-10-06T22:15:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.437+0000 7fa89519c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.437+0000 7fa89519c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33966/0 (socket says 172.21.15.57:33966) 2023-10-06T22:15:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.437+0000 7fa89519c700 1 -- 172.21.15.57:0/4031593517 learned_addr learned my addr 172.21.15.57:0/4031593517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.438+0000 7fa885ffb700 1 -- 172.21.15.57:0/3417198696 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fa88000fa90 con 0x7fa8901007f0 2023-10-06T22:15:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.439+0000 7fa887fff700 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f429c900 0x55f7f429ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.439+0000 7fa89519c700 1 -- 172.21.15.57:0/4031593517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f429c900 msgr2=0x55f7f429ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.439+0000 7fa89519c700 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f429c900 0x55f7f429ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.439+0000 7fa89519c700 1 -- 172.21.15.57:0/4031593517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4295bf0 msgr2=0x55f7f4296070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.439+0000 7fa89499b700 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4295bf0 0x55f7f4296070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.439+0000 7fa89519c700 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4295bf0 0x55f7f4296070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.439+0000 7fa89519c700 1 -- 172.21.15.57:0/4031593517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7f400fde0 con 0x55f7f4297240 2023-10-06T22:15:45.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.440+0000 7fa885ffb700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa8780a1830 0x7fa8780a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.440+0000 7fa89519c700 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42956b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa888009440 tx=0x7fa888009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe8eaf469927e57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.440+0000 7fa885ffb700 1 -- 172.21.15.57:0/3417198696 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fa8800ce800 con 0x7fa8901007f0 2023-10-06T22:15:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.441+0000 7fa876ffd700 1 -- 172.21.15.57:0/4031593517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800e070 con 0x55f7f4297240 2023-10-06T22:15:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.441+0000 7fa876ffd700 1 -- 172.21.15.57:0/4031593517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa88800f040 con 0x55f7f4297240 2023-10-06T22:15:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.441+0000 7fa876ffd700 1 -- 172.21.15.57:0/4031593517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888015b60 con 0x55f7f4297240 2023-10-06T22:15:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.441+0000 7fa89499b700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa8780a1830 0x7fa8780a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.441+0000 7fa8aa565b80 1 -- 172.21.15.57:0/4031593517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 msgr2=0x55f7f42956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.441+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42956b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa888009440 tx=0x7fa888009920 comp rx=0 tx=0).stop 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 -- 172.21.15.57:0/4031593517 shutdown_connections 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f429c900 0x55f7f429ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42956b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/4031593517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4295bf0 0x55f7f4296070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 -- 172.21.15.57:0/4031593517 >> 172.21.15.57:0/4031593517 conn(0x55f7f4206260 msgr2=0x55f7f428d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 -- 172.21.15.57:0/4031593517 shutdown_connections 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 -- 172.21.15.57:0/4031593517 wait complete. 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa89499b700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa8780a1830 0x7fa8780a3cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa88c005950 tx=0x7fa88c0058e0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.442+0000 7fa8aa565b80 1 -- start start 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 1 Processor -- start 2023-10-06T22:15:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f428f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f428f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f42b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f41e4e30 con 0x55f7f429c900 2023-10-06T22:15:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f4181710 con 0x55f7f4297240 2023-10-06T22:15:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f41813b0 con 0x55f7f4295bf0 2023-10-06T22:15:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.443+0000 7fa8aa565b80 10 client.0 _ll_register_callbacks cb 0x55f7f3fde800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f42b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f42b9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33980/0 (socket says 172.21.15.57:33980) 2023-10-06T22:15:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 -- 172.21.15.57:0/751127246 learned_addr learned my addr 172.21.15.57:0/751127246 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89499b700 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f428f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 -- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 msgr2=0x55f7f428f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f428f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 -- 172.21.15.57:0/751127246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 msgr2=0x55f7f428f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f428f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.444+0000 7fa89519c700 1 -- 172.21.15.57:0/751127246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8880090f0 con 0x55f7f429c900 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.445+0000 7fa89519c700 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f42b9140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d50 tx=0x7fa888016f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2effaee26c9ab3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.445+0000 7fa86ffff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.445+0000 7fa86ffff700 1 -- 172.21.15.57:0/751127246 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800e040 con 0x55f7f429c900 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.445+0000 7fa86ffff700 1 -- 172.21.15.57:0/751127246 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fa8880154c0 con 0x55f7f429c900 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.445+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7f4285160 con 0x55f7f429c900 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.446+0000 7fa86ffff700 1 -- 172.21.15.57:0/751127246 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa888013960 con 0x55f7f429c900 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.446+0000 7fa8aa565b80 10 client.15888 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.446+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f7f3ff8c60 con 0x55f7f429c900 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.446+0000 7fa86ffff700 1 -- 172.21.15.57:0/751127246 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7f4285160 con 0x55f7f429c900 2023-10-06T22:15:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa86ffff700 1 -- 172.21.15.57:0/751127246 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa88801c050 con 0x55f7f429c900 2023-10-06T22:15:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa8aa565b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa8aa565b80 10 client.15888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:15:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa8aa565b80 20 client.15888 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f42bd480 0x55f7f42bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa8aa565b80 4 client.15888 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7f41dde10 con 0x55f7f42bd480 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.447+0000 7fa89499b700 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f42bd480 0x55f7f42bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.448+0000 7fa89499b700 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f42bd480 0x55f7f42bf8a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d04c5306c48ffb2d server_cookie=fd101d0465858684 in_seq=0 out_seq=0 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.448+0000 7fa86ffff700 10 client.15888 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa86ffff700 1 -- 172.21.15.57:0/751127246 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f7f41dde10 con 0x55f7f42bd480 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa86ffff700 10 client.15888 handle_command_reply: tid=0 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 client.15888 shutdown 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 msgr2=0x55f7f42b9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f42b9140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d50 tx=0x7fa888016f10 comp rx=0 tx=0).stop 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 shutdown_connections 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f42bd480 0x55f7f42bf8a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f42b9140 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f428f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa86ffff700 0 client.15888 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/751127246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f428f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.449+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 >> 172.21.15.57:0/751127246 conn(0x55f7f4206260 msgr2=0x55f7f428d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.450+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 shutdown_connections 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.450+0000 7fa8aa565b80 1 -- 172.21.15.57:0/751127246 wait complete. 2023-10-06T22:15:45.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.450+0000 7fa8aa565b80 20 client.15888 trim_cache size 0 max 16384 2023-10-06T22:15:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.476+0000 7fa8aa565b80 1 Processor -- start 2023-10-06T22:15:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.476+0000 7fa8aa565b80 1 -- start start 2023-10-06T22:15:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f429b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f429c900 0x55f7f429cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f4065920 con 0x55f7f4297240 2023-10-06T22:15:45.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f42b9f20 con 0x55f7f429c900 2023-10-06T22:15:45.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f429bd70 con 0x55f7f4295bf0 2023-10-06T22:15:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f429b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f429b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58808/0 (socket says 172.21.15.57:58808) 2023-10-06T22:15:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 -- 172.21.15.57:0/1569587988 learned_addr learned my addr 172.21.15.57:0/1569587988 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89519c700 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f429c900 0x55f7f429cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 -- 172.21.15.57:0/1569587988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f429c900 msgr2=0x55f7f429cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f429c900 0x55f7f429cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 -- 172.21.15.57:0/1569587988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 msgr2=0x55f7f42b6130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42b6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.477+0000 7fa89499b700 1 -- 172.21.15.57:0/1569587988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7f4036250 con 0x55f7f4295bf0 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.478+0000 7fa89499b700 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f429b320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55f7f429bc20 tx=0x7fa88c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3cf320e60ae65a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.478+0000 7fa8777fe700 1 -- 172.21.15.57:0/1569587988 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c012a90 con 0x55f7f4295bf0 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.478+0000 7fa8777fe700 1 -- 172.21.15.57:0/1569587988 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa88c026040 con 0x55f7f4295bf0 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.478+0000 7fa8777fe700 1 -- 172.21.15.57:0/1569587988 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c026d90 con 0x55f7f4295bf0 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 -- 172.21.15.57:0/1569587988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 msgr2=0x55f7f429b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f429b320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55f7f429bc20 tx=0x7fa88c016000 comp rx=0 tx=0).stop 2023-10-06T22:15:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 -- 172.21.15.57:0/1569587988 shutdown_connections 2023-10-06T22:15:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f429c900 0x55f7f429cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f429b320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/1569587988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f4297240 0x55f7f42b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 -- 172.21.15.57:0/1569587988 >> 172.21.15.57:0/1569587988 conn(0x55f7f4206260 msgr2=0x55f7f428bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 -- 172.21.15.57:0/1569587988 shutdown_connections 2023-10-06T22:15:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.479+0000 7fa8aa565b80 1 -- 172.21.15.57:0/1569587988 wait complete. 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 -- start start 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 Processor -- start 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f42b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f42b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f430a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f430a830 con 0x55f7f429c900 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f430a9a0 con 0x55f7f4297240 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7f430ab60 con 0x55f7f4295bf0 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.480+0000 7fa8aa565b80 10 client.0 _ll_register_callbacks cb 0x7fa888009310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa89519c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f430a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa887fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f42b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa887fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f42b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57990/0 (socket says 172.21.15.57:57990) 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa887fff700 1 -- 172.21.15.57:0/3930197281 learned_addr learned my addr 172.21.15.57:0/3930197281 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa89519c700 1 -- 172.21.15.57:0/3930197281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 msgr2=0x55f7f42b4300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa89519c700 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f42b4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa89519c700 1 -- 172.21.15.57:0/3930197281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 msgr2=0x55f7f42b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa89519c700 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f42b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.481+0000 7fa89519c700 1 -- 172.21.15.57:0/3930197281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c010e40 con 0x55f7f429c900 2023-10-06T22:15:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.482+0000 7fa89519c700 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f430a280 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa888007b70 tx=0x7fa888004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6525cd4e94fe3430 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.482+0000 7fa86ffff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.482+0000 7fa86ffff700 1 -- 172.21.15.57:0/3930197281 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800e070 con 0x55f7f429c900 2023-10-06T22:15:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.482+0000 7fa86ffff700 1 -- 172.21.15.57:0/3930197281 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fa888013970 con 0x55f7f429c900 2023-10-06T22:15:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.482+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa88800f040 con 0x55f7f429c900 2023-10-06T22:15:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa86ffff700 1 -- 172.21.15.57:0/3930197281 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa888013b30 con 0x55f7f429c900 2023-10-06T22:15:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 10 client.15891 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f7f400e2b0 con 0x55f7f429c900 2023-10-06T22:15:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa86ffff700 1 -- 172.21.15.57:0/3930197281 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa888039080 con 0x55f7f429c900 2023-10-06T22:15:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa86ffff700 1 -- 172.21.15.57:0/3930197281 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa88801c070 con 0x55f7f429c900 2023-10-06T22:15:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 10 client.15891 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:15:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 20 client.15891 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f4311e50 0x55f7f4314240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 4 client.15891 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.483+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7f41ddb30 con 0x55f7f4311e50 2023-10-06T22:15:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.484+0000 7fa89499b700 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f4311e50 0x55f7f4314240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.484+0000 7fa86ffff700 10 client.15891 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.484+0000 7fa89499b700 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f4311e50 0x55f7f4314240 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd8a6c559f762279 server_cookie=d7888e002e26bb38 in_seq=0 out_seq=0 2023-10-06T22:15:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa86ffff700 1 -- 172.21.15.57:0/3930197281 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7f41ddb30 con 0x55f7f4311e50 2023-10-06T22:15:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa86ffff700 10 client.15891 handle_command_reply: tid=0 2023-10-06T22:15:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 client.15891 shutdown 2023-10-06T22:15:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 msgr2=0x55f7f430a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f430a280 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa888007b70 tx=0x7fa888004f60 comp rx=0 tx=0).stop 2023-10-06T22:15:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 shutdown_connections 2023-10-06T22:15:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55f7f4311e50 0x55f7f4314240 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f7f429c900 0x55f7f430a280 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7f4295bf0 0x55f7f42b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa86ffff700 0 client.15891 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:15:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 --2- 172.21.15.57:0/3930197281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7f4297240 0x55f7f42b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 >> 172.21.15.57:0/3930197281 conn(0x55f7f4206260 msgr2=0x55f7f429cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.485+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 shutdown_connections 2023-10-06T22:15:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.486+0000 7fa8aa565b80 1 -- 172.21.15.57:0/3930197281 wait complete. 2023-10-06T22:15:45.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.486+0000 7fa8aa565b80 20 client.15891 trim_cache size 0 max 16384 2023-10-06T22:15:45.494 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.486+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa8780a1830 msgr2=0x7fa8780a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.486+0000 7fa89599d700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa8780a1830 0x7fa8780a3cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa88c005950 tx=0x7fa88c0058e0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.486+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8901007f0 msgr2=0x7fa890197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.486+0000 7fa89599d700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8901007f0 0x7fa890197ff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa88000b650 tx=0x7fa8800057f0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 shutdown_connections 2023-10-06T22:15:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa8780a1830 0x7fa8780a3cf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa890105c70 0x7fa8901a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8900ffe30 0x7fa890197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 --2- 172.21.15.57:0/3417198696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8901007f0 0x7fa890197ff0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 >> 172.21.15.57:0/3417198696 conn(0x7fa8900769d0 msgr2=0x7fa8900fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 shutdown_connections 2023-10-06T22:15:45.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.487+0000 7fa89599d700 1 -- 172.21.15.57:0/3417198696 wait complete. 2023-10-06T22:15:45.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T22:15:45.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 -- 172.21.15.57:0/1236016571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 msgr2=0x7f07a8073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 --2- 172.21.15.57:0/1236016571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 0x7f07a8073630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009990 tx=0x7f07a4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 -- 172.21.15.57:0/1236016571 shutdown_connections 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 --2- 172.21.15.57:0/1236016571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a8111e50 0x7f07a8114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 --2- 172.21.15.57:0/1236016571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 0x7f07a8073630 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 --2- 172.21.15.57:0/1236016571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07a8073b70 0x7f07a8111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 -- 172.21.15.57:0/1236016571 >> 172.21.15.57:0/1236016571 conn(0x7f07a80fc950 msgr2=0x7f07a80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 -- 172.21.15.57:0/1236016571 shutdown_connections 2023-10-06T22:15:45.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.857+0000 7f07affff700 1 -- 172.21.15.57:0/1236016571 wait complete. 2023-10-06T22:15:45.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.859+0000 7f07affff700 1 Processor -- start 2023-10-06T22:15:45.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.859+0000 7f07affff700 1 -- start start 2023-10-06T22:15:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 0x7f07a819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07affff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a8073b70 0x7f07a819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07affff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07a8111e50 0x7f07a81a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07affff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a81a2510 con 0x7f07a8111e50 2023-10-06T22:15:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07affff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a81a2680 con 0x7f07a8073250 2023-10-06T22:15:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07affff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a81a2840 con 0x7f07a8073b70 2023-10-06T22:15:45.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07a8111e50 0x7f07a81a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07ad59a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a8073b70 0x7f07a819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07a8111e50 0x7f07a81a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34002/0 (socket says 172.21.15.57:34002) 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.860+0000 7f07ae59c700 1 -- 172.21.15.57:0/1745478882 learned_addr learned my addr 172.21.15.57:0/1745478882 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07add9b700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 0x7f07a819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07ad59a700 1 -- 172.21.15.57:0/1745478882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 msgr2=0x7f07a819d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07ad59a700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 0x7f07a819d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07ad59a700 1 -- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07a8111e50 msgr2=0x7f07a81a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07ad59a700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07a8111e50 0x7f07a81a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07ad59a700 1 -- 172.21.15.57:0/1745478882 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a40095f0 con 0x7f07a8073b70 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07add9b700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 0x7f07a819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:45.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f07ad59a700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a8073b70 0x7f07a819da70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f079800c970 tx=0x7f079800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.861+0000 7f079effd700 1 -- 172.21.15.57:0/1745478882 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798004440 con 0x7f07a8073b70 2023-10-06T22:15:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.862+0000 7f079effd700 1 -- 172.21.15.57:0/1745478882 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079800ce60 con 0x7f07a8073b70 2023-10-06T22:15:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.862+0000 7f079effd700 1 -- 172.21.15.57:0/1745478882 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798003c70 con 0x7f07a8073b70 2023-10-06T22:15:45.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.862+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a81a2b20 con 0x7f07a8073b70 2023-10-06T22:15:45.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.862+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a81aa360 con 0x7f07a8073b70 2023-10-06T22:15:45.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.867+0000 7f079effd700 1 -- 172.21.15.57:0/1745478882 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f0798007cb0 con 0x7f07a8073b70 2023-10-06T22:15:45.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.869+0000 7f07c39e4b80 1 Processor -- start 2023-10-06T22:15:45.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.869+0000 7f07c39e4b80 1 -- start start 2023-10-06T22:15:45.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x5599128374a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x559912842350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 0x559912844c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599127a0460 con 0x5599128370c0 2023-10-06T22:15:45.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559912791c20 con 0x5599128379e0 2023-10-06T22:15:45.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991278f540 con 0x559912842890 2023-10-06T22:15:45.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x5599128374a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x5599128374a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34008/0 (socket says 172.21.15.57:34008) 2023-10-06T22:15:45.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07ae59c700 1 -- 172.21.15.57:0/1268449168 learned_addr learned my addr 172.21.15.57:0/1268449168 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.870+0000 7f07add9b700 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x559912842350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f079effd700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07940a1790 0x7f07940a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f07add9b700 1 -- 172.21.15.57:0/1268449168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 msgr2=0x559912844c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f07add9b700 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 0x559912844c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f07add9b700 1 -- 172.21.15.57:0/1268449168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 msgr2=0x5599128374a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f07add9b700 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x5599128374a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f07add9b700 1 -- 172.21.15.57:0/1268449168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599125b5de0 con 0x5599128379e0 2023-10-06T22:15:45.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f079effd700 1 -- 172.21.15.57:0/1745478882 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f07980ccf20 con 0x7f07a8073b70 2023-10-06T22:15:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f07add9b700 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x559912842350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07a4006010 tx=0x7f07a400b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8032b8c303b150 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.871+0000 7f07ae59c700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07940a1790 0x7f07940a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.872+0000 7f0793fff700 1 -- 172.21.15.57:0/1268449168 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400beb0 con 0x5599128379e0 2023-10-06T22:15:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.872+0000 7f0793fff700 1 -- 172.21.15.57:0/1268449168 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a401e040 con 0x5599128379e0 2023-10-06T22:15:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.872+0000 7f0793fff700 1 -- 172.21.15.57:0/1268449168 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4004c20 con 0x5599128379e0 2023-10-06T22:15:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1268449168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 msgr2=0x559912842350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x559912842350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07a4006010 tx=0x7f07a400b610 comp rx=0 tx=0).stop 2023-10-06T22:15:45.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07ae59c700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07940a1790 0x7f07940a3c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009fd0 tx=0x7f07a0004ad0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1268449168 shutdown_connections 2023-10-06T22:15:45.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 0x559912844c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x5599128374a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1268449168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x559912842350 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1268449168 >> 172.21.15.57:0/1268449168 conn(0x5599127ac260 msgr2=0x559912833450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1268449168 shutdown_connections 2023-10-06T22:15:45.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.873+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1268449168 wait complete. 2023-10-06T22:15:45.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.874+0000 7f07c39e4b80 1 -- start start 2023-10-06T22:15:45.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.875+0000 7f07c39e4b80 1 Processor -- start 2023-10-06T22:15:45.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128370c0 0x55991283f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x55991283fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559912842890 0x5599128400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991278ae30 con 0x559912842890 2023-10-06T22:15:45.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559912727710 con 0x5599128379e0 2023-10-06T22:15:45.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599127273b0 con 0x5599128370c0 2023-10-06T22:15:45.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07c39e4b80 10 client.0 _ll_register_callbacks cb 0x559912584800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:45.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559912842890 0x5599128400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559912842890 0x5599128400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34022/0 (socket says 172.21.15.57:34022) 2023-10-06T22:15:45.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07ae59c700 1 -- 172.21.15.57:0/1114665375 learned_addr learned my addr 172.21.15.57:0/1114665375 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.876+0000 7f07add9b700 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128370c0 0x55991283f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.877+0000 7f07ae59c700 1 -- 172.21.15.57:0/1114665375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128370c0 msgr2=0x55991283f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.877+0000 7f07ae59c700 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128370c0 0x55991283f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.877+0000 7f07ae59c700 1 -- 172.21.15.57:0/1114665375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 msgr2=0x55991283fba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:15:45.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.877+0000 7f07ae59c700 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x55991283fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.877+0000 7f07ae59c700 1 -- 172.21.15.57:0/1114665375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a8114bf0 con 0x559912842890 2023-10-06T22:15:45.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.877+0000 7f07ae59c700 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559912842890 0x5599128400e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f07a0017e70 tx=0x7f07a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347ca3e273e5001c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.877+0000 7f07917fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:45.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.878+0000 7f07917fa700 1 -- 172.21.15.57:0/1114665375 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000e040 con 0x559912842890 2023-10-06T22:15:45.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.878+0000 7f07917fa700 1 -- 172.21.15.57:0/1114665375 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f07a001d030 con 0x559912842890 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.878+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55991282af10 con 0x559912842890 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.878+0000 7f07917fa700 1 -- 172.21.15.57:0/1114665375 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a0017ea0 con 0x559912842890 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.878+0000 7f07c39e4b80 10 client.15900 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.878+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55991259ec60 con 0x559912842890 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07917fa700 1 -- 172.21.15.57:0/1114665375 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55991282af10 con 0x559912842890 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07917fa700 1 -- 172.21.15.57:0/1114665375 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07a00163a0 con 0x559912842890 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07c39e4b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:45.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07c39e4b80 10 client.15900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:15:45.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07c39e4b80 20 client.15900 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:45.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x559912852430 0x559912854850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07c39e4b80 4 client.15900 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:45.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.879+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559912783e10 con 0x559912852430 2023-10-06T22:15:45.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.880+0000 7f07add9b700 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x559912852430 0x559912854850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.880+0000 7f07add9b700 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x559912852430 0x559912854850 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f800e4da2587bb3 server_cookie=9ead6c969cb516f5 in_seq=0 out_seq=0 2023-10-06T22:15:45.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.880+0000 7f07917fa700 10 client.15900 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.881+0000 7f07917fa700 1 -- 172.21.15.57:0/1114665375 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559912783e10 con 0x559912852430 2023-10-06T22:15:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.881+0000 7f07917fa700 10 client.15900 handle_command_reply: tid=0 2023-10-06T22:15:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.881+0000 7f07c39e4b80 1 client.15900 shutdown 2023-10-06T22:15:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.881+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559912842890 msgr2=0x5599128400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.881+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559912842890 0x5599128400e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f07a0017e70 tx=0x7f07a000b040 comp rx=0 tx=0).stop 2023-10-06T22:15:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 shutdown_connections 2023-10-06T22:15:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x559912852430 0x559912854850 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559912842890 0x5599128400e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07917fa700 0 client.15900 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128370c0 0x55991283f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/1114665375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128379e0 0x55991283fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 >> 172.21.15.57:0/1114665375 conn(0x5599127ac260 msgr2=0x559912833040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 shutdown_connections 2023-10-06T22:15:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 1 -- 172.21.15.57:0/1114665375 wait complete. 2023-10-06T22:15:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.882+0000 7f07c39e4b80 20 client.15900 trim_cache size 0 max 16384 2023-10-06T22:15:45.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.908+0000 7f07c39e4b80 1 Processor -- start 2023-10-06T22:15:45.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.909+0000 7f07c39e4b80 1 -- start start 2023-10-06T22:15:45.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.909+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128370c0 0x55991283ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.909+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128379e0 0x55991283e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.909+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 0x55991283c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.909+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991260b920 con 0x5599128379e0 2023-10-06T22:15:45.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.909+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991283dbc0 con 0x5599128370c0 2023-10-06T22:15:45.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.909+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991283b4b0 con 0x559912842890 2023-10-06T22:15:45.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128370c0 0x55991283ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128370c0 0x55991283ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:58024/0 (socket says 172.21.15.57:58024) 2023-10-06T22:15:45.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 -- 172.21.15.57:0/3231670823 learned_addr learned my addr 172.21.15.57:0/3231670823 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07ae59c700 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 0x55991283c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07ad59a700 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128379e0 0x55991283e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 -- 172.21.15.57:0/3231670823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 msgr2=0x55991283c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 0x55991283c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 -- 172.21.15.57:0/3231670823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128379e0 msgr2=0x55991283e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128379e0 0x55991283e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 -- 172.21.15.57:0/3231670823 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599125dc250 con 0x5599128370c0 2023-10-06T22:15:45.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07ad59a700 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128379e0 0x55991283e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:15:45.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.910+0000 7f07add9b700 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128370c0 0x55991283ff10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f07a400b6f0 tx=0x7f07a400b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12246d7be2dd29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.911+0000 7f079cff9700 1 -- 172.21.15.57:0/3231670823 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4033490 con 0x5599128370c0 2023-10-06T22:15:45.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.911+0000 7f079cff9700 1 -- 172.21.15.57:0/3231670823 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a4005640 con 0x5599128370c0 2023-10-06T22:15:45.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.911+0000 7f079cff9700 1 -- 172.21.15.57:0/3231670823 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4004be0 con 0x5599128370c0 2023-10-06T22:15:45.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 -- 172.21.15.57:0/3231670823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128370c0 msgr2=0x55991283ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128370c0 0x55991283ff10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f07a400b6f0 tx=0x7f07a400b610 comp rx=0 tx=0).stop 2023-10-06T22:15:45.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 -- 172.21.15.57:0/3231670823 shutdown_connections 2023-10-06T22:15:45.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559912842890 0x55991283c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5599128370c0 0x55991283ff10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/3231670823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128379e0 0x55991283e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 -- 172.21.15.57:0/3231670823 >> 172.21.15.57:0/3231670823 conn(0x5599127ac260 msgr2=0x55991283de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 -- 172.21.15.57:0/3231670823 shutdown_connections 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.912+0000 7f07c39e4b80 1 -- 172.21.15.57:0/3231670823 wait complete. 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.913+0000 7f07c39e4b80 1 -- start start 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.913+0000 7f07c39e4b80 1 Processor -- start 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x559912846b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128379e0 0x559912847080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07c39e4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559912842890 0x5599128475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599128b6350 con 0x5599128370c0 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991289c240 con 0x559912842890 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07c39e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991289c400 con 0x5599128379e0 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07c39e4b80 10 client.0 _ll_register_callbacks cb 0x7f07a00177e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559912842890 0x5599128475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559912842890 0x5599128475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:58030/0 (socket says 172.21.15.57:58030) 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07ae59c700 1 -- 172.21.15.57:0/278195030 learned_addr learned my addr 172.21.15.57:0/278195030 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07ad59a700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128379e0 0x559912847080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.914+0000 7f07add9b700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x559912846b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07ad59a700 1 -- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559912842890 msgr2=0x5599128475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07ad59a700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559912842890 0x5599128475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07ad59a700 1 -- 172.21.15.57:0/278195030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 msgr2=0x559912846b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:45.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07ad59a700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x559912846b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07ad59a700 1 -- 172.21.15.57:0/278195030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599125b5de0 con 0x5599128379e0 2023-10-06T22:15:45.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07ae59c700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559912842890 0x5599128475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:45.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.915+0000 7f07add9b700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x559912846b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:45.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.916+0000 7f07ad59a700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128379e0 0x559912847080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07980ca540 tx=0x7f07980998c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2d80710de2aadf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:45.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.916+0000 7f07917fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.917+0000 7f07917fa700 1 -- 172.21.15.57:0/278195030 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079809a160 con 0x5599128379e0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.918+0000 7f07917fa700 1 -- 172.21.15.57:0/278195030 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0798099d00 con 0x5599128379e0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.918+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07a401e040 con 0x5599128379e0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.918+0000 7f07917fa700 1 -- 172.21.15.57:0/278195030 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079809de50 con 0x5599128379e0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.918+0000 7f07c39e4b80 10 client.25903 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.918+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5599125b42b0 con 0x5599128379e0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.918+0000 7f07917fa700 1 -- 172.21.15.57:0/278195030 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a401e040 con 0x5599128379e0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07917fa700 1 -- 172.21.15.57:0/278195030 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07980a1c00 con 0x5599128379e0 2023-10-06T22:15:45.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07c39e4b80 10 client.25903 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:45.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07c39e4b80 10 client.25903 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:15:45.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07c39e4b80 20 client.25903 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55991289db50 0x55991289ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07c39e4b80 4 client.25903 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559912783b30 con 0x55991289db50 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.919+0000 7f07add9b700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55991289db50 0x55991289ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.920+0000 7f07add9b700 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55991289db50 0x55991289ff70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e32b1aa6b93647a server_cookie=d050209c37dca742 in_seq=0 out_seq=0 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.920+0000 7f07917fa700 10 client.25903 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.920+0000 7f07917fa700 1 -- 172.21.15.57:0/278195030 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559912783b30 con 0x55991289db50 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.920+0000 7f07917fa700 10 client.25903 handle_command_reply: tid=0 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.920+0000 7f07c39e4b80 1 client.25903 shutdown 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.920+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128379e0 msgr2=0x559912847080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.920+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128379e0 0x559912847080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07980ca540 tx=0x7f07980998c0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 shutdown_connections 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x55991289db50 0x55991289ff70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559912842890 0x5599128475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599128370c0 0x559912846b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 --2- 172.21.15.57:0/278195030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599128379e0 0x559912847080 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 >> 172.21.15.57:0/278195030 conn(0x5599127ac260 msgr2=0x559912832ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07917fa700 0 client.25903 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 shutdown_connections 2023-10-06T22:15:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 1 -- 172.21.15.57:0/278195030 wait complete. 2023-10-06T22:15:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.921+0000 7f07c39e4b80 20 client.25903 trim_cache size 0 max 16384 2023-10-06T22:15:45.927 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:45.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07940a1790 msgr2=0x7f07940a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07940a1790 0x7f07940a3c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009fd0 tx=0x7f07a0004ad0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a8073b70 msgr2=0x7f07a819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:45.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a8073b70 0x7f07a819da70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f079800c970 tx=0x7f079800cc80 comp rx=0 tx=0).stop 2023-10-06T22:15:45.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 shutdown_connections 2023-10-06T22:15:45.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f07940a1790 0x7f07940a3c50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07a8111e50 0x7f07a81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a8073250 0x7f07a819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 --2- 172.21.15.57:0/1745478882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a8073b70 0x7f07a819da70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 >> 172.21.15.57:0/1745478882 conn(0x7f07a80fc950 msgr2=0x7f07a80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.922+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 shutdown_connections 2023-10-06T22:15:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:45.923+0000 7f07affff700 1 -- 172.21.15.57:0/1745478882 wait complete. 2023-10-06T22:15:45.953 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T22:15:46.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.297+0000 7f7541d9d700 1 -- 172.21.15.57:0/162116155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101ea0 msgr2=0x7f753c102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.297+0000 7f7541d9d700 1 --2- 172.21.15.57:0/162116155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101ea0 0x7f753c102280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f75300098f0 tx=0x7f7530009c00 comp rx=0 tx=0).stop 2023-10-06T22:15:46.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.298+0000 7f7541d9d700 1 -- 172.21.15.57:0/162116155 shutdown_connections 2023-10-06T22:15:46.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.298+0000 7f7541d9d700 1 --2- 172.21.15.57:0/162116155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c10b270 0x7f753c10d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.298+0000 7f7541d9d700 1 --2- 172.21.15.57:0/162116155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101ea0 0x7f753c102280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.298+0000 7f7541d9d700 1 --2- 172.21.15.57:0/162116155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f753c1027c0 0x7f753c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.298+0000 7f7541d9d700 1 -- 172.21.15.57:0/162116155 >> 172.21.15.57:0/162116155 conn(0x7f753c0fb660 msgr2=0x7f753c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:46.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.298+0000 7f7541d9d700 1 -- 172.21.15.57:0/162116155 shutdown_connections 2023-10-06T22:15:46.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.298+0000 7f7541d9d700 1 -- 172.21.15.57:0/162116155 wait complete. 2023-10-06T22:15:46.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.299+0000 7f7541d9d700 1 Processor -- start 2023-10-06T22:15:46.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.300+0000 7f7541d9d700 1 -- start start 2023-10-06T22:15:46.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.300+0000 7f7541d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c101ea0 0x7f753c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.300+0000 7f7541d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c1027c0 0x7f753c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.300+0000 7f7541d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f753c10b270 0x7f753c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.300+0000 7f7541d9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c19bd30 con 0x7f753c101ea0 2023-10-06T22:15:46.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.300+0000 7f7541d9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c19bea0 con 0x7f753c1027c0 2023-10-06T22:15:46.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.300+0000 7f7541d9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c19c060 con 0x7f753c10b270 2023-10-06T22:15:46.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c101ea0 0x7f753c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c101ea0 0x7f753c196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34062/0 (socket says 172.21.15.57:34062) 2023-10-06T22:15:46.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 -- 172.21.15.57:0/3669890124 learned_addr learned my addr 172.21.15.57:0/3669890124 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:46.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f754159c700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f753c10b270 0x7f753c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f753bfff700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c1027c0 0x7f753c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 -- 172.21.15.57:0/3669890124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f753c10b270 msgr2=0x7f753c19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f753c10b270 0x7f753c19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 -- 172.21.15.57:0/3669890124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c1027c0 msgr2=0x7f753c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c1027c0 0x7f753c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.301+0000 7f7540d9b700 1 -- 172.21.15.57:0/3669890124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7530009520 con 0x7f753c101ea0 2023-10-06T22:15:46.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.302+0000 7f7540d9b700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c101ea0 0x7f753c196d50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7530009810 tx=0x7f753000fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:46.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.302+0000 7f7539ffb700 1 -- 172.21.15.57:0/3669890124 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753001c070 con 0x7f753c101ea0 2023-10-06T22:15:46.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.302+0000 7f7539ffb700 1 -- 172.21.15.57:0/3669890124 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7530005040 con 0x7f753c101ea0 2023-10-06T22:15:46.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.302+0000 7f7539ffb700 1 -- 172.21.15.57:0/3669890124 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75300038e0 con 0x7f753c101ea0 2023-10-06T22:15:46.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.302+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c19c2e0 con 0x7f753c101ea0 2023-10-06T22:15:46.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.302+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c19c560 con 0x7f753c101ea0 2023-10-06T22:15:46.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.308+0000 7f7539ffb700 1 -- 172.21.15.57:0/3669890124 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f7530020030 con 0x7f753c101ea0 2023-10-06T22:15:46.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.310+0000 7f7556921b80 1 Processor -- start 2023-10-06T22:15:46.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.310+0000 7f7539ffb700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f752c0a1830 0x7f752c0a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.310+0000 7f7556921b80 1 -- start start 2023-10-06T22:15:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.310+0000 7f7539ffb700 1 -- 172.21.15.57:0/3669890124 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f75300ce0a0 con 0x7f753c101ea0 2023-10-06T22:15:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.311+0000 7f754159c700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f752c0a1830 0x7f752c0a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7556921b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d4e5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7556921b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4f0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7556921b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7556921b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d44e460 con 0x55695d4e52a0 2023-10-06T22:15:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7556921b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d43fc20 con 0x55695d4f0a70 2023-10-06T22:15:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7556921b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d43d540 con 0x55695d4e5bc0 2023-10-06T22:15:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f754159c700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f752c0a1830 0x7f752c0a3cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7534009380 tx=0x7f753400b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:46.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d4e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f754159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d4e5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34076/0 (socket says 172.21.15.57:34076) 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.312+0000 7f7540d9b700 1 -- 172.21.15.57:0/739368738 learned_addr learned my addr 172.21.15.57:0/739368738 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f753bfff700 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f754159c700 1 -- 172.21.15.57:0/739368738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 msgr2=0x55695d4f0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f754159c700 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4f0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f754159c700 1 -- 172.21.15.57:0/739368738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 msgr2=0x55695d4e5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f754159c700 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d4e5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f754159c700 1 -- 172.21.15.57:0/739368738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55695d263de0 con 0x55695d4f0a70 2023-10-06T22:15:46.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f754159c700 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4f2e60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7534015760 tx=0x7f7534013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaf9e540cc8f02b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.313+0000 7f7526ffd700 1 -- 172.21.15.57:0/739368738 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753400e050 con 0x55695d4f0a70 2023-10-06T22:15:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.314+0000 7f7526ffd700 1 -- 172.21.15.57:0/739368738 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f753400f040 con 0x55695d4f0a70 2023-10-06T22:15:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.314+0000 7f7526ffd700 1 -- 172.21.15.57:0/739368738 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75340169b0 con 0x55695d4f0a70 2023-10-06T22:15:46.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.314+0000 7f7556921b80 1 -- 172.21.15.57:0/739368738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 msgr2=0x55695d4f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4f2e60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7534015760 tx=0x7f7534013000 comp rx=0 tx=0).stop 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 -- 172.21.15.57:0/739368738 shutdown_connections 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4f2e60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d4e5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 --2- 172.21.15.57:0/739368738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4f0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 -- 172.21.15.57:0/739368738 >> 172.21.15.57:0/739368738 conn(0x55695d45a260 msgr2=0x55695d4e1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 -- 172.21.15.57:0/739368738 shutdown_connections 2023-10-06T22:15:46.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.315+0000 7f7556921b80 1 -- 172.21.15.57:0/739368738 wait complete. 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.316+0000 7f7556921b80 1 -- start start 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.316+0000 7f7556921b80 1 Processor -- start 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f7556921b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e52a0 0x55695d4ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f7556921b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f7556921b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4f0a70 0x55695d4ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f7556921b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d438e30 con 0x55695d4f0a70 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f7556921b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d3d5710 con 0x55695d4e52a0 2023-10-06T22:15:46.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f7556921b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d3d53b0 con 0x55695d4e5bc0 2023-10-06T22:15:46.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f7556921b80 10 client.0 _ll_register_callbacks cb 0x55695d232800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:46.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f754159c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4f0a70 0x55695d4ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f754159c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4f0a70 0x55695d4ee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34084/0 (socket says 172.21.15.57:34084) 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.317+0000 7f754159c700 1 -- 172.21.15.57:0/1544896517 learned_addr learned my addr 172.21.15.57:0/1544896517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f753bfff700 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f7540d9b700 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e52a0 0x55695d4ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f753bfff700 1 -- 172.21.15.57:0/1544896517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e52a0 msgr2=0x55695d4ed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f753bfff700 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e52a0 0x55695d4ed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f753bfff700 1 -- 172.21.15.57:0/1544896517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4f0a70 msgr2=0x55695d4ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f753bfff700 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4f0a70 0x55695d4ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f753bfff700 1 -- 172.21.15.57:0/1544896517 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7534015440 con 0x55695d4e5bc0 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f753bfff700 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4eddf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7528005fd0 tx=0x7f75280129b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4e022d77b0316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.318+0000 7f751bfff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:46.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.319+0000 7f751bfff700 1 -- 172.21.15.57:0/1544896517 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528017070 con 0x55695d4e5bc0 2023-10-06T22:15:46.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.319+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55695d4d9160 con 0x55695d4e5bc0 2023-10-06T22:15:46.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.319+0000 7f751bfff700 1 -- 172.21.15.57:0/1544896517 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7528004030 con 0x55695d4e5bc0 2023-10-06T22:15:46.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.319+0000 7f751bfff700 1 -- 172.21.15.57:0/1544896517 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75280041f0 con 0x55695d4e5bc0 2023-10-06T22:15:46.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.319+0000 7f7556921b80 10 client.25915 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:46.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.319+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55695d24cc60 con 0x55695d4e5bc0 2023-10-06T22:15:46.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.319+0000 7f751bfff700 1 -- 172.21.15.57:0/1544896517 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55695d4d9160 con 0x55695d4e5bc0 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f751bfff700 1 -- 172.21.15.57:0/1544896517 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7528026480 con 0x55695d4e5bc0 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f7556921b80 10 client.25915 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f7556921b80 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f7556921b80 20 client.25915 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f7556921b80 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d4fefe0 0x55695d4ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f7556921b80 4 client.25915 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55695d431e10 con 0x55695d4fefe0 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.320+0000 7f7540d9b700 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d4fefe0 0x55695d4ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.321+0000 7f7540d9b700 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d4fefe0 0x55695d4ff3f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a9d13c07288a4b server_cookie=e761c29d68dc9981 in_seq=0 out_seq=0 2023-10-06T22:15:46.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.321+0000 7f751bfff700 10 client.25915 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.322+0000 7f751bfff700 1 -- 172.21.15.57:0/1544896517 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55695d431e10 con 0x55695d4fefe0 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.322+0000 7f751bfff700 10 client.25915 handle_command_reply: tid=0 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.322+0000 7f7556921b80 1 client.25915 shutdown 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.322+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 msgr2=0x55695d4eddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.322+0000 7f7556921b80 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4eddf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7528005fd0 tx=0x7f75280129b0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 shutdown_connections 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d4fefe0 0x55695d4ff3f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4f0a70 0x55695d4ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e52a0 0x55695d4ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 --2- 172.21.15.57:0/1544896517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e5bc0 0x55695d4eddf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 >> 172.21.15.57:0/1544896517 conn(0x55695d45a260 msgr2=0x55695d4e1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:46.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f751bfff700 0 client.25915 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:46.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 shutdown_connections 2023-10-06T22:15:46.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 1 -- 172.21.15.57:0/1544896517 wait complete. 2023-10-06T22:15:46.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.323+0000 7f7556921b80 20 client.25915 trim_cache size 0 max 16384 2023-10-06T22:15:46.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 Processor -- start 2023-10-06T22:15:46.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 -- start start 2023-10-06T22:15:46.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e52a0 0x55695d4ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e5bc0 0x55695d500270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d2b9920 con 0x55695d4e5bc0 2023-10-06T22:15:46.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d4e2270 con 0x55695d4f0a70 2023-10-06T22:15:46.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.350+0000 7f7556921b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d4fc670 con 0x55695d4e52a0 2023-10-06T22:15:46.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e52a0 0x55695d4ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f753bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e5bc0 0x55695d500270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e52a0 0x55695d4ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58910/0 (socket says 172.21.15.57:58910) 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 -- 172.21.15.57:0/2521384357 learned_addr learned my addr 172.21.15.57:0/2521384357 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f754159c700 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 -- 172.21.15.57:0/2521384357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 msgr2=0x55695d4eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4eac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 -- 172.21.15.57:0/2521384357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e5bc0 msgr2=0x55695d500270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e5bc0 0x55695d500270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.351+0000 7f7540d9b700 1 -- 172.21.15.57:0/2521384357 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55695d28a250 con 0x55695d4e52a0 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.352+0000 7f7540d9b700 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e52a0 0x55695d4ffd30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f753009ae50 tx=0x7f7530009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6baa742358b57372 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.352+0000 7f75277fe700 1 -- 172.21.15.57:0/2521384357 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753009f060 con 0x55695d4e52a0 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.352+0000 7f75277fe700 1 -- 172.21.15.57:0/2521384357 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75300a3210 con 0x55695d4e52a0 2023-10-06T22:15:46.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.352+0000 7f75277fe700 1 -- 172.21.15.57:0/2521384357 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530005490 con 0x55695d4e52a0 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 -- 172.21.15.57:0/2521384357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e52a0 msgr2=0x55695d4ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e52a0 0x55695d4ffd30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f753009ae50 tx=0x7f7530009c00 comp rx=0 tx=0).stop 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 -- 172.21.15.57:0/2521384357 shutdown_connections 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4f0a70 0x55695d4eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4e52a0 0x55695d4ffd30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 --2- 172.21.15.57:0/2521384357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e5bc0 0x55695d500270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 -- 172.21.15.57:0/2521384357 >> 172.21.15.57:0/2521384357 conn(0x55695d45a260 msgr2=0x55695d500d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 -- 172.21.15.57:0/2521384357 shutdown_connections 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.353+0000 7f7556921b80 1 -- 172.21.15.57:0/2521384357 wait complete. 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.354+0000 7f7556921b80 1 -- start start 2023-10-06T22:15:46.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.354+0000 7f7556921b80 1 Processor -- start 2023-10-06T22:15:46.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7556921b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d50d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7556921b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e5bc0 0x55695d50daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7556921b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4f0a70 0x55695d55b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7556921b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d55beb0 con 0x55695d4e52a0 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7556921b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d55bff0 con 0x55695d4e5bc0 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7556921b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695d55c130 con 0x55695d4f0a70 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7556921b80 10 client.0 _ll_register_callbacks cb 0x55695d232800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d50d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7540d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d50d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34108/0 (socket says 172.21.15.57:34108) 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f7540d9b700 1 -- 172.21.15.57:0/2409489933 learned_addr learned my addr 172.21.15.57:0/2409489933 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.355+0000 7f753bfff700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e5bc0 0x55695d50daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f754159c700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4f0a70 0x55695d55b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f7540d9b700 1 -- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4f0a70 msgr2=0x55695d55b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f7540d9b700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4f0a70 0x55695d55b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f7540d9b700 1 -- 172.21.15.57:0/2409489933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e5bc0 msgr2=0x55695d50daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f7540d9b700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e5bc0 0x55695d50daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f7540d9b700 1 -- 172.21.15.57:0/2409489933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c19c2e0 con 0x55695d4e52a0 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f754159c700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4f0a70 0x55695d55b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f7540d9b700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d50d560 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7530097980 tx=0x7f753009f060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4449c116fb21e24b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:46.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f751bfff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f751bfff700 1 -- 172.21.15.57:0/2409489933 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75300993b0 con 0x55695d4e52a0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f751bfff700 1 -- 172.21.15.57:0/2409489933 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7530097620 con 0x55695d4e52a0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.356+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f753400f040 con 0x55695d4e52a0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.357+0000 7f751bfff700 1 -- 172.21.15.57:0/2409489933 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75300977e0 con 0x55695d4e52a0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.357+0000 7f7556921b80 10 client.15921 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.357+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55695d2622b0 con 0x55695d4e52a0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.357+0000 7f751bfff700 1 -- 172.21.15.57:0/2409489933 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f753400f040 con 0x55695d4e52a0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.357+0000 7f751bfff700 1 -- 172.21.15.57:0/2409489933 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7530015020 con 0x55695d4e52a0 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.358+0000 7f7556921b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.358+0000 7f7556921b80 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.358+0000 7f7556921b80 20 client.15921 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:46.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.358+0000 7f7556921b80 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d55d4c0 0x55695d55f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:46.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.358+0000 7f7556921b80 4 client.15921 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:15:46.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.358+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55695d431b30 con 0x55695d55d4c0 2023-10-06T22:15:46.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.358+0000 7f753bfff700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d55d4c0 0x55695d55f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:46.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.359+0000 7f753bfff700 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d55d4c0 0x55695d55f8e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2615d95030ba3f2 server_cookie=2a402a518c349eb2 in_seq=0 out_seq=0 2023-10-06T22:15:46.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.359+0000 7f751bfff700 10 client.15921 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:46.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.359+0000 7f751bfff700 1 -- 172.21.15.57:0/2409489933 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55695d431b30 con 0x55695d55d4c0 2023-10-06T22:15:46.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.359+0000 7f751bfff700 10 client.15921 handle_command_reply: tid=0 2023-10-06T22:15:46.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.359+0000 7f7556921b80 1 client.15921 shutdown 2023-10-06T22:15:46.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.359+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 msgr2=0x55695d50d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.359+0000 7f7556921b80 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d50d560 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7530097980 tx=0x7f753009f060 comp rx=0 tx=0).stop 2023-10-06T22:15:46.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 shutdown_connections 2023-10-06T22:15:46.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55695d55d4c0 0x55695d55f8e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55695d4f0a70 0x55695d55b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55695d4e52a0 0x55695d50d560 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f751bfff700 0 client.15921 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:46.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 --2- 172.21.15.57:0/2409489933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55695d4e5bc0 0x55695d50daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 >> 172.21.15.57:0/2409489933 conn(0x55695d45a260 msgr2=0x55695d4ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:46.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 shutdown_connections 2023-10-06T22:15:46.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 1 -- 172.21.15.57:0/2409489933 wait complete. 2023-10-06T22:15:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.360+0000 7f7556921b80 20 client.15921 trim_cache size 0 max 16384 2023-10-06T22:15:46.369 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f752c0a1830 msgr2=0x7f752c0a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f752c0a1830 0x7f752c0a3cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7534009380 tx=0x7f753400b040 comp rx=0 tx=0).stop 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c101ea0 msgr2=0x7f753c196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c101ea0 0x7f753c196d50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7530009810 tx=0x7f753000fa90 comp rx=0 tx=0).stop 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 shutdown_connections 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f752c0a1830 0x7f752c0a3cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f753c10b270 0x7f753c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f753c101ea0 0x7f753c196d50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 --2- 172.21.15.57:0/3669890124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c1027c0 0x7f753c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.361+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 >> 172.21.15.57:0/3669890124 conn(0x7f753c0fb660 msgr2=0x7f753c10d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.362+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 shutdown_connections 2023-10-06T22:15:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:46.362+0000 7f7541d9d700 1 -- 172.21.15.57:0/3669890124 wait complete. 2023-10-06T22:15:47.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:46 smithi156 ceph-mon[134358]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:15:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:46 smithi190 ceph-mon[134891]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:15:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:46 smithi057 ceph-mon[98492]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-06T22:15:47.393 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T22:15:47.393 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T22:15:47.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.742+0000 7faa261ff700 1 -- 172.21.15.57:0/3762734207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20102910 msgr2=0x7faa2010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.742+0000 7faa261ff700 1 --2- 172.21.15.57:0/3762734207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20102910 0x7faa2010adf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa1400faf0 tx=0x7faa1400fe00 comp rx=0 tx=0).stop 2023-10-06T22:15:47.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.742+0000 7faa261ff700 1 -- 172.21.15.57:0/3762734207 shutdown_connections 2023-10-06T22:15:47.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.742+0000 7faa261ff700 1 --2- 172.21.15.57:0/3762734207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2010b410 0x7faa2010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.742+0000 7faa261ff700 1 --2- 172.21.15.57:0/3762734207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20101ff0 0x7faa201023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.742+0000 7faa261ff700 1 --2- 172.21.15.57:0/3762734207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20102910 0x7faa2010adf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.742+0000 7faa261ff700 1 -- 172.21.15.57:0/3762734207 >> 172.21.15.57:0/3762734207 conn(0x7faa200fb830 msgr2=0x7faa200fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:47.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.743+0000 7faa261ff700 1 -- 172.21.15.57:0/3762734207 shutdown_connections 2023-10-06T22:15:47.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.743+0000 7faa261ff700 1 -- 172.21.15.57:0/3762734207 wait complete. 2023-10-06T22:15:47.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.744+0000 7faa261ff700 1 Processor -- start 2023-10-06T22:15:47.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.745+0000 7faa261ff700 1 -- start start 2023-10-06T22:15:47.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.745+0000 7faa261ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20101ff0 0x7faa2019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.745+0000 7faa261ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20102910 0x7faa2019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.745+0000 7faa261ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2010b410 0x7faa201a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.745+0000 7faa261ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201a2610 con 0x7faa2010b410 2023-10-06T22:15:47.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.745+0000 7faa261ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201a2780 con 0x7faa20101ff0 2023-10-06T22:15:47.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.745+0000 7faa261ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201a2940 con 0x7faa20102910 2023-10-06T22:15:47.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20102910 0x7faa2019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20102910 0x7faa2019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58942/0 (socket says 172.21.15.57:58942) 2023-10-06T22:15:47.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 -- 172.21.15.57:0/2829355640 learned_addr learned my addr 172.21.15.57:0/2829355640 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:47.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1ffff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2010b410 0x7faa201a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20101ff0 0x7faa2019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 -- 172.21.15.57:0/2829355640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20101ff0 msgr2=0x7faa2019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20101ff0 0x7faa2019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 -- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2010b410 msgr2=0x7faa201a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2010b410 0x7faa201a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1effd700 1 -- 172.21.15.57:0/2829355640 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1400f750 con 0x7faa20102910 2023-10-06T22:15:47.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.746+0000 7faa1ffff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2010b410 0x7faa201a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:47.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.747+0000 7faa1effd700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20102910 0x7faa2019db70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa14005320 tx=0x7faa14005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:47.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.747+0000 7faa1cff9700 1 -- 172.21.15.57:0/2829355640 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14003a90 con 0x7faa20102910 2023-10-06T22:15:47.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.747+0000 7faa1cff9700 1 -- 172.21.15.57:0/2829355640 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa14005710 con 0x7faa20102910 2023-10-06T22:15:47.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.747+0000 7faa1cff9700 1 -- 172.21.15.57:0/2829355640 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14020e80 con 0x7faa20102910 2023-10-06T22:15:47.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.747+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa201a2bc0 con 0x7faa20102910 2023-10-06T22:15:47.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.747+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa201a2e40 con 0x7faa20102910 2023-10-06T22:15:47.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.753+0000 7faa1cff9700 1 -- 172.21.15.57:0/2829355640 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7faa14003bf0 con 0x7faa20102910 2023-10-06T22:15:47.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.754+0000 7faa39b80b80 1 Processor -- start 2023-10-06T22:15:47.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.755+0000 7faa39b80b80 1 -- start start 2023-10-06T22:15:47.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.755+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2140 0x55e54f0f2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.755+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2a60 0x55e54f0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.755+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.755+0000 7faa39b80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f05b4e0 con 0x55e54f0fd910 2023-10-06T22:15:47.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.755+0000 7faa39b80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f04cca0 con 0x55e54f0f2a60 2023-10-06T22:15:47.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.755+0000 7faa39b80b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f04a5c0 con 0x55e54f0f2140 2023-10-06T22:15:47.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.756+0000 7faa1cff9700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7faa080a1790 0x7faa080a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.756+0000 7faa1cff9700 1 -- 172.21.15.57:0/2829355640 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7faa140ce000 con 0x7faa20102910 2023-10-06T22:15:47.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.756+0000 7faa1ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2140 0x55e54f0f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.756+0000 7faa1ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2140 0x55e54f0f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58948/0 (socket says 172.21.15.57:58948) 2023-10-06T22:15:47.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.756+0000 7faa1ffff700 1 -- 172.21.15.57:0/1397421385 learned_addr learned my addr 172.21.15.57:0/1397421385 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:47.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.756+0000 7faa1ffff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7faa080a1790 0x7faa080a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.756+0000 7faa1effd700 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2a60 0x55e54f0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1ffff700 1 -- 172.21.15.57:0/1397421385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2a60 msgr2=0x55e54f0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:47.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2a60 0x55e54f0fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1ffff700 1 -- 172.21.15.57:0/1397421385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 msgr2=0x55e54f0ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:47.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0ffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1ffff700 1 -- 172.21.15.57:0/1397421385 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54ee70e60 con 0x55e54f0f2140 2023-10-06T22:15:47.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1effd700 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:47.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2140 0x55e54f0f2520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faa18007f90 tx=0x7faa18018640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89baad3ad69d5b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:47.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.757+0000 7faa1ffff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7faa080a1790 0x7faa080a3c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa18018990 tx=0x7faa1800b000 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:47.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.758+0000 7faa05ffb700 1 -- 172.21.15.57:0/1397421385 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800e040 con 0x55e54f0f2140 2023-10-06T22:15:47.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.759+0000 7faa05ffb700 1 -- 172.21.15.57:0/1397421385 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa18009b50 con 0x55e54f0f2140 2023-10-06T22:15:47.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.759+0000 7faa05ffb700 1 -- 172.21.15.57:0/1397421385 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18005280 con 0x55e54f0f2140 2023-10-06T22:15:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 -- 172.21.15.57:0/1397421385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2140 msgr2=0x55e54f0f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2140 0x55e54f0f2520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faa18007f90 tx=0x7faa18018640 comp rx=0 tx=0).stop 2023-10-06T22:15:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 -- 172.21.15.57:0/1397421385 shutdown_connections 2023-10-06T22:15:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2140 0x55e54f0f2520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1397421385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2a60 0x55e54f0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 -- 172.21.15.57:0/1397421385 >> 172.21.15.57:0/1397421385 conn(0x55e54f0672e0 msgr2=0x55e54f0ee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 -- 172.21.15.57:0/1397421385 shutdown_connections 2023-10-06T22:15:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.760+0000 7faa39b80b80 1 -- 172.21.15.57:0/1397421385 wait complete. 2023-10-06T22:15:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.761+0000 7faa39b80b80 1 -- start start 2023-10-06T22:15:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.761+0000 7faa39b80b80 1 Processor -- start 2023-10-06T22:15:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2a60 0x55e54f0fac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0fd910 0x55e54f0fb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa39b80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f045eb0 con 0x55e54f0f2a60 2023-10-06T22:15:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa39b80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54efe2790 con 0x55e54f0f2140 2023-10-06T22:15:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa39b80b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54efe2430 con 0x55e54f0fd910 2023-10-06T22:15:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa39b80b80 10 client.0 _ll_register_callbacks cb 0x55e54ee3f880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.762+0000 7faa1effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2a60 0x55e54f0fac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0fd910 0x55e54f0fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2a60 0x55e54f0fac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34134/0 (socket says 172.21.15.57:34134) 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1effd700 1 -- 172.21.15.57:0/1708599779 learned_addr learned my addr 172.21.15.57:0/1708599779 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1ffff700 1 -- 172.21.15.57:0/1708599779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 msgr2=0x55e54f0fa6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0fa6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1ffff700 1 -- 172.21.15.57:0/1708599779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2a60 msgr2=0x55e54f0fac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2a60 0x55e54f0fac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1ffff700 1 -- 172.21.15.57:0/1708599779 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa201a2bc0 con 0x55e54f0fd910 2023-10-06T22:15:47.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1effd700 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2a60 0x55e54f0fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:47.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0fd910 0x55e54f0fb150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faa18019010 tx=0x7faa18018ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a144f09fe511df37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:47.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.763+0000 7fa9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:47.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.764+0000 7fa9feffd700 1 -- 172.21.15.57:0/1708599779 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800e070 con 0x55e54f0fd910 2023-10-06T22:15:47.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.764+0000 7fa9feffd700 1 -- 172.21.15.57:0/1708599779 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7faa1802c630 con 0x55e54f0fd910 2023-10-06T22:15:47.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.764+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e54f0e5f90 con 0x55e54f0fd910 2023-10-06T22:15:47.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.764+0000 7fa9feffd700 1 -- 172.21.15.57:0/1708599779 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa1802c7f0 con 0x55e54f0fd910 2023-10-06T22:15:47.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.764+0000 7faa39b80b80 10 client.25933 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:47.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.764+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7faa18007b40 con 0x55e54f0fd910 2023-10-06T22:15:47.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.765+0000 7fa9feffd700 1 -- 172.21.15.57:0/1708599779 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e54f0e5f90 con 0x55e54f0fd910 2023-10-06T22:15:47.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7fa9feffd700 1 -- 172.21.15.57:0/1708599779 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faa1802a040 con 0x55e54f0fd910 2023-10-06T22:15:47.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7faa39b80b80 10 client.25933 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:47.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7faa39b80b80 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:47.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7faa39b80b80 20 client.25933 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:47.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f10d280 0x55e54f10f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7faa39b80b80 4 client.25933 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:47.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e54f03ee90 con 0x55e54f10d280 2023-10-06T22:15:47.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.766+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f10d280 0x55e54f10f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.767+0000 7fa9feffd700 10 client.25933 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:47.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.767+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f10d280 0x55e54f10f6a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4cc651d5adab76 server_cookie=bdb98e714b8a111c in_seq=0 out_seq=0 2023-10-06T22:15:47.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7fa9feffd700 1 -- 172.21.15.57:0/1708599779 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e54f03ee90 con 0x55e54f10d280 2023-10-06T22:15:47.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7fa9feffd700 10 client.25933 handle_command_reply: tid=0 2023-10-06T22:15:47.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 client.25933 shutdown 2023-10-06T22:15:47.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0fd910 msgr2=0x55e54f0fb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0fd910 0x55e54f0fb150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faa18019010 tx=0x7faa18018ef0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 shutdown_connections 2023-10-06T22:15:47.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f10d280 0x55e54f10f6a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0fd910 0x55e54f0fb150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1708599779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2a60 0x55e54f0fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 >> 172.21.15.57:0/1708599779 conn(0x55e54f0672e0 msgr2=0x55e54f0ee060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.768+0000 7fa9feffd700 0 client.25933 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.769+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 shutdown_connections 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.769+0000 7faa39b80b80 1 -- 172.21.15.57:0/1708599779 wait complete. 2023-10-06T22:15:47.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.769+0000 7faa39b80b80 20 client.25933 trim_cache size 0 max 16384 2023-10-06T22:15:47.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.804+0000 7faa39b80b80 1 Processor -- start 2023-10-06T22:15:47.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.804+0000 7faa39b80b80 1 -- start start 2023-10-06T22:15:47.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f0f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa39b80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54eec69a0 con 0x55e54f0fd910 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa39b80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f0e5b60 con 0x55e54f0f2140 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa39b80b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f0ed2c0 con 0x55e54f0f2a60 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0f9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:58116/0 (socket says 172.21.15.57:58116) 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa1f7fe700 1 -- 172.21.15.57:0/1483456748 learned_addr learned my addr 172.21.15.57:0/1483456748 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.805+0000 7faa1effd700 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f0f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.806+0000 7faa1f7fe700 1 -- 172.21.15.57:0/1483456748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 msgr2=0x55e54f0f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:47.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.806+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f0f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.806+0000 7faa1f7fe700 1 -- 172.21.15.57:0/1483456748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 msgr2=0x55e54f0fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:47.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.806+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0fada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.806+0000 7faa1f7fe700 1 -- 172.21.15.57:0/1483456748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54f0fa240 con 0x55e54f0f2140 2023-10-06T22:15:47.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.806+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0f9c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e54f10ffc0 tx=0x7faa10009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7531c363576025b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:47.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.806+0000 7faa067fc700 1 -- 172.21.15.57:0/1483456748 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10017040 con 0x55e54f0f2140 2023-10-06T22:15:47.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.807+0000 7faa067fc700 1 -- 172.21.15.57:0/1483456748 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa100194b0 con 0x55e54f0f2140 2023-10-06T22:15:47.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.807+0000 7faa067fc700 1 -- 172.21.15.57:0/1483456748 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10003cb0 con 0x55e54f0f2140 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.807+0000 7faa39b80b80 1 -- 172.21.15.57:0/1483456748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 msgr2=0x55e54f0f9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.807+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0f9c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e54f10ffc0 tx=0x7faa10009260 comp rx=0 tx=0).stop 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 -- 172.21.15.57:0/1483456748 shutdown_connections 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0fd910 0x55e54f0fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0f2140 0x55e54f0f9c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1483456748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f0f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 -- 172.21.15.57:0/1483456748 >> 172.21.15.57:0/1483456748 conn(0x55e54f0672e0 msgr2=0x55e54f0fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 -- 172.21.15.57:0/1483456748 shutdown_connections 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 -- 172.21.15.57:0/1483456748 wait complete. 2023-10-06T22:15:47.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.808+0000 7faa39b80b80 1 -- start start 2023-10-06T22:15:47.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.809+0000 7faa39b80b80 1 Processor -- start 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.809+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2140 0x55e54f104580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa39b80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0fd910 0x55e54f103100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa39b80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f1037e0 con 0x55e54f0f2140 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa39b80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f161100 con 0x55e54f0fd910 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa39b80b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54f161240 con 0x55e54f0f2a60 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa39b80b80 10 client.0 _ll_register_callbacks cb 0x7faa14003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f1029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58970/0 (socket says 172.21.15.57:58970) 2023-10-06T22:15:47.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1effd700 1 -- 172.21.15.57:0/1124692108 learned_addr learned my addr 172.21.15.57:0/1124692108 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0fd910 0x55e54f103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2140 0x55e54f104580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1ffff700 1 -- 172.21.15.57:0/1124692108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 msgr2=0x55e54f1029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f1029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1ffff700 1 -- 172.21.15.57:0/1124692108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2140 msgr2=0x55e54f104580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2140 0x55e54f104580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.810+0000 7faa1ffff700 1 -- 172.21.15.57:0/1124692108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa10009c10 con 0x55e54f0fd910 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.811+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2140 0x55e54f104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:47.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.811+0000 7faa1ffff700 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0fd910 0x55e54f103100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa18007b10 tx=0x7faa18018e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f835649f1283b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.811+0000 7fa9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.811+0000 7fa9feffd700 1 -- 172.21.15.57:0/1124692108 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800e040 con 0x55e54f0fd910 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.811+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa18009b50 con 0x55e54f0fd910 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.811+0000 7fa9feffd700 1 -- 172.21.15.57:0/1124692108 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7faa18004c90 con 0x55e54f0fd910 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7fa9feffd700 1 -- 172.21.15.57:0/1124692108 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa1802c840 con 0x55e54f0fd910 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 10 client.25940 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e54f0ea1c0 con 0x55e54f0fd910 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7fa9feffd700 1 -- 172.21.15.57:0/1124692108 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa18009b50 con 0x55e54f0fd910 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7fa9feffd700 1 -- 172.21.15.57:0/1124692108 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faa180186b0 con 0x55e54f0fd910 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 10 client.25940 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:47.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 20 client.25940 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f162520 0x55e54f164940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 4 client.25940 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T22:15:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.812+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e54f03ebb0 con 0x55e54f162520 2023-10-06T22:15:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.813+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f162520 0x55e54f164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.814+0000 7faa1f7fe700 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f162520 0x55e54f164940 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ceea7655ca7aff2 server_cookie=f35a9de7b70328d2 in_seq=0 out_seq=0 2023-10-06T22:15:47.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.814+0000 7fa9feffd700 10 client.25940 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:47.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7fa9feffd700 1 -- 172.21.15.57:0/1124692108 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e54f03ebb0 con 0x55e54f162520 2023-10-06T22:15:47.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7fa9feffd700 10 client.25940 handle_command_reply: tid=0 2023-10-06T22:15:47.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 client.25940 shutdown 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0fd910 msgr2=0x55e54f103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0fd910 0x55e54f103100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa18007b10 tx=0x7faa18018e20 comp rx=0 tx=0).stop 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 shutdown_connections 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e54f162520 0x55e54f164940 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e54f0fd910 0x55e54f103100 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e54f0f2140 0x55e54f104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 --2- 172.21.15.57:0/1124692108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e54f0f2a60 0x55e54f1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 >> 172.21.15.57:0/1124692108 conn(0x55e54f0672e0 msgr2=0x55e54f0faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.815+0000 7fa9feffd700 0 client.25940 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.816+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 shutdown_connections 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.816+0000 7faa39b80b80 1 -- 172.21.15.57:0/1124692108 wait complete. 2023-10-06T22:15:47.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.816+0000 7faa39b80b80 20 client.25940 trim_cache size 0 max 16384 2023-10-06T22:15:47.820 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:47.820 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-06T22:15:47.820 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "7c351d07-1e5e-40b4-94e6-3b0ad6c9ab56", 2023-10-06T22:15:47.820 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-06T22:15:47.820 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7faa080a1790 msgr2=0x7faa080a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7faa080a1790 0x7faa080a3c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa18018990 tx=0x7faa1800b000 comp rx=0 tx=0).stop 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20102910 msgr2=0x7faa2019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20102910 0x7faa2019db70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa14005320 tx=0x7faa14005400 comp rx=0 tx=0).stop 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 shutdown_connections 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7faa080a1790 0x7faa080a3c50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa2010b410 0x7faa201a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa20101ff0 0x7faa2019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 --2- 172.21.15.57:0/2829355640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20102910 0x7faa2019db70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:47.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 >> 172.21.15.57:0/2829355640 conn(0x7faa200fb830 msgr2=0x7faa2010d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 shutdown_connections 2023-10-06T22:15:47.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:47.817+0000 7faa261ff700 1 -- 172.21.15.57:0/2829355640 wait complete. 2023-10-06T22:15:47.849 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 -- 172.21.15.57:0/2079548054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d7810d680 msgr2=0x7f1d7810faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 --2- 172.21.15.57:0/2079548054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d7810d680 0x7f1d7810faf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000b0a0 tx=0x7f1d7000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 -- 172.21.15.57:0/2079548054 shutdown_connections 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 --2- 172.21.15.57:0/2079548054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d7810d680 0x7f1d7810faf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 --2- 172.21.15.57:0/2079548054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d780732e0 0x7f1d780736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 --2- 172.21.15.57:0/2079548054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d78073c00 0x7f1d7810d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 -- 172.21.15.57:0/2079548054 >> 172.21.15.57:0/2079548054 conn(0x7f1d780fc950 msgr2=0x7f1d780fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 -- 172.21.15.57:0/2079548054 shutdown_connections 2023-10-06T22:15:48.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.193+0000 7f1d7edff700 1 -- 172.21.15.57:0/2079548054 wait complete. 2023-10-06T22:15:48.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.194+0000 7f1d7edff700 1 Processor -- start 2023-10-06T22:15:48.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.194+0000 7f1d7edff700 1 -- start start 2023-10-06T22:15:48.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d780732e0 0x7f1d78199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d78073c00 0x7f1d781997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d7810d680 0x7f1d7819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7edff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7819e260 con 0x7f1d7810d680 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7edff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7819e3d0 con 0x7f1d78073c00 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7edff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7819e590 con 0x7f1d780732e0 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d780732e0 0x7f1d78199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d780732e0 0x7f1d78199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58988/0 (socket says 172.21.15.57:58988) 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d7810d680 0x7f1d7819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/1726413520 learned_addr learned my addr 172.21.15.57:0/1726413520 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d77fff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d78073c00 0x7f1d781997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7d39c700 1 -- 172.21.15.57:0/1726413520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d780732e0 msgr2=0x7f1d78199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7d39c700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d780732e0 0x7f1d78199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7d39c700 1 -- 172.21.15.57:0/1726413520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d78073c00 msgr2=0x7f1d781997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7d39c700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d78073c00 0x7f1d781997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7d39c700 1 -- 172.21.15.57:0/1726413520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d70009d00 con 0x7f1d7810d680 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.195+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d780732e0 0x7f1d78199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.196+0000 7f1d7d39c700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d7810d680 0x7f1d7819dc20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009fd0 tx=0x7f1d70007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.196+0000 7f1d75ffb700 1 -- 172.21.15.57:0/1726413520 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000e040 con 0x7f1d7810d680 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.196+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7819e810 con 0x7f1d7810d680 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.196+0000 7f1d75ffb700 1 -- 172.21.15.57:0/1726413520 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d7001d470 con 0x7f1d7810d680 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.196+0000 7f1d75ffb700 1 -- 172.21.15.57:0/1726413520 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d700052a0 con 0x7f1d7810d680 2023-10-06T22:15:48.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.196+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d781a60b0 con 0x7f1d7810d680 2023-10-06T22:15:48.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.201+0000 7f1d9273ab80 1 Processor -- start 2023-10-06T22:15:48.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.201+0000 7f1d75ffb700 1 -- 172.21.15.57:0/1726413520 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f1d7001d5e0 con 0x7f1d7810d680 2023-10-06T22:15:48.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.201+0000 7f1d9273ab80 1 -- start start 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602600c0 0x5611602604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602609e0 0x56116026b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56116026b890 0x56116026dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611601c9460 con 0x5611602600c0 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611601bac20 con 0x56116026b890 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611601b8540 con 0x5611602609e0 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602600c0 0x5611602604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602600c0 0x5611602604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34182/0 (socket says 172.21.15.57:34182) 2023-10-06T22:15:48.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d77fff700 1 -- 172.21.15.57:0/757683280 learned_addr learned my addr 172.21.15.57:0/757683280 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602609e0 0x56116026b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.202+0000 7f1d77fff700 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56116026b890 0x56116026dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/757683280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56116026b890 msgr2=0x56116026dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56116026b890 0x56116026dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/757683280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602600c0 msgr2=0x5611602604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602600c0 0x5611602604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/757683280 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115ffdede0 con 0x5611602609e0 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d77fff700 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602600c0 0x5611602604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:48.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602609e0 0x56116026b350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c011570 tx=0x7f1d6c011880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60cb668c1047b90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d75ffb700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1d680a1710 0x7f1d680a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d62ffd700 1 -- 172.21.15.57:0/757683280 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c004230 con 0x5611602609e0 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d62ffd700 1 -- 172.21.15.57:0/757683280 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6c004390 con 0x5611602609e0 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d62ffd700 1 -- 172.21.15.57:0/757683280 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c019830 con 0x5611602609e0 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d75ffb700 1 -- 172.21.15.57:0/1726413520 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f1d700ce5b0 con 0x7f1d7810d680 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.203+0000 7f1d77fff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1d680a1710 0x7f1d680a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 -- 172.21.15.57:0/757683280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602609e0 msgr2=0x56116026b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602609e0 0x56116026b350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c011570 tx=0x7f1d6c011880 comp rx=0 tx=0).stop 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 -- 172.21.15.57:0/757683280 shutdown_connections 2023-10-06T22:15:48.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d77fff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1d680a1710 0x7f1d680a3bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1d64005950 tx=0x7f1d640058e0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56116026b890 0x56116026dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602600c0 0x5611602604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/757683280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602609e0 0x56116026b350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 -- 172.21.15.57:0/757683280 >> 172.21.15.57:0/757683280 conn(0x5611601d5260 msgr2=0x56116025c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 -- 172.21.15.57:0/757683280 shutdown_connections 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.204+0000 7f1d9273ab80 1 -- 172.21.15.57:0/757683280 wait complete. 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.205+0000 7f1d9273ab80 1 -- start start 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.205+0000 7f1d9273ab80 1 Processor -- start 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x56116026b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x561160268490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160268aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611601b3e30 con 0x56116026b890 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561160150710 con 0x5611602609e0 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611601503b0 con 0x5611602600c0 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d9273ab80 10 client.0 _ll_register_callbacks cb 0x56115ffad800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160268aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x56116026b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160268aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34190/0 (socket says 172.21.15.57:34190) 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7d39c700 1 -- 172.21.15.57:0/703762753 learned_addr learned my addr 172.21.15.57:0/703762753 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d77fff700 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x561160268490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/703762753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 msgr2=0x561160268490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x561160268490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/703762753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 msgr2=0x561160268aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160268aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/703762753 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64009db0 con 0x5611602600c0 2023-10-06T22:15:48.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.206+0000 7f1d7d39c700 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160268aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:48.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x56116026b280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009850 tx=0x7f1d6c005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e90f0aec90f3ef0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:48.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d57fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:15:48.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d57fff700 1 -- 172.21.15.57:0/703762753 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c004950 con 0x5611602600c0 2023-10-06T22:15:48.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561160253f10 con 0x5611602600c0 2023-10-06T22:15:48.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d57fff700 1 -- 172.21.15.57:0/703762753 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f1d6c004ab0 con 0x5611602600c0 2023-10-06T22:15:48.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d57fff700 1 -- 172.21.15.57:0/703762753 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d6c004c70 con 0x5611602600c0 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d9273ab80 10 client.25951 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.207+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f1d6c009d60 con 0x5611602600c0 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d57fff700 1 -- 172.21.15.57:0/703762753 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561160253f10 con 0x5611602600c0 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d57fff700 1 -- 172.21.15.57:0/703762753 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d6c021040 con 0x5611602600c0 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d9273ab80 10 client.25951 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d9273ab80 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d9273ab80 20 client.25951 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56116027b200 0x56116027d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d9273ab80 4 client.25951 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611601ace10 con 0x56116027b200 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.208+0000 7f1d77fff700 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56116027b200 0x56116027d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.209+0000 7f1d77fff700 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56116027b200 0x56116027d620 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700b0adb09c0ef7a server_cookie=96931aec4d4f33dc in_seq=0 out_seq=0 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.209+0000 7f1d57fff700 10 client.25951 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d57fff700 1 -- 172.21.15.57:0/703762753 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611601ace10 con 0x56116027b200 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d57fff700 10 client.25951 handle_command_reply: tid=0 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 client.25951 shutdown 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 msgr2=0x56116026b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x56116026b280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009850 tx=0x7f1d6c005360 comp rx=0 tx=0).stop 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 shutdown_connections 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56116027b200 0x56116027d620 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160268aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x56116026b280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d57fff700 0 client.25951 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:48.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/703762753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x561160268490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 >> 172.21.15.57:0/703762753 conn(0x5611601d5260 msgr2=0x56116025bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:48.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 shutdown_connections 2023-10-06T22:15:48.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 1 -- 172.21.15.57:0/703762753 wait complete. 2023-10-06T22:15:48.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.210+0000 7f1d9273ab80 20 client.25951 trim_cache size 0 max 16384 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.228+0000 7f1d9273ab80 1 Processor -- start 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.228+0000 7f1d9273ab80 1 -- start start 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x5611602723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x56116026adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160267d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561160034920 con 0x56116026b890 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561160261350 con 0x5611602609e0 2023-10-06T22:15:48.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561160272910 con 0x5611602600c0 2023-10-06T22:15:48.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d7d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160267d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d7cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x5611602723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d7d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160267d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34204/0 (socket says 172.21.15.57:34204) 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d7d39c700 1 -- 172.21.15.57:0/3044637993 learned_addr learned my addr 172.21.15.57:0/3044637993 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.229+0000 7f1d77fff700 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x56116026adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/3044637993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 msgr2=0x56116026adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x56116026adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/3044637993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 msgr2=0x561160267d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160267d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d7cb9b700 1 -- 172.21.15.57:0/3044637993 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561160258140 con 0x5611602600c0 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d7d39c700 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160267d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:48.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x5611602723d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0118c0 tx=0x7f1d6c005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f900808b3bf05c60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:48.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d637fe700 1 -- 172.21.15.57:0/3044637993 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c003880 con 0x5611602600c0 2023-10-06T22:15:48.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d637fe700 1 -- 172.21.15.57:0/3044637993 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6c005480 con 0x5611602600c0 2023-10-06T22:15:48.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.230+0000 7f1d637fe700 1 -- 172.21.15.57:0/3044637993 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c004ac0 con 0x5611602600c0 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 -- 172.21.15.57:0/3044637993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 msgr2=0x5611602723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x5611602723d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0118c0 tx=0x7f1d6c005810 comp rx=0 tx=0).stop 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 -- 172.21.15.57:0/3044637993 shutdown_connections 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56116026b890 0x561160267d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611602600c0 0x5611602723d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/3044637993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602609e0 0x56116026adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 -- 172.21.15.57:0/3044637993 >> 172.21.15.57:0/3044637993 conn(0x5611601d5260 msgr2=0x5611602637b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 -- 172.21.15.57:0/3044637993 shutdown_connections 2023-10-06T22:15:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 -- 172.21.15.57:0/3044637993 wait complete. 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.231+0000 7f1d9273ab80 1 -- start start 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 1 Processor -- start 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602600c0 0x56116026e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602609e0 0x56116026e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56116026b890 0x56116026ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611602727e0 con 0x5611602609e0 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561160272950 con 0x5611602600c0 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611602ce020 con 0x56116026b890 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d9273ab80 10 client.0 _ll_register_callbacks cb 0x7f1d64009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602609e0 0x56116026e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602609e0 0x56116026e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34218/0 (socket says 172.21.15.57:34218) 2023-10-06T22:15:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d77fff700 1 -- 172.21.15.57:0/2912730384 learned_addr learned my addr 172.21.15.57:0/2912730384 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.232+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602600c0 0x56116026e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d7d39c700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56116026b890 0x56116026ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d77fff700 1 -- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56116026b890 msgr2=0x56116026ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d77fff700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56116026b890 0x56116026ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d77fff700 1 -- 172.21.15.57:0/2912730384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602600c0 msgr2=0x56116026e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d77fff700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602600c0 0x56116026e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d77fff700 1 -- 172.21.15.57:0/2912730384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d781a60b0 con 0x5611602609e0 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d7d39c700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56116026b890 0x56116026ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d77fff700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602609e0 0x56116026e8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1d64005c20 tx=0x7f1d640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935fa15f893ae6d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d57fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.233+0000 7f1d57fff700 1 -- 172.21.15.57:0/2912730384 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64026040 con 0x5611602609e0 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611601acb30 con 0x5611602609e0 2023-10-06T22:15:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d57fff700 1 -- 172.21.15.57:0/2912730384 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f1d64011a50 con 0x5611602609e0 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d57fff700 1 -- 172.21.15.57:0/2912730384 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d64011c10 con 0x5611602609e0 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 10 client.15957 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f1d6c009d60 con 0x5611602609e0 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d57fff700 1 -- 172.21.15.57:0/2912730384 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611601acb30 con 0x5611602609e0 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d57fff700 1 -- 172.21.15.57:0/2912730384 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d64021410 con 0x5611602609e0 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 20 client.15957 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611602cf7e0 0x5611602d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 4 client.15957 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.234+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56115fffff70 con 0x5611602cf7e0 2023-10-06T22:15:48.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.235+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611602cf7e0 0x5611602d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.235+0000 7f1d7cb9b700 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611602cf7e0 0x5611602d1c00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbcb5d0179640877 server_cookie=ec217b42e3c4aeb7 in_seq=0 out_seq=0 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.235+0000 7f1d57fff700 10 client.15957 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.235+0000 7f1d57fff700 1 -- 172.21.15.57:0/2912730384 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56115fffff70 con 0x5611602cf7e0 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.235+0000 7f1d57fff700 10 client.15957 handle_command_reply: tid=0 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.235+0000 7f1d9273ab80 1 client.15957 shutdown 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602609e0 msgr2=0x56116026e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602609e0 0x56116026e8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1d64005c20 tx=0x7f1d640047c0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 shutdown_connections 2023-10-06T22:15:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611602cf7e0 0x5611602d1c00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56116026b890 0x56116026ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5611602600c0 0x56116026e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 --2- 172.21.15.57:0/2912730384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5611602609e0 0x56116026e8f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d57fff700 0 client.15957 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 >> 172.21.15.57:0/2912730384 conn(0x5611601d5260 msgr2=0x561160261fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 shutdown_connections 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 1 -- 172.21.15.57:0/2912730384 wait complete. 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.236+0000 7f1d9273ab80 20 client.15957 trim_cache size 0 max 16384 2023-10-06T22:15:48.243 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: "7c351d07-1e5e-40b4-94e6-3b0ad6c9ab56": { 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "7c351d07-1e5e-40b4-94e6-3b0ad6c9ab56", 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:15:48.244 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1d680a1710 msgr2=0x7f1d680a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1d680a1710 0x7f1d680a3bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1d64005950 tx=0x7f1d640058e0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d7810d680 msgr2=0x7f1d7819dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d7810d680 0x7f1d7819dc20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009fd0 tx=0x7f1d70007930 comp rx=0 tx=0).stop 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 shutdown_connections 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1d680a1710 0x7f1d680a3bd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d7810d680 0x7f1d7819dc20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d780732e0 0x7f1d78199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 --2- 172.21.15.57:0/1726413520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d78073c00 0x7f1d781997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:48.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 >> 172.21.15.57:0/1726413520 conn(0x7f1d780fc950 msgr2=0x7f1d7810f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:48.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 shutdown_connections 2023-10-06T22:15:48.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:15:48.237+0000 7f1d7edff700 1 -- 172.21.15.57:0/1726413520 wait complete. 2023-10-06T22:15:48.257 INFO:tasks.cephfs.filesystem:scrub status for tag:7c351d07-1e5e-40b4-94e6-3b0ad6c9ab56 - {'path': '/', 'tag': '7c351d07-1e5e-40b4-94e6-3b0ad6c9ab56', 'options': 'recursive,force'} 2023-10-06T22:15:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:48 smithi156 ceph-mon[134358]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T22:15:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:48 smithi156 ceph-mon[134358]: scrub queued for path: / 2023-10-06T22:15:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:48 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:48 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:15:49.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:48 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:49.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:48 smithi190 ceph-mon[134891]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T22:15:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:48 smithi190 ceph-mon[134891]: scrub queued for path: / 2023-10-06T22:15:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:48 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:48 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:15:49.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:48 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:48 smithi057 ceph-mon[98492]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-06T22:15:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:48 smithi057 ceph-mon[98492]: scrub queued for path: / 2023-10-06T22:15:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:48 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:48 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:15:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:48 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:15:50.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:49 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:15:50.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:49 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:15:50.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:49 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:15:51.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:50 smithi156 ceph-mon[134358]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T22:15:51.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:50 smithi190 ceph-mon[134891]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T22:15:51.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:50 smithi057 ceph-mon[98492]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T22:15:51.708 INFO:tasks.workunit.client.0.smithi057.stderr:Updating files: 61% (7473/12065) Updating files: 62% (7481/12065) Updating files: 63% (7601/12065) Updating files: 64% (7722/12065) Updating files: 65% (7843/12065) Updating files: 66% (7963/12065) Updating files: 67% (8084/12065) Updating files: 68% (8205/12065) Updating files: 69% (8325/12065) Updating files: 70% (8446/12065) Updating files: 71% (8567/12065) Updating files: 72% (8687/12065) Updating files: 73% (8808/12065) Updating files: 74% (8929/12065) Updating files: 75% (9049/12065) Updating files: 76% (9170/12065) Updating files: 77% (9291/12065) Updating files: 78% (9411/12065) Updating files: 79% (9532/12065) Updating files: 80% (9652/12065) Updating files: 81% (9773/12065) Updating files: 82% (9894/12065) Updating files: 83% (10014/12065) Updating files: 84% (10135/12065) Updating files: 85% (10256/12065) Updating files: 86% (10376/12065) Updating files: 87% (10497/12065) Updating files: 88% (10618/12065) Updating files: 89% (10738/12065) Updating files: 90% (10859/12065) Updating files: 91% (10980/12065) Updating files: 92% (11100/12065) Updating files: 93% (11221/12065) Updating files: 94% (11342/12065) Updating files: 95% (11462/12065) Updating files: 96% (11583/12065) Updating files: 97% (11704/12065) Updating files: 98% (11824/12065) Updating files: 98% (11920/12065) Updating files: 99% (11945/12065) Updating files: 100% (12065/12065) Updating files: 100% (12065/12065), done. 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr:Note: switching to '14d8d032507701c76ca224e769ee65a96b660a8c'. 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr:state without impacting any branches by switching back to a branch. 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-06T22:15:52.195 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr: git switch -c 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr:Or undo this operation with: 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr: git switch - 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:52.196 INFO:tasks.workunit.client.0.smithi057.stderr:HEAD is now at 14d8d032507 Merge remote-tracking branch 'rishabh/ceph-auth-caps-val-caps' into rishabh-cephfs-mon 2023-10-06T22:15:52.198 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-06T22:15:52.253 INFO:tasks.workunit.client.0.smithi057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-06T22:15:52.258 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T22:15:52.258 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-06T22:15:52.830 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-06T22:15:52.931 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-06T22:15:52.995 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T22:15:52.999 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T22:15:52.999 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-06T22:15:53.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:52 smithi156 ceph-mon[134358]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T22:15:53.062 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T22:15:53.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:52 smithi190 ceph-mon[134891]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T22:15:53.070 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:15:53.070 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-06T22:15:53.113 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-10-06T22:15:53.115 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-10-06T22:15:53.115 DEBUG:teuthology.orchestra.run.smithi057:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=14d8d032507701c76ca224e769ee65a96b660a8c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-10-06T22:15:53.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:52 smithi057 ceph-mon[98492]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-10-06T22:15:53.212 INFO:tasks.workunit.client.0.smithi057.stdout:getting blogbench 2023-10-06T22:15:53.213 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'getting blogbench' 2023-10-06T22:15:53.213 INFO:tasks.workunit.client.0.smithi057.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-10-06T22:15:53.259 INFO:tasks.workunit.client.0.smithi057.stderr:--2023-10-06 22:15:53-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-10-06T22:15:53.269 INFO:tasks.workunit.client.0.smithi057.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-10-06T22:15:53.291 INFO:tasks.workunit.client.0.smithi057.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-10-06T22:15:53.311 INFO:tasks.workunit.client.0.smithi057.stderr:HTTP request sent, awaiting response... 200 OK 2023-10-06T22:15:53.312 INFO:tasks.workunit.client.0.smithi057.stderr:Length: 103355 (101K) [application/octet-stream] 2023-10-06T22:15:53.313 INFO:tasks.workunit.client.0.smithi057.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-10-06T22:15:53.317 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:53.353 INFO:tasks.workunit.client.0.smithi057.stderr: 0K .......... .......... .......... .......... .......... 49% 1.25M 0s 2023-10-06T22:15:53.374 INFO:tasks.workunit.client.0.smithi057.stderr: 50K .......... .......... .......... .......... .......... 99% 2.40M 0s 2023-10-06T22:15:53.374 INFO:tasks.workunit.client.0.smithi057.stderr: 100K 100% 1779G=0.06s 2023-10-06T22:15:53.374 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:53.374 INFO:tasks.workunit.client.0.smithi057.stderr:2023-10-06 22:15:53 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-10-06T22:15:53.374 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-06T22:15:53.374 INFO:tasks.workunit.client.0.smithi057.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-10-06T22:15:53.409 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/ 2023-10-06T22:15:53.411 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/m4/ 2023-10-06T22:15:53.412 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-10-06T22:15:53.414 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/ 2023-10-06T22:15:53.415 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/blogbench.8 2023-10-06T22:15:53.416 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/Makefile.am 2023-10-06T22:15:53.416 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/Makefile.in 2023-10-06T22:15:53.416 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/ 2023-10-06T22:15:53.419 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/mysnprintf.c 2023-10-06T22:15:53.421 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/mysnprintf.h 2023-10-06T22:15:53.421 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench.c 2023-10-06T22:15:53.421 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench.h 2023-10-06T22:15:53.421 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/fakesnprintf.c 2023-10-06T22:15:53.421 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/fakesnprintf.h 2023-10-06T22:15:53.421 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench_p.h 2023-10-06T22:15:53.422 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-10-06T22:15:53.422 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/Makefile.am 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/Makefile.in 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/process.c 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/globals.h 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/commenter.c 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/rewriter.c 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/reader.c 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/writer.c 2023-10-06T22:15:53.423 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/helpers.c 2023-10-06T22:15:53.424 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/NEWS 2023-10-06T22:15:53.434 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/compile 2023-10-06T22:15:53.434 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/depcomp 2023-10-06T22:15:53.434 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/aclocal.m4 2023-10-06T22:15:53.434 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/README 2023-10-06T22:15:53.435 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/configure 2023-10-06T22:15:53.437 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/configure.ac 2023-10-06T22:15:53.438 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.guess 2023-10-06T22:15:53.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/install-sh 2023-10-06T22:15:53.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.sub 2023-10-06T22:15:53.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/missing 2023-10-06T22:15:53.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/mkinstalldirs 2023-10-06T22:15:53.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/Makefile.am 2023-10-06T22:15:53.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/Makefile.in 2023-10-06T22:15:53.440 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.h.in 2023-10-06T22:15:53.440 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/AUTHORS 2023-10-06T22:15:53.440 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/INSTALL 2023-10-06T22:15:53.440 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/ChangeLog 2023-10-06T22:15:53.440 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/COPYING 2023-10-06T22:15:53.440 INFO:tasks.workunit.client.0.smithi057.stderr:+ cd blogbench-1.0/ 2023-10-06T22:15:53.440 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'making blogbench' 2023-10-06T22:15:53.441 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./configure 2023-10-06T22:15:53.441 INFO:tasks.workunit.client.0.smithi057.stdout:making blogbench 2023-10-06T22:15:53.669 INFO:tasks.workunit.client.0.smithi057.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-06T22:15:54.680 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether build environment is sane... yes 2023-10-06T22:15:54.697 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gawk... gawk 2023-10-06T22:15:54.726 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether make sets $(MAKE)... yes 2023-10-06T22:15:54.757 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gcc... gcc 2023-10-06T22:15:54.859 INFO:tasks.workunit.client.0.smithi057.stdout:checking for C compiler default output file name... a.out 2023-10-06T22:15:54.864 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether the C compiler works... yes 2023-10-06T22:15:54.868 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether we are cross compiling... no 2023-10-06T22:15:54.929 INFO:tasks.workunit.client.0.smithi057.stdout:checking for suffix of executables... 2023-10-06T22:15:54.995 INFO:tasks.workunit.client.0.smithi057.stdout:checking for suffix of object files... o 2023-10-06T22:15:55.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:54 smithi156 ceph-mon[134358]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T22:15:55.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:15:55.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:55.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:54 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:55.060 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether we are using the GNU C compiler... yes 2023-10-06T22:15:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:54 smithi190 ceph-mon[134891]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T22:15:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:15:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:54 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:55.130 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether gcc accepts -g... yes 2023-10-06T22:15:55.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:54 smithi057 ceph-mon[98492]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-10-06T22:15:55.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:15:55.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:55.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:54 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:55.204 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gcc option to accept ANSI C... none needed 2023-10-06T22:15:55.270 INFO:tasks.workunit.client.0.smithi057.stdout:checking for style of include used by make... GNU 2023-10-06T22:15:55.396 INFO:tasks.workunit.client.0.smithi057.stdout:checking dependency style of gcc... gcc3 2023-10-06T22:15:55.396 INFO:tasks.workunit.client.0.smithi057.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-06T22:15:55.474 INFO:tasks.workunit.client.0.smithi057.stdout:checking how to run the C preprocessor... gcc -E 2023-10-06T22:15:55.543 INFO:tasks.workunit.client.0.smithi057.stdout:checking for egrep... grep -E 2023-10-06T22:15:55.560 INFO:tasks.workunit.client.0.smithi057.stdout:checking for AIX... no 2023-10-06T22:15:55.658 INFO:tasks.workunit.client.0.smithi057.stdout:checking for library containing strerror... none required 2023-10-06T22:15:55.888 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ANSI C header files... yes 2023-10-06T22:15:55.970 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-10-06T22:15:56.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:55 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:15:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:55 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:15:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:55 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:56.052 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/types.h... yes 2023-10-06T22:15:56.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:55 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:15:56.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:55 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:15:56.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:55 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:56.140 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/stat.h... yes 2023-10-06T22:15:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:55 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:15:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:55 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:15:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:55 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:15:56.229 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdlib.h... yes 2023-10-06T22:15:56.319 INFO:tasks.workunit.client.0.smithi057.stdout:checking for string.h... yes 2023-10-06T22:15:56.416 INFO:tasks.workunit.client.0.smithi057.stdout:checking for memory.h... yes 2023-10-06T22:15:56.506 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strings.h... yes 2023-10-06T22:15:56.598 INFO:tasks.workunit.client.0.smithi057.stdout:checking for inttypes.h... yes 2023-10-06T22:15:56.690 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdint.h... yes 2023-10-06T22:15:56.787 INFO:tasks.workunit.client.0.smithi057.stdout:checking for unistd.h... yes 2023-10-06T22:15:56.812 INFO:tasks.workunit.client.0.smithi057.stdout:checking for memory.h... (cached) yes 2023-10-06T22:15:56.828 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strings.h... (cached) yes 2023-10-06T22:15:56.844 INFO:tasks.workunit.client.0.smithi057.stdout:checking for unistd.h... (cached) yes 2023-10-06T22:15:56.932 INFO:tasks.workunit.client.0.smithi057.stdout:checking locale.h usability... yes 2023-10-06T22:15:56.965 INFO:tasks.workunit.client.0.smithi057.stdout:checking locale.h presence... yes 2023-10-06T22:15:56.967 INFO:tasks.workunit.client.0.smithi057.stdout:checking for locale.h... yes 2023-10-06T22:15:56.994 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdlib.h... (cached) yes 2023-10-06T22:15:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:56 smithi156 ceph-mon[134358]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T22:15:57.010 INFO:tasks.workunit.client.0.smithi057.stdout:checking for string.h... (cached) yes 2023-10-06T22:15:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:56 smithi190 ceph-mon[134891]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T22:15:57.099 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/socket.h usability... yes 2023-10-06T22:15:57.135 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/socket.h presence... yes 2023-10-06T22:15:57.137 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/socket.h... yes 2023-10-06T22:15:57.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:56 smithi057 ceph-mon[98492]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-06T22:15:57.233 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/time.h usability... yes 2023-10-06T22:15:57.266 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/time.h presence... yes 2023-10-06T22:15:57.268 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/time.h... yes 2023-10-06T22:15:57.364 INFO:tasks.workunit.client.0.smithi057.stdout:checking fcntl.h usability... yes 2023-10-06T22:15:57.397 INFO:tasks.workunit.client.0.smithi057.stdout:checking fcntl.h presence... yes 2023-10-06T22:15:57.401 INFO:tasks.workunit.client.0.smithi057.stdout:checking for fcntl.h... yes 2023-10-06T22:15:57.493 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/fcntl.h usability... yes 2023-10-06T22:15:57.525 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/fcntl.h presence... yes 2023-10-06T22:15:57.527 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/fcntl.h... yes 2023-10-06T22:15:57.633 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/param.h usability... yes 2023-10-06T22:15:57.670 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/param.h presence... yes 2023-10-06T22:15:57.672 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/param.h... yes 2023-10-06T22:15:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 -- 172.21.15.57:0/2211570833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc1061f0 msgr2=0x7f7afc10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 --2- 172.21.15.57:0/2211570833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc1061f0 0x7f7afc10dd70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7af400b0a0 tx=0x7f7af400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 -- 172.21.15.57:0/2211570833 shutdown_connections 2023-10-06T22:15:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 --2- 172.21.15.57:0/2211570833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc1061f0 0x7f7afc10dd70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 --2- 172.21.15.57:0/2211570833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc1010d0 0x7f7afc1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 --2- 172.21.15.57:0/2211570833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc101a80 0x7f7afc105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 -- 172.21.15.57:0/2211570833 >> 172.21.15.57:0/2211570833 conn(0x7f7afc0fc950 msgr2=0x7f7afc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 -- 172.21.15.57:0/2211570833 shutdown_connections 2023-10-06T22:15:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.683+0000 7f7b015ff700 1 -- 172.21.15.57:0/2211570833 wait complete. 2023-10-06T22:15:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 Processor -- start 2023-10-06T22:15:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 -- start start 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc1010d0 0x7f7afc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc1061f0 0x7f7afc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc19dee0 0x7f7afc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc19e960 con 0x7f7afc1010d0 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc19ead0 con 0x7f7afc1061f0 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.684+0000 7f7b015ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc19ec90 con 0x7f7afc19dee0 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc1010d0 0x7f7afc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc19dee0 0x7f7afc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc1061f0 0x7f7afc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc19dee0 0x7f7afc19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48348/0 (socket says 172.21.15.57:48348) 2023-10-06T22:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 -- 172.21.15.57:0/1056611764 learned_addr learned my addr 172.21.15.57:0/1056611764 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc1010d0 0x7f7afc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45756/0 (socket says 172.21.15.57:45756) 2023-10-06T22:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 -- 172.21.15.57:0/1056611764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc1061f0 msgr2=0x7f7afc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc1061f0 0x7f7afc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 -- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc1010d0 msgr2=0x7f7afc199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc1010d0 0x7f7afc199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 -- 172.21.15.57:0/1056611764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009d00 con 0x7f7afc19dee0 2023-10-06T22:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.685+0000 7f7afb7fe700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc19dee0 0x7f7afc19e360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7af400b070 tx=0x7f7af400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.686+0000 7f7ae3fff700 1 -- 172.21.15.57:0/1056611764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af400e040 con 0x7f7afc19dee0 2023-10-06T22:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.686+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc107d20 con 0x7f7afc19dee0 2023-10-06T22:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.686+0000 7f7ae3fff700 1 -- 172.21.15.57:0/1056611764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7af4009480 con 0x7f7afc19dee0 2023-10-06T22:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.686+0000 7f7ae3fff700 1 -- 172.21.15.57:0/1056611764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40052a0 con 0x7f7afc19dee0 2023-10-06T22:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.686+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc108220 con 0x7f7afc19dee0 2023-10-06T22:15:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.689+0000 7f7ae3fff700 1 -- 172.21.15.57:0/1056611764 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f7af401d480 con 0x7f7afc19dee0 2023-10-06T22:15:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 Processor -- start 2023-10-06T22:15:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 -- start start 2023-10-06T22:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd101e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd1047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd060400 con 0x55b6cd1023d0 2023-10-06T22:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd051bc0 con 0x55b6cd0f6c00 2023-10-06T22:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd04f4e0 con 0x55b6cd0f7520 2023-10-06T22:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.691+0000 7f7ae3fff700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7ae40a1710 0x7f7ae40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57980/0 (socket says 172.21.15.57:57980) 2023-10-06T22:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afaffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afa7fc700 1 -- 172.21.15.57:0/1288454143 learned_addr learned my addr 172.21.15.57:0/1288454143 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7ae3fff700 1 -- 172.21.15.57:0/1056611764 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7af40ce2c0 con 0x7f7afc19dee0 2023-10-06T22:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd1047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afb7fe700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7ae40a1710 0x7f7ae40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afaffd700 1 -- 172.21.15.57:0/1288454143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 msgr2=0x55b6cd0f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afaffd700 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afaffd700 1 -- 172.21.15.57:0/1288454143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 msgr2=0x55b6cd1047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afaffd700 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd1047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afaffd700 1 -- 172.21.15.57:0/1288454143 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6cce75d80 con 0x55b6cd0f7520 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.692+0000 7f7afaffd700 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd101e90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009780 tx=0x7f7aec009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed2cc8e079edd9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.693+0000 7f7afb7fe700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7ae40a1710 0x7f7ae40a3bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7af40cb900 tx=0x7f7af400b3b0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.693+0000 7f7ae17fa700 1 -- 172.21.15.57:0/1288454143 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec019070 con 0x55b6cd0f7520 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.693+0000 7f7ae17fa700 1 -- 172.21.15.57:0/1288454143 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aec01a040 con 0x55b6cd0f7520 2023-10-06T22:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.693+0000 7f7ae17fa700 1 -- 172.21.15.57:0/1288454143 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec01e600 con 0x55b6cd0f7520 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 -- 172.21.15.57:0/1288454143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 msgr2=0x55b6cd101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd101e90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009780 tx=0x7f7aec009260 comp rx=0 tx=0).stop 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 -- 172.21.15.57:0/1288454143 shutdown_connections 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/1288454143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd101e90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 -- 172.21.15.57:0/1288454143 >> 172.21.15.57:0/1288454143 conn(0x55b6cd06c200 msgr2=0x55b6cd0f3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 -- 172.21.15.57:0/1288454143 shutdown_connections 2023-10-06T22:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 -- 172.21.15.57:0/1288454143 wait complete. 2023-10-06T22:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.694+0000 7f7b14f89b80 1 -- start start 2023-10-06T22:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 1 Processor -- start 2023-10-06T22:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd1023d0 0x55b6cd0ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd04add0 con 0x55b6cd0f6c00 2023-10-06T22:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ccfe76b0 con 0x55b6cd1023d0 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ccfe7350 con 0x55b6cd0f7520 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7b14f89b80 10 client.0 _ll_register_callbacks cb 0x55b6cce447a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45766/0 (socket says 172.21.15.57:45766) 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.695+0000 7f7afa7fc700 1 -- 172.21.15.57:0/658988581 learned_addr learned my addr 172.21.15.57:0/658988581 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afaffd700 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd1023d0 0x55b6cd0ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afa7fc700 1 -- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 msgr2=0x55b6cd0ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afa7fc700 1 -- 172.21.15.57:0/658988581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd1023d0 msgr2=0x55b6cd0ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd1023d0 0x55b6cd0ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afa7fc700 1 -- 172.21.15.57:0/658988581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec014e00 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd101dd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7af00095c0 tx=0x7f7af00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56285a270dcd633f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7adeff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7adeff5700 1 -- 172.21.15.57:0/658988581 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0013ea0 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6cd0ead50 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.696+0000 7f7adeff5700 1 -- 172.21.15.57:0/658988581 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7af0003c20 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7adeff5700 1 -- 172.21.15.57:0/658988581 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af0003de0 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 10 client.15969 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b6cce5ec00 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7adeff5700 1 -- 172.21.15.57:0/658988581 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6cd0ead50 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7adeff5700 1 -- 172.21.15.57:0/658988581 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7af0005460 con 0x55b6cd0f6c00 2023-10-06T22:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 20 client.15969 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd111ec0 0x55b6cd1142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 4 client.15969 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.697+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6cd043db0 con 0x55b6cd111ec0 2023-10-06T22:15:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.698+0000 7f7afaffd700 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd111ec0 0x55b6cd1142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.698+0000 7f7afaffd700 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd111ec0 0x55b6cd1142e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=521be83c83e971cf server_cookie=6a75a9a47bd3ec7f in_seq=0 out_seq=0 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.698+0000 7f7adeff5700 10 client.15969 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7adeff5700 1 -- 172.21.15.57:0/658988581 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6cd043db0 con 0x55b6cd111ec0 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7adeff5700 10 client.15969 handle_command_reply: tid=0 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 client.15969 shutdown 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 msgr2=0x55b6cd101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd101dd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7af00095c0 tx=0x7f7af00047c0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 shutdown_connections 2023-10-06T22:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd111ec0 0x55b6cd1142e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd1023d0 0x55b6cd0ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd101dd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/658988581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7adeff5700 0 client.15969 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.699+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 >> 172.21.15.57:0/658988581 conn(0x55b6cd06c200 msgr2=0x55b6cd0f2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.700+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 shutdown_connections 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.700+0000 7f7b14f89b80 1 -- 172.21.15.57:0/658988581 wait complete. 2023-10-06T22:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.700+0000 7f7b14f89b80 20 client.15969 trim_cache size 0 max 16384 2023-10-06T22:15:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 Processor -- start 2023-10-06T22:15:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 -- start start 2023-10-06T22:15:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd10e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ccecb8c0 con 0x55b6cd1023d0 2023-10-06T22:15:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd0fffe0 con 0x55b6cd0f6c00 2023-10-06T22:15:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.716+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd0faf00 con 0x55b6cd0f7520 2023-10-06T22:15:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0f17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48380/0 (socket says 172.21.15.57:48380) 2023-10-06T22:15:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 -- 172.21.15.57:0/3870918284 learned_addr learned my addr 172.21.15.57:0/3870918284 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd10e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 -- 172.21.15.57:0/3870918284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 msgr2=0x55b6cd0f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 -- 172.21.15.57:0/3870918284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 msgr2=0x55b6cd10e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd10e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afaffd700 1 -- 172.21.15.57:0/3870918284 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6cce9c1f0 con 0x55b6cd0f7520 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.717+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.718+0000 7f7afaffd700 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0f17b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7aec021040 tx=0x7f7aec009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2980cce26813ced8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.718+0000 7f7ae1ffb700 1 -- 172.21.15.57:0/3870918284 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec02e070 con 0x55b6cd0f7520 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.718+0000 7f7ae1ffb700 1 -- 172.21.15.57:0/3870918284 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aec003b40 con 0x55b6cd0f7520 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.718+0000 7f7ae1ffb700 1 -- 172.21.15.57:0/3870918284 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec019040 con 0x55b6cd0f7520 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3870918284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 msgr2=0x55b6cd0f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0f17b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7aec021040 tx=0x7f7aec009fa0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3870918284 shutdown_connections 2023-10-06T22:15:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd1023d0 0x55b6cd10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3870918284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd0f7520 0x55b6cd0f17b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3870918284 >> 172.21.15.57:0/3870918284 conn(0x55b6cd06c200 msgr2=0x55b6cd118c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3870918284 shutdown_connections 2023-10-06T22:15:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3870918284 wait complete. 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 -- start start 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.719+0000 7f7b14f89b80 1 Processor -- start 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f7520 0x55b6cd0fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7b14f89b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd1023d0 0x55b6cd0fb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd0fcf70 con 0x55b6cd0f6c00 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd0fd0e0 con 0x55b6cd0f7520 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7b14f89b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6cd0fbd90 con 0x55b6cd1023d0 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7b14f89b80 10 client.0 _ll_register_callbacks cb 0x7f7af0004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:15:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7afa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45786/0 (socket says 172.21.15.57:45786) 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7afa7fc700 1 -- 172.21.15.57:0/3056784324 learned_addr learned my addr 172.21.15.57:0/3056784324 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.720+0000 7f7afaffd700 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f7520 0x55b6cd0fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd1023d0 0x55b6cd0fb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7afa7fc700 1 -- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd1023d0 msgr2=0x55b6cd0fb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd1023d0 0x55b6cd0fb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7afa7fc700 1 -- 172.21.15.57:0/3056784324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f7520 msgr2=0x55b6cd0fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f7520 0x55b6cd0fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7afa7fc700 1 -- 172.21.15.57:0/3056784324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6cce75d80 con 0x55b6cd0f6c00 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7afa7fc700 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0fc8e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005950 tx=0x7f7af0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1beb29d6e8637b70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7adeff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:15:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7adeff5700 1 -- 172.21.15.57:0/3056784324 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0013ea0 con 0x55b6cd0f6c00 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7aec01a040 con 0x55b6cd0f6c00 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.721+0000 7f7adeff5700 1 -- 172.21.15.57:0/3056784324 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7af0036060 con 0x55b6cd0f6c00 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7adeff5700 1 -- 172.21.15.57:0/3056784324 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af0003e50 con 0x55b6cd0f6c00 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 10 client.15975 fetch_fsmap learned FSMap version 28 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b6cce74250 con 0x55b6cd0f6c00 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7adeff5700 1 -- 172.21.15.57:0/3056784324 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7af003f020 con 0x55b6cd0f6c00 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7adeff5700 1 -- 172.21.15.57:0/3056784324 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7af0004b70 con 0x55b6cd0f6c00 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 28 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 20 client.15975 populate_metadata read hostname 'smithi057' 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd166d50 0x55b6cd169170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:15:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 4 client.15975 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:15:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.722+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6cd043ad0 con 0x55b6cd166d50 2023-10-06T22:15:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.723+0000 7f7afaffd700 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd166d50 0x55b6cd169170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:15:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.723+0000 7f7afaffd700 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd166d50 0x55b6cd169170 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93853282cd5c05bb server_cookie=c28b05c1e0250679 in_seq=0 out_seq=0 2023-10-06T22:15:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.723+0000 7f7adeff5700 10 client.15975 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.769+0000 7f7adeff5700 1 -- 172.21.15.57:0/3056784324 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6cd043ad0 con 0x55b6cd166d50 2023-10-06T22:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.769+0000 7f7adeff5700 10 client.15975 handle_command_reply: tid=0 2023-10-06T22:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.769+0000 7f7b14f89b80 1 client.15975 shutdown 2023-10-06T22:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 msgr2=0x55b6cd0fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0fc8e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005950 tx=0x7f7af0009c30 comp rx=0 tx=0).stop 2023-10-06T22:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 shutdown_connections 2023-10-06T22:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6cd166d50 0x55b6cd169170 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6cd1023d0 0x55b6cd0fb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6cd0f6c00 0x55b6cd0fc8e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 --2- 172.21.15.57:0/3056784324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6cd0f7520 0x55b6cd0fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 >> 172.21.15.57:0/3056784324 conn(0x55b6cd06c200 msgr2=0x55b6cd10d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7adeff5700 0 client.15975 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 shutdown_connections 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.770+0000 7f7b14f89b80 1 -- 172.21.15.57:0/3056784324 wait complete. 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b14f89b80 20 client.15975 trim_cache size 0 max 16384 2023-10-06T22:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7ae40a1710 msgr2=0x7f7ae40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7ae40a1710 0x7f7ae40a3bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7af40cb900 tx=0x7f7af400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc19dee0 msgr2=0x7f7afc19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc19dee0 0x7f7afc19e360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7af400b070 tx=0x7f7af400bc80 comp rx=0 tx=0).stop 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 shutdown_connections 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7ae40a1710 0x7f7ae40a3bd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7afc19dee0 0x7f7afc19e360 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7afc1010d0 0x7f7afc199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 --2- 172.21.15.57:0/1056611764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7afc1061f0 0x7f7afc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 >> 172.21.15.57:0/1056611764 conn(0x7f7afc0fc950 msgr2=0x7f7afc0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 shutdown_connections 2023-10-06T22:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:15:57.771+0000 7f7b015ff700 1 -- 172.21.15.57:0/1056611764 wait complete. 2023-10-06T22:15:57.777 INFO:tasks.workunit.client.0.smithi057.stdout:checking ioctl.h usability... no 2023-10-06T22:15:57.809 INFO:tasks.workunit.client.0.smithi057.stdout:checking ioctl.h presence... no 2023-10-06T22:15:57.811 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ioctl.h... no 2023-10-06T22:15:57.895 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/ioctl.h usability... yes 2023-10-06T22:15:57.927 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/ioctl.h presence... yes 2023-10-06T22:15:57.929 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/ioctl.h... yes 2023-10-06T22:15:58.021 INFO:tasks.workunit.client.0.smithi057.stdout:checking return type of signal handlers... void 2023-10-06T22:15:58.098 INFO:tasks.workunit.client.0.smithi057.stdout:checking for size_t... yes 2023-10-06T22:15:58.164 INFO:tasks.workunit.client.0.smithi057.stdout:checking for dev_t... yes 2023-10-06T22:15:58.238 INFO:tasks.workunit.client.0.smithi057.stdout:checking for short... yes 2023-10-06T22:15:58.349 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of short... 2 2023-10-06T22:15:58.428 INFO:tasks.workunit.client.0.smithi057.stdout:checking for int... yes 2023-10-06T22:15:58.531 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of int... 4 2023-10-06T22:15:58.609 INFO:tasks.workunit.client.0.smithi057.stdout:checking for long... yes 2023-10-06T22:15:58.769 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of long... 8 2023-10-06T22:15:58.861 INFO:tasks.workunit.client.0.smithi057.stdout:checking for long long... yes 2023-10-06T22:15:58.971 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of long long... 8 2023-10-06T22:15:59.047 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ssize_t... yes 2023-10-06T22:15:59.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:15:58 smithi190 ceph-mon[134891]: pgmap v40: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-10-06T22:15:59.116 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sig_atomic_t... yes 2023-10-06T22:15:59.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:15:58 smithi057 ceph-mon[98492]: pgmap v40: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-10-06T22:15:59.181 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether gcc needs -traditional... no 2023-10-06T22:15:59.243 INFO:tasks.workunit.client.0.smithi057.stdout:checking for an ANSI C-conforming const... yes 2023-10-06T22:15:59.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:15:58 smithi156 ceph-mon[134358]: pgmap v40: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-10-06T22:15:59.291 INFO:tasks.workunit.client.0.smithi057.stdout:checking for inline... inline 2023-10-06T22:15:59.385 INFO:tasks.workunit.client.0.smithi057.stdout:checking for working alloca.h... yes 2023-10-06T22:15:59.467 INFO:tasks.workunit.client.0.smithi057.stdout:checking for alloca... yes 2023-10-06T22:15:59.568 INFO:tasks.workunit.client.0.smithi057.stdout:checking for working memcmp... yes 2023-10-06T22:15:59.662 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strftime... yes 2023-10-06T22:15:59.781 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-10-06T22:15:59.896 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether stat accepts an empty string... no 2023-10-06T22:15:59.992 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vprintf... yes 2023-10-06T22:16:00.086 INFO:tasks.workunit.client.0.smithi057.stdout:checking for _doprnt... no 2023-10-06T22:16:00.143 INFO:tasks.workunit.client.0.smithi057.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-10-06T22:16:00.177 INFO:tasks.workunit.client.0.smithi057.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-10-06T22:16:00.275 INFO:tasks.workunit.client.0.smithi057.stdout:checking for the pthreads library -lpthreads... no 2023-10-06T22:16:00.360 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work without any flags... no 2023-10-06T22:16:00.389 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work with -Kthread... no 2023-10-06T22:16:00.418 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work with -kthread... no 2023-10-06T22:16:00.504 INFO:tasks.workunit.client.0.smithi057.stdout:checking for the pthreads library -llthread... no 2023-10-06T22:16:00.597 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work with -pthread... yes 2023-10-06T22:16:00.678 INFO:tasks.workunit.client.0.smithi057.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-10-06T22:16:00.678 INFO:tasks.workunit.client.0.smithi057.stdout:checking if more special flags are required for pthreads... no 2023-10-06T22:16:00.678 INFO:tasks.workunit.client.0.smithi057.stdout:checking for cc_r... gcc 2023-10-06T22:16:00.794 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setrlimit... yes 2023-10-06T22:16:00.899 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setproctitle... no 2023-10-06T22:16:00.999 INFO:tasks.workunit.client.0.smithi057.stdout:checking for getopt_long... yes 2023-10-06T22:16:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:00 smithi190 ceph-mon[134891]: pgmap v41: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-10-06T22:16:01.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:00 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:01.106 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strtoull... yes 2023-10-06T22:16:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:00 smithi057 ceph-mon[98492]: pgmap v41: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-10-06T22:16:01.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:00 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:01.213 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strtoq... yes 2023-10-06T22:16:01.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:00 smithi156 ceph-mon[134358]: pgmap v41: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-10-06T22:16:01.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:00 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:01.319 INFO:tasks.workunit.client.0.smithi057.stdout:checking for snprintf... yes 2023-10-06T22:16:01.426 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vsnprintf... yes 2023-10-06T22:16:01.534 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vfprintf... yes 2023-10-06T22:16:01.642 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gethostname... yes 2023-10-06T22:16:01.750 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setlocale... yes 2023-10-06T22:16:01.860 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sync... yes 2023-10-06T22:16:01.965 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether snprintf is C99 conformant... done 2023-10-06T22:16:01.989 INFO:tasks.workunit.client.0.smithi057.stdout:configure: creating ./config.status 2023-10-06T22:16:02.279 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating Makefile 2023-10-06T22:16:02.321 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating src/Makefile 2023-10-06T22:16:02.366 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating man/Makefile 2023-10-06T22:16:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:02 smithi190 ceph-mon[134891]: pgmap v42: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-10-06T22:16:03.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:02 smithi057 ceph-mon[98492]: pgmap v42: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-10-06T22:16:03.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:02 smithi156 ceph-mon[134358]: pgmap v42: 97 pgs: 97 active+clean; 2.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-10-06T22:16:04.679 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating config.h 2023-10-06T22:16:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:04 smithi190 ceph-mon[134891]: pgmap v43: 97 pgs: 97 active+clean; 4.7 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 367 KiB/s wr, 99 op/s 2023-10-06T22:16:05.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:04 smithi057 ceph-mon[98492]: pgmap v43: 97 pgs: 97 active+clean; 4.7 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 367 KiB/s wr, 99 op/s 2023-10-06T22:16:05.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:04 smithi156 ceph-mon[134358]: pgmap v43: 97 pgs: 97 active+clean; 4.7 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 367 KiB/s wr, 99 op/s 2023-10-06T22:16:07.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:06 smithi190 ceph-mon[134891]: pgmap v44: 97 pgs: 97 active+clean; 4.7 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 367 KiB/s wr, 93 op/s 2023-10-06T22:16:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:06 smithi057 ceph-mon[98492]: pgmap v44: 97 pgs: 97 active+clean; 4.7 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 367 KiB/s wr, 93 op/s 2023-10-06T22:16:07.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:06 smithi156 ceph-mon[134358]: pgmap v44: 97 pgs: 97 active+clean; 4.7 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 367 KiB/s wr, 93 op/s 2023-10-06T22:16:08.409 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: executing depfiles commands 2023-10-06T22:16:08.736 INFO:tasks.workunit.client.0.smithi057.stderr:+ make 2023-10-06T22:16:08.750 INFO:tasks.workunit.client.0.smithi057.stdout:make all-recursive 2023-10-06T22:16:08.752 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-06T22:16:08.759 INFO:tasks.workunit.client.0.smithi057.stdout:Making all in src 2023-10-06T22:16:08.772 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-10-06T22:16:08.772 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-10-06T22:16:08.773 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-10-06T22:16:08.915 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-10-06T22:16:08.915 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-10-06T22:16:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:08 smithi190 ceph-mon[134891]: pgmap v45: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 398 KiB/s wr, 102 op/s 2023-10-06T22:16:09.083 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-10-06T22:16:09.084 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-10-06T22:16:09.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:08 smithi057 ceph-mon[98492]: pgmap v45: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 398 KiB/s wr, 102 op/s 2023-10-06T22:16:09.193 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-10-06T22:16:09.193 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-10-06T22:16:09.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:08 smithi156 ceph-mon[134358]: pgmap v45: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 398 KiB/s wr, 102 op/s 2023-10-06T22:16:09.303 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-10-06T22:16:09.303 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-10-06T22:16:09.408 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-10-06T22:16:09.408 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-10-06T22:16:09.513 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-10-06T22:16:09.514 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-10-06T22:16:09.646 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-10-06T22:16:09.646 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-10-06T22:16:09.736 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-10-06T22:16:09.736 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-10-06T22:16:09.842 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-10-06T22:16:09.843 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-10-06T22:16:09.931 INFO:tasks.workunit.client.0.smithi057.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-10-06T22:16:09.969 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-10-06T22:16:09.970 INFO:tasks.workunit.client.0.smithi057.stdout:Making all in man 2023-10-06T22:16:09.978 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-10-06T22:16:09.978 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Nothing to be done for 'all'. 2023-10-06T22:16:09.978 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-10-06T22:16:09.981 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-06T22:16:09.983 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-06T22:16:09.984 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-06T22:16:09.985 INFO:tasks.workunit.client.0.smithi057.stderr:+ cd src 2023-10-06T22:16:09.985 INFO:tasks.workunit.client.0.smithi057.stderr:+ mkdir blogtest_in 2023-10-06T22:16:09.991 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'running blogbench' 2023-10-06T22:16:09.991 INFO:tasks.workunit.client.0.smithi057.stdout:running blogbench 2023-10-06T22:16:09.992 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./blogbench -d blogtest_in 2023-10-06T22:16:09.993 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-06T22:16:09.993 INFO:tasks.workunit.client.0.smithi057.stdout:Frequency = 10 secs 2023-10-06T22:16:09.993 INFO:tasks.workunit.client.0.smithi057.stdout:Scratch dir = [blogtest_in] 2023-10-06T22:16:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/3493236353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e940ffc70 msgr2=0x7f7e94100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/3493236353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e940ffc70 0x7f7e94100050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009990 tx=0x7f7e88009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/3493236353 shutdown_connections 2023-10-06T22:16:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/3493236353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9410d820 0x7f7e9410fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/3493236353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e940ffc70 0x7f7e94100050 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/3493236353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94100620 0x7f7e9410d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/3493236353 >> 172.21.15.57:0/3493236353 conn(0x7f7e940fb860 msgr2=0x7f7e940fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/3493236353 shutdown_connections 2023-10-06T22:16:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.934+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/3493236353 wait complete. 2023-10-06T22:16:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.936+0000 7f7e9b1ff700 1 Processor -- start 2023-10-06T22:16:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.936+0000 7f7e9b1ff700 1 -- start start 2023-10-06T22:16:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e9b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e940ffc70 0x7f7e94199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e9b1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e94100620 0x7f7e94199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e9b1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9410d820 0x7f7e9419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e9b1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9419e3f0 con 0x7f7e9410d820 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e9b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9419e560 con 0x7f7e940ffc70 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e9b1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9419e720 con 0x7f7e94100620 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e98f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e940ffc70 0x7f7e94199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e98f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e940ffc70 0x7f7e94199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:44442/0 (socket says 172.21.15.57:44442) 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e98f9b700 1 -- 172.21.15.57:0/1600201438 learned_addr learned my addr 172.21.15.57:0/1600201438 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e93fff700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e94100620 0x7f7e94199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e9979c700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9410d820 0x7f7e9419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e98f9b700 1 -- 172.21.15.57:0/1600201438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e94100620 msgr2=0x7f7e94199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.937+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e94100620 0x7f7e94199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e98f9b700 1 -- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9410d820 msgr2=0x7f7e9419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9410d820 0x7f7e9419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e98f9b700 1 -- 172.21.15.57:0/1600201438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009c40 con 0x7f7e940ffc70 2023-10-06T22:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e940ffc70 0x7f7e94199200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005910 tx=0x7f7e88005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e91ffb700 1 -- 172.21.15.57:0/1600201438 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8801d070 con 0x7f7e940ffc70 2023-10-06T22:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e91ffb700 1 -- 172.21.15.57:0/1600201438 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e88022470 con 0x7f7e940ffc70 2023-10-06T22:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e880095f0 con 0x7f7e940ffc70 2023-10-06T22:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e91ffb700 1 -- 172.21.15.57:0/1600201438 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88017650 con 0x7f7e940ffc70 2023-10-06T22:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.938+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9419ec30 con 0x7f7e940ffc70 2023-10-06T22:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.942+0000 7f7e91ffb700 1 -- 172.21.15.57:0/1600201438 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 26) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f7e880225e0 con 0x7f7e940ffc70 2023-10-06T22:16:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 Processor -- start 2023-10-06T22:16:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 -- start start 2023-10-06T22:16:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7e91ffb700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7e840a1710 0x7f7e840a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f67fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f72e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f733d0 0x55a1c5f757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5ed1400 con 0x55a1c5f68520 2023-10-06T22:16:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5ec2bc0 con 0x55a1c5f733d0 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5ec04e0 con 0x55a1c5f67c00 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.943+0000 7f7e91ffb700 1 -- 172.21.15.57:0/1600201438 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7e880ce910 con 0x7f7e940ffc70 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f72e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e98f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f67fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e9979c700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7e840a1710 0x7f7e840a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f72e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60908/0 (socket says 172.21.15.57:60908) 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e98f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f67fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:55120/0 (socket says 172.21.15.57:55120) 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 -- 172.21.15.57:0/3618834681 learned_addr learned my addr 172.21.15.57:0/3618834681 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e9979c700 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f733d0 0x55a1c5f757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 -- 172.21.15.57:0/3618834681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 msgr2=0x55a1c5f67fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f67fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 -- 172.21.15.57:0/3618834681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f733d0 msgr2=0x55a1c5f757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f733d0 0x55a1c5f757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 -- 172.21.15.57:0/3618834681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1c5ce6d80 con 0x55a1c5f68520 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e9979c700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7e840a1710 0x7f7e840a3bd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e9406a6b0 tx=0x7f7e8c009450 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.944+0000 7f7e93fff700 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f72e90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009d20 tx=0x7f7e80011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b11a71adbb6df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.945+0000 7f7e7effd700 1 -- 172.21.15.57:0/3618834681 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80019630 con 0x55a1c5f68520 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.945+0000 7f7e7effd700 1 -- 172.21.15.57:0/3618834681 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e80019c50 con 0x55a1c5f68520 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.945+0000 7f7e7effd700 1 -- 172.21.15.57:0/3618834681 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000ebc0 con 0x55a1c5f68520 2023-10-06T22:16:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.945+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3618834681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 msgr2=0x55a1c5f72e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.945+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f72e90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009d20 tx=0x7f7e80011720 comp rx=0 tx=0).stop 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3618834681 shutdown_connections 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f733d0 0x55a1c5f757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f67fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3618834681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f72e90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3618834681 >> 172.21.15.57:0/3618834681 conn(0x55a1c5edd200 msgr2=0x55a1c5f64080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3618834681 shutdown_connections 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3618834681 wait complete. 2023-10-06T22:16:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 -- start start 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.946+0000 7f7eaeb71b80 1 Processor -- start 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f70090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f733d0 0x55a1c5f707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5ebbdd0 con 0x55a1c5f68520 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5e586b0 con 0x55a1c5f67c00 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5e58350 con 0x55a1c5f733d0 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7eaeb71b80 10 client.0 _ll_register_callbacks cb 0x55a1c5cb57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7e98f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7e98f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f72e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:44468/0 (socket says 172.21.15.57:44468) 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7e98f9b700 1 -- 172.21.15.57:0/3331846794 learned_addr learned my addr 172.21.15.57:0/3331846794 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7e93fff700 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f70090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7e9979c700 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f733d0 0x55a1c5f707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7e98f9b700 1 -- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f733d0 msgr2=0x55a1c5f707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.947+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f733d0 0x55a1c5f707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e98f9b700 1 -- 172.21.15.57:0/3331846794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 msgr2=0x55a1c5f70090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f70090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e98f9b700 1 -- 172.21.15.57:0/3331846794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e80009980 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f72e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006010 tx=0x7f7e8809b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e66a76b39587e8df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e73fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e73fff700 1 -- 172.21.15.57:0/3331846794 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8809b860 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e73fff700 1 -- 172.21.15.57:0/3331846794 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7e8809f9b0 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1c5f5bd50 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e73fff700 1 -- 172.21.15.57:0/3331846794 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e880a3b60 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7eaeb71b80 10 client.25988 fetch_fsmap learned FSMap version 29 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a1c5ccfc00 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.948+0000 7f7e73fff700 1 -- 172.21.15.57:0/3331846794 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e880c0020 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7e73fff700 1 -- 172.21.15.57:0/3331846794 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7e880a36d0 con 0x55a1c5f67c00 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7eaeb71b80 10 client.25988 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T22:16:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7eaeb71b80 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7eaeb71b80 20 client.25988 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5f82ae0 0x55a1c5f84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7eaeb71b80 4 client.25988 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1c5eb4db0 con 0x55a1c5f82ae0 2023-10-06T22:16:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.949+0000 7f7e93fff700 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5f82ae0 0x55a1c5f84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.950+0000 7f7e93fff700 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5f82ae0 0x55a1c5f84ed0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88bfd95933afb68a server_cookie=685dbf7ca417c2c9 in_seq=0 out_seq=0 2023-10-06T22:16:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.950+0000 7f7e73fff700 10 client.25988 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.950+0000 7f7e73fff700 1 -- 172.21.15.57:0/3331846794 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a1c5eb4db0 con 0x55a1c5f82ae0 2023-10-06T22:16:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.950+0000 7f7e73fff700 10 client.25988 handle_command_reply: tid=0 2023-10-06T22:16:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.950+0000 7f7eaeb71b80 1 client.25988 shutdown 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 msgr2=0x55a1c5f72e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f72e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006010 tx=0x7f7e8809b420 comp rx=0 tx=0).stop 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 shutdown_connections 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5f82ae0 0x55a1c5f84ed0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f733d0 0x55a1c5f707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7e73fff700 0 client.25988 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f72e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/3331846794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f68520 0x55a1c5f70090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 >> 172.21.15.57:0/3331846794 conn(0x55a1c5edd200 msgr2=0x55a1c5f63c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 shutdown_connections 2023-10-06T22:16:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/3331846794 wait complete. 2023-10-06T22:16:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.951+0000 7f7eaeb71b80 20 client.25988 trim_cache size 0 max 16384 2023-10-06T22:16:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 Processor -- start 2023-10-06T22:16:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 -- start start 2023-10-06T22:16:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f70040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f68520 0x55a1c5f79f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5f72940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5d3c8c0 con 0x55a1c5f733d0 2023-10-06T22:16:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5f64dc0 con 0x55a1c5f67c00 2023-10-06T22:16:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.966+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5f7f7b0 con 0x55a1c5f68520 2023-10-06T22:16:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5f72940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5f72940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60924/0 (socket says 172.21.15.57:60924) 2023-10-06T22:16:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 -- 172.21.15.57:0/38746056 learned_addr learned my addr 172.21.15.57:0/38746056 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f70040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e93fff700 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f68520 0x55a1c5f79f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 -- 172.21.15.57:0/38746056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f68520 msgr2=0x55a1c5f79f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f68520 0x55a1c5f79f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 -- 172.21.15.57:0/38746056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 msgr2=0x55a1c5f70040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f70040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 -- 172.21.15.57:0/38746056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1c5d0d1f0 con 0x55a1c5f733d0 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.967+0000 7f7e9979c700 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5f72940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c008870 tx=0x7f7e8c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8589fff9b7e2ff24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7e7f7fe700 1 -- 172.21.15.57:0/38746056 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00e070 con 0x55a1c5f733d0 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7e7f7fe700 1 -- 172.21.15.57:0/38746056 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8c00f040 con 0x55a1c5f733d0 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7e7f7fe700 1 -- 172.21.15.57:0/38746056 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c028df0 con 0x55a1c5f733d0 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/38746056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 msgr2=0x55a1c5f72940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5f72940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c008870 tx=0x7f7e8c01d000 comp rx=0 tx=0).stop 2023-10-06T22:16:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/38746056 shutdown_connections 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5f72940 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f67c00 0x55a1c5f70040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/38746056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f68520 0x55a1c5f79f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/38746056 >> 172.21.15.57:0/38746056 conn(0x55a1c5edd200 msgr2=0x55a1c5f6f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/38746056 shutdown_connections 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.968+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/38746056 wait complete. 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.969+0000 7f7eaeb71b80 1 -- start start 2023-10-06T22:16:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.969+0000 7f7eaeb71b80 1 Processor -- start 2023-10-06T22:16:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f6cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f68520 0x55a1c5f6d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7eaeb71b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5fd68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5f6d810 con 0x55a1c5f733d0 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5f6d980 con 0x55a1c5f68520 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7eaeb71b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5fd6ea0 con 0x55a1c5f67c00 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7eaeb71b80 10 client.0 _ll_register_callbacks cb 0x7f7e88097c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5fd68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5fd68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60932/0 (socket says 172.21.15.57:60932) 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 -- 172.21.15.57:0/702697240 learned_addr learned my addr 172.21.15.57:0/702697240 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e93fff700 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f68520 0x55a1c5f6d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f6cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 -- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 msgr2=0x55a1c5f6cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f6cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 -- 172.21.15.57:0/702697240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f68520 msgr2=0x55a1c5f6d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f68520 0x55a1c5f6d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.970+0000 7f7e9979c700 1 -- 172.21.15.57:0/702697240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8c008e70 con 0x55a1c5f733d0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7e9979c700 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5fd68e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009f70 tx=0x7f7e8c028670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f8c658556c1656 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7e73fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7e73fff700 1 -- 172.21.15.57:0/702697240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c034070 con 0x55a1c5f733d0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e80019c50 con 0x55a1c5f733d0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7e73fff700 1 -- 172.21.15.57:0/702697240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7e8c021040 con 0x55a1c5f733d0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7e73fff700 1 -- 172.21.15.57:0/702697240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e8c00e040 con 0x55a1c5f733d0 2023-10-06T22:16:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7eaeb71b80 10 client.15993 fetch_fsmap learned FSMap version 29 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a1c5ce5250 con 0x55a1c5f733d0 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.971+0000 7f7e73fff700 1 -- 172.21.15.57:0/702697240 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e80019c50 con 0x55a1c5f733d0 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7e73fff700 1 -- 172.21.15.57:0/702697240 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7e8c037030 con 0x55a1c5f733d0 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7eaeb71b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7eaeb71b80 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7eaeb71b80 20 client.15993 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5fd8530 0x55a1c5fda950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7eaeb71b80 4 client.15993 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1c5eb4ad0 con 0x55a1c5fd8530 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5fd8530 0x55a1c5fda950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7e98f9b700 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5fd8530 0x55a1c5fda950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf7c442550f8fb2 server_cookie=46c5b93b22f20b3f in_seq=0 out_seq=0 2023-10-06T22:16:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.972+0000 7f7e73fff700 10 client.15993 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7e73fff700 1 -- 172.21.15.57:0/702697240 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1c5eb4ad0 con 0x55a1c5fd8530 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7e73fff700 10 client.15993 handle_command_reply: tid=0 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 client.15993 shutdown 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 msgr2=0x55a1c5fd68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5fd68e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009f70 tx=0x7f7e8c028670 comp rx=0 tx=0).stop 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 shutdown_connections 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55a1c5fd8530 0x55a1c5fda950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1c5f733d0 0x55a1c5fd68e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1c5f67c00 0x55a1c5f6cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7e73fff700 0 client.15993 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 --2- 172.21.15.57:0/702697240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a1c5f68520 0x55a1c5f6d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 >> 172.21.15.57:0/702697240 conn(0x55a1c5edd200 msgr2=0x55a1c5f6a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 shutdown_connections 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 1 -- 172.21.15.57:0/702697240 wait complete. 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.992+0000 7f7eaeb71b80 20 client.15993 trim_cache size 0 max 16384 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:16:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:16:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7e840a1710 msgr2=0x7f7e840a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7e840a1710 0x7f7e840a3bd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e9406a6b0 tx=0x7f7e8c009450 comp rx=0 tx=0).stop 2023-10-06T22:16:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e940ffc70 msgr2=0x7f7e94199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e940ffc70 0x7f7e94199200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005910 tx=0x7f7e88005720 comp rx=0 tx=0).stop 2023-10-06T22:16:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 shutdown_connections 2023-10-06T22:16:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7e840a1710 0x7f7e840a3bd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e9410d820 0x7f7e9419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e940ffc70 0x7f7e94199200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 --2- 172.21.15.57:0/1600201438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e94100620 0x7f7e94199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 >> 172.21.15.57:0/1600201438 conn(0x7f7e940fb860 msgr2=0x7f7e9410eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 shutdown_connections 2023-10-06T22:16:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:10.993+0000 7f7e9b1ff700 1 -- 172.21.15.57:0/1600201438 wait complete. 2023-10-06T22:16:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:10 smithi190 ceph-mon[134891]: pgmap v46: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 324 KiB/s wr, 95 op/s 2023-10-06T22:16:11.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:10 smithi057 ceph-mon[98492]: pgmap v46: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 324 KiB/s wr, 95 op/s 2023-10-06T22:16:11.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:10 smithi156 ceph-mon[134358]: pgmap v46: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 324 KiB/s wr, 95 op/s 2023-10-06T22:16:12.011 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 3 writers... 2023-10-06T22:16:12.012 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 1 rewriters... 2023-10-06T22:16:12.012 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 5 commenters... 2023-10-06T22:16:12.012 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 100 readers... 2023-10-06T22:16:12.021 INFO:tasks.workunit.client.0.smithi057.stdout:Benchmarking for 30 iterations. 2023-10-06T22:16:12.022 INFO:tasks.workunit.client.0.smithi057.stdout:The test will run during 5 minutes. 2023-10-06T22:16:12.022 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-06T22:16:12.022 INFO:tasks.workunit.client.0.smithi057.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-10-06T22:16:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:12 smithi190 ceph-mon[134891]: pgmap v47: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 324 KiB/s wr, 95 op/s 2023-10-06T22:16:13.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:12 smithi057 ceph-mon[98492]: pgmap v47: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 324 KiB/s wr, 95 op/s 2023-10-06T22:16:13.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:12 smithi156 ceph-mon[134358]: pgmap v47: 97 pgs: 97 active+clean; 5.1 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 324 KiB/s wr, 95 op/s 2023-10-06T22:16:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:14 smithi190 ceph-mon[134891]: pgmap v48: 97 pgs: 97 active+clean; 6.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 499 KiB/s wr, 131 op/s 2023-10-06T22:16:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:14 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:15.177 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:14 smithi057 ceph-mon[98492]: pgmap v48: 97 pgs: 97 active+clean; 6.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 499 KiB/s wr, 131 op/s 2023-10-06T22:16:15.178 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:14 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:15.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:14 smithi156 ceph-mon[134358]: pgmap v48: 97 pgs: 97 active+clean; 6.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 499 KiB/s wr, 131 op/s 2023-10-06T22:16:15.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:14 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:17.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:16 smithi057 ceph-mon[98492]: pgmap v49: 97 pgs: 97 active+clean; 6.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 207 KiB/s wr, 44 op/s 2023-10-06T22:16:17.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:16 smithi156 ceph-mon[134358]: pgmap v49: 97 pgs: 97 active+clean; 6.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 207 KiB/s wr, 44 op/s 2023-10-06T22:16:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:16 smithi190 ceph-mon[134891]: pgmap v49: 97 pgs: 97 active+clean; 6.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 207 KiB/s wr, 44 op/s 2023-10-06T22:16:18.258 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:16:18.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.480+0000 7f0eea5ff700 1 -- 172.21.15.57:0/4229482601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 msgr2=0x7f0ee410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.480+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/4229482601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee410d930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009f90 tx=0x7f0edc00b330 comp rx=0 tx=0).stop 2023-10-06T22:16:18.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.481+0000 7f0eea5ff700 1 -- 172.21.15.57:0/4229482601 shutdown_connections 2023-10-06T22:16:18.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.481+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/4229482601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee410d930 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.481+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/4229482601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ee41020f0 0x7f0ee41024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.481+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/4229482601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ee4102a10 0x7f0ee410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.481+0000 7f0eea5ff700 1 -- 172.21.15.57:0/4229482601 >> 172.21.15.57:0/4229482601 conn(0x7f0ee40fb890 msgr2=0x7f0ee40fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:18.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.481+0000 7f0eea5ff700 1 -- 172.21.15.57:0/4229482601 shutdown_connections 2023-10-06T22:16:18.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.481+0000 7f0eea5ff700 1 -- 172.21.15.57:0/4229482601 wait complete. 2023-10-06T22:16:18.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.482+0000 7f0eea5ff700 1 Processor -- start 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.482+0000 7f0eea5ff700 1 -- start start 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.482+0000 7f0eea5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ee41020f0 0x7f0ee4197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.482+0000 7f0eea5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ee4102a10 0x7f0ee4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0eea5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0eea5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee419c030 con 0x7f0ee4102a10 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0eea5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee419c1a0 con 0x7f0ee41020f0 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0eea5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee419c360 con 0x7f0ee410b4c0 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ee4102a10 0x7f0ee4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee419b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48650/0 (socket says 172.21.15.57:48650) 2023-10-06T22:16:18.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/386710700 learned_addr learned my addr 172.21.15.57:0/386710700 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ee4102a10 0x7f0ee4197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58234/0 (socket says 172.21.15.57:58234) 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ee41020f0 0x7f0ee4197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/386710700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ee41020f0 msgr2=0x7f0ee4197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ee41020f0 0x7f0ee4197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ee4102a10 msgr2=0x7f0ee4197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ee4102a10 0x7f0ee4197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.483+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/386710700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80095f0 con 0x7f0ee410b4c0 2023-10-06T22:16:18.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.484+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee419b9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00b6c0 tx=0x7f0edc009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.484+0000 7f0ee17fa700 1 -- 172.21.15.57:0/386710700 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00e040 con 0x7f0ee410b4c0 2023-10-06T22:16:18.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.484+0000 7f0ee17fa700 1 -- 172.21.15.57:0/386710700 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc01d470 con 0x7f0ee410b4c0 2023-10-06T22:16:18.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.484+0000 7f0ee17fa700 1 -- 172.21.15.57:0/386710700 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0052a0 con 0x7f0ee410b4c0 2023-10-06T22:16:18.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.484+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc009c40 con 0x7f0ee410b4c0 2023-10-06T22:16:18.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.484+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee40ffc70 con 0x7f0ee410b4c0 2023-10-06T22:16:18.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.488+0000 7f0ee17fa700 1 -- 172.21.15.57:0/386710700 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0edc01d5e0 con 0x7f0ee410b4c0 2023-10-06T22:16:18.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.489+0000 7f0efde7bb80 1 Processor -- start 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.489+0000 7f0efde7bb80 1 -- start start 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.489+0000 7f0ee17fa700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0ecc0a1670 0x7f0ecc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf0c0 0x55b6ebebf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee17fa700 1 -- 172.21.15.57:0/386710700 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0edc0ce3d0 con 0x7f0ee410b4c0 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebebf9e0 0x55b6ebeca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebeca890 0x55b6ebeccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebe28460 con 0x55b6ebebf0c0 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebe19c20 con 0x55b6ebeca890 2023-10-06T22:16:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebe17540 con 0x55b6ebebf9e0 2023-10-06T22:16:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee37fe700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0ecc0a1670 0x7f0ecc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf0c0 0x55b6ebebf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebebf9e0 0x55b6ebeca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee8b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebeca890 0x55b6ebeccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebebf9e0 0x55b6ebeca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48658/0 (socket says 172.21.15.57:48658) 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee37fe700 1 -- 172.21.15.57:0/703503638 learned_addr learned my addr 172.21.15.57:0/703503638 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf0c0 0x55b6ebebf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58250/0 (socket says 172.21.15.57:58250) 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/703503638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebebf9e0 msgr2=0x55b6ebeca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebebf9e0 0x55b6ebeca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/703503638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf0c0 msgr2=0x55b6ebebf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf0c0 0x55b6ebebf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/703503638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6ebc3dde0 con 0x55b6ebeca890 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.490+0000 7f0ee37fe700 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebebf9e0 0x55b6ebeca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.491+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebeca890 0x55b6ebeccc80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0cbb30 tx=0x7f0edc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=814ca78a19e65c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.491+0000 7f0ee37fe700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0ecc0a1670 0x7f0ecc0a3b30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009fd0 tx=0x7f0ed80047c0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.491+0000 7f0eca7fc700 1 -- 172.21.15.57:0/703503638 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc09adb0 con 0x55b6ebeca890 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.491+0000 7f0eca7fc700 1 -- 172.21.15.57:0/703503638 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc09bb50 con 0x55b6ebeca890 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.491+0000 7f0eca7fc700 1 -- 172.21.15.57:0/703503638 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc09b6b0 con 0x55b6ebeca890 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 -- 172.21.15.57:0/703503638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebeca890 msgr2=0x55b6ebeccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebeca890 0x55b6ebeccc80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0cbb30 tx=0x7f0edc009920 comp rx=0 tx=0).stop 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 -- 172.21.15.57:0/703503638 shutdown_connections 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebeca890 0x55b6ebeccc80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf0c0 0x55b6ebebf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/703503638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebebf9e0 0x55b6ebeca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 -- 172.21.15.57:0/703503638 >> 172.21.15.57:0/703503638 conn(0x55b6ebe34260 msgr2=0x55b6ebebb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 -- 172.21.15.57:0/703503638 shutdown_connections 2023-10-06T22:16:18.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 -- 172.21.15.57:0/703503638 wait complete. 2023-10-06T22:16:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.492+0000 7f0efde7bb80 1 -- start start 2023-10-06T22:16:18.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.493+0000 7f0efde7bb80 1 Processor -- start 2023-10-06T22:16:18.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.493+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf0c0 0x55b6ebeca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.493+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.493+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebec7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.493+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebe12e30 con 0x55b6ebebf9e0 2023-10-06T22:16:18.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.493+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebdaf710 con 0x55b6ebebf0c0 2023-10-06T22:16:18.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.493+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebdaf3b0 con 0x55b6ebeca890 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0efde7bb80 10 client.0 _ll_register_callbacks cb 0x55b6ebc0c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf0c0 0x55b6ebeca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf0c0 0x55b6ebeca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55094/0 (socket says 172.21.15.57:55094) 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 -- 172.21.15.57:0/1419779350 learned_addr learned my addr 172.21.15.57:0/1419779350 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebec7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee37fe700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 -- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 msgr2=0x55b6ebec7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebec7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 -- 172.21.15.57:0/1419779350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 msgr2=0x55b6ebec7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 -- 172.21.15.57:0/1419779350 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed8010980 con 0x55b6ebebf0c0 2023-10-06T22:16:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee37fe700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf0c0 0x55b6ebeca2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4012860 tx=0x7f0ed4012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ca1bd0eea95afc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ec37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.494+0000 7f0ec37fe700 1 -- 172.21.15.57:0/1419779350 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4017070 con 0x55b6ebebf0c0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6ebeb2f10 con 0x55b6ebebf0c0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0ec37fe700 1 -- 172.21.15.57:0/1419779350 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0ed400f5e0 con 0x55b6ebebf0c0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0ec37fe700 1 -- 172.21.15.57:0/1419779350 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ed40122e0 con 0x55b6ebebf0c0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 10 client.26006 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f0ee40ffc70 con 0x55b6ebebf0c0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0ec37fe700 1 -- 172.21.15.57:0/1419779350 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6ebeb2f10 con 0x55b6ebebf0c0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0ec37fe700 1 -- 172.21.15.57:0/1419779350 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ed4010930 con 0x55b6ebebf0c0 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 10 client.26006 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 20 client.26006 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebeda230 0x55b6ebedc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 4 client.26006 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.495+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6ebe0be10 con 0x55b6ebeda230 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.496+0000 7f0ee37fe700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebeda230 0x55b6ebedc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.496+0000 7f0ee37fe700 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebeda230 0x55b6ebedc650 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3ff295399f0dbb server_cookie=9623d9debedc7d03 in_seq=0 out_seq=0 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.496+0000 7f0ec37fe700 10 client.26006 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0ec37fe700 1 -- 172.21.15.57:0/1419779350 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6ebe0be10 con 0x55b6ebeda230 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0ec37fe700 10 client.26006 handle_command_reply: tid=0 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 client.26006 shutdown 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf0c0 msgr2=0x55b6ebeca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf0c0 0x55b6ebeca2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4012860 tx=0x7f0ed4012b70 comp rx=0 tx=0).stop 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 shutdown_connections 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebeda230 0x55b6ebedc650 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebec7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf0c0 0x55b6ebeca2e0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4012860 tx=0x7f0ed4012b70 comp rx=0 tx=0).stop 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/1419779350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 >> 172.21.15.57:0/1419779350 conn(0x55b6ebe34260 msgr2=0x55b6ebebafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0ec37fe700 0 client.26006 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 shutdown_connections 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.497+0000 7f0efde7bb80 1 -- 172.21.15.57:0/1419779350 wait complete. 2023-10-06T22:16:18.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.498+0000 7f0efde7bb80 20 client.26006 trim_cache size 0 max 16384 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.517+0000 7f0efde7bb80 1 Processor -- start 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.517+0000 7f0efde7bb80 1 -- start start 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebed6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebec6e80 0x55b6ebec9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebc93920 con 0x55b6ebec6e80 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebeb5520 con 0x55b6ebebf9e0 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebebcd40 con 0x55b6ebeca890 2023-10-06T22:16:18.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebed6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55110/0 (socket says 172.21.15.57:55110) 2023-10-06T22:16:18.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 -- 172.21.15.57:0/3946520637 learned_addr learned my addr 172.21.15.57:0/3946520637 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:18.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebec6e80 0x55b6ebec9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 -- 172.21.15.57:0/3946520637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 msgr2=0x55b6ebed6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebed6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 -- 172.21.15.57:0/3946520637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebec6e80 msgr2=0x55b6ebec9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebec6e80 0x55b6ebec9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee3fff700 1 -- 172.21.15.57:0/3946520637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6ebc26c60 con 0x55b6ebebf9e0 2023-10-06T22:16:18.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.518+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebec6e80 0x55b6ebec9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:18.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.519+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec1f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4012830 tx=0x7f0ed40122e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671a2b5131bcaa6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.519+0000 7f0ecaffd700 1 -- 172.21.15.57:0/3946520637 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4017070 con 0x55b6ebebf9e0 2023-10-06T22:16:18.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.519+0000 7f0ecaffd700 1 -- 172.21.15.57:0/3946520637 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed4018040 con 0x55b6ebebf9e0 2023-10-06T22:16:18.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.519+0000 7f0ecaffd700 1 -- 172.21.15.57:0/3946520637 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40056e0 con 0x55b6ebebf9e0 2023-10-06T22:16:18.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.519+0000 7f0efde7bb80 1 -- 172.21.15.57:0/3946520637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf9e0 msgr2=0x55b6ebec1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.519+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec1f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4012830 tx=0x7f0ed40122e0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 -- 172.21.15.57:0/3946520637 shutdown_connections 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebec6e80 0x55b6ebec9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebebf9e0 0x55b6ebec1f60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/3946520637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebed6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 -- 172.21.15.57:0/3946520637 >> 172.21.15.57:0/3946520637 conn(0x55b6ebe34260 msgr2=0x55b6ebeb9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 -- 172.21.15.57:0/3946520637 shutdown_connections 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 -- 172.21.15.57:0/3946520637 wait complete. 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 -- start start 2023-10-06T22:16:18.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.520+0000 7f0efde7bb80 1 Processor -- start 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebed1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebec6e80 0x55b6ebecf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0efde7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebecf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebecffb0 con 0x55b6ebebf9e0 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebed0120 con 0x55b6ebec6e80 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0efde7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ebed02e0 con 0x55b6ebeca890 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0efde7bb80 10 client.0 _ll_register_callbacks cb 0x7f0ed4009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0ee8b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebecf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0ee8b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebecf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48682/0 (socket says 172.21.15.57:48682) 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/816146043 learned_addr learned my addr 172.21.15.57:0/816146043 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:18.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.521+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebed1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/816146043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebec6e80 msgr2=0x55b6ebecf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebec6e80 0x55b6ebecf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/816146043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 msgr2=0x55b6ebed1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebed1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ee8b9c700 1 -- 172.21.15.57:0/816146043 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee40ffc70 con 0x55b6ebeca890 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ee8b9c700 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebecf9e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0edc097950 tx=0x7f0edc09b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f63eab723fdcf552 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ec37fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ec37fe700 1 -- 172.21.15.57:0/816146043 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc09f5a0 con 0x55b6ebeca890 2023-10-06T22:16:18.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ec37fe700 1 -- 172.21.15.57:0/816146043 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0edc0a3750 con 0x55b6ebeca890 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0edc09bb50 con 0x55b6ebeca890 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.522+0000 7f0ec37fe700 1 -- 172.21.15.57:0/816146043 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0edc09b450 con 0x55b6ebeca890 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 10 client.26011 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b6ebc64250 con 0x55b6ebeca890 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0ec37fe700 1 -- 172.21.15.57:0/816146043 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc080020 con 0x55b6ebeca890 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0ec37fe700 1 -- 172.21.15.57:0/816146043 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0edc09b7b0 con 0x55b6ebeca890 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 10 client.26011 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 20 client.26011 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebf2f2e0 0x55b6ebf31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 4 client.26011 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:16:18.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b6ebe0bb30 con 0x55b6ebf2f2e0 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.523+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebf2f2e0 0x55b6ebf31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0ee3fff700 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebf2f2e0 0x55b6ebf31700 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802c23bc230dd75c server_cookie=7d23b465f1a42017 in_seq=0 out_seq=0 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0ec37fe700 10 client.26011 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0ec37fe700 1 -- 172.21.15.57:0/816146043 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b6ebe0bb30 con 0x55b6ebf2f2e0 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0ec37fe700 10 client.26011 handle_command_reply: tid=0 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 client.26011 shutdown 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 msgr2=0x55b6ebecf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebecf9e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0edc097950 tx=0x7f0edc09b350 comp rx=0 tx=0).stop 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 shutdown_connections 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b6ebf2f2e0 0x55b6ebf31700 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ebeca890 0x55b6ebecf9e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6ebebf9e0 0x55b6ebed1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 --2- 172.21.15.57:0/816146043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6ebec6e80 0x55b6ebecf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0ec37fe700 0 client.26011 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 >> 172.21.15.57:0/816146043 conn(0x55b6ebe34260 msgr2=0x55b6ebec28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 shutdown_connections 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.524+0000 7f0efde7bb80 1 -- 172.21.15.57:0/816146043 wait complete. 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0efde7bb80 20 client.26011 trim_cache size 0 max 16384 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:16:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-06T22:16:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-06T22:16:18.536 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:16:18.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0ecc0a1670 msgr2=0x7f0ecc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0ecc0a1670 0x7f0ecc0a3b30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009fd0 tx=0x7f0ed80047c0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 msgr2=0x7f0ee419b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee419b9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00b6c0 tx=0x7f0edc009510 comp rx=0 tx=0).stop 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 shutdown_connections 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0ecc0a1670 0x7f0ecc0a3b30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ee410b4c0 0x7f0ee419b9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ee41020f0 0x7f0ee4197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 --2- 172.21.15.57:0/386710700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ee4102a10 0x7f0ee4197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 >> 172.21.15.57:0/386710700 conn(0x7f0ee40fb890 msgr2=0x7f0ee4102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 shutdown_connections 2023-10-06T22:16:18.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.525+0000 7f0eea5ff700 1 -- 172.21.15.57:0/386710700 wait complete. 2023-10-06T22:16:18.546 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T22:16:18.546 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:16:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.769+0000 7f350b59e700 1 -- 172.21.15.57:0/863865363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c10d550 msgr2=0x7f350c10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.769+0000 7f350b59e700 1 --2- 172.21.15.57:0/863865363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c10d550 0x7f350c10f980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f350400b0a0 tx=0x7f350400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.770+0000 7f350b59e700 1 -- 172.21.15.57:0/863865363 shutdown_connections 2023-10-06T22:16:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.770+0000 7f350b59e700 1 --2- 172.21.15.57:0/863865363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c10d550 0x7f350c10f980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.770+0000 7f350b59e700 1 --2- 172.21.15.57:0/863865363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f350c073280 0x7f350c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.770+0000 7f350b59e700 1 --2- 172.21.15.57:0/863865363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f350c073ba0 0x7f350c10d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.770+0000 7f350b59e700 1 -- 172.21.15.57:0/863865363 >> 172.21.15.57:0/863865363 conn(0x7f350c0fc7a0 msgr2=0x7f350c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:18.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.770+0000 7f350b59e700 1 -- 172.21.15.57:0/863865363 shutdown_connections 2023-10-06T22:16:18.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.770+0000 7f350b59e700 1 -- 172.21.15.57:0/863865363 wait complete. 2023-10-06T22:16:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 Processor -- start 2023-10-06T22:16:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 -- start start 2023-10-06T22:16:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c073280 0x7f350c198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f350c073ba0 0x7f350c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f350c10d550 0x7f350c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19df70 con 0x7f350c073ba0 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19e0e0 con 0x7f350c073280 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350b59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19e2a0 con 0x7f350c10d550 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.771+0000 7f350a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c073280 0x7f350c198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c073280 0x7f350c198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55142/0 (socket says 172.21.15.57:55142) 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 -- 172.21.15.57:0/782263494 learned_addr learned my addr 172.21.15.57:0/782263494 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f3509d9b700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f350c073ba0 0x7f350c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350ad9d700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f350c10d550 0x7f350c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 -- 172.21.15.57:0/782263494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f350c10d550 msgr2=0x7f350c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f350c10d550 0x7f350c19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 -- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f350c073ba0 msgr2=0x7f350c1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f350c073ba0 0x7f350c1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:18.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 -- 172.21.15.57:0/782263494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc009520 con 0x7f350c073280 2023-10-06T22:16:18.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f350a59c700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c073280 0x7f350c198f90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00eb30 tx=0x7f34fc00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.772+0000 7f34fb7fe700 1 -- 172.21.15.57:0/782263494 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc009880 con 0x7f350c073280 2023-10-06T22:16:18.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.773+0000 7f34fb7fe700 1 -- 172.21.15.57:0/782263494 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc0044e0 con 0x7f350c073280 2023-10-06T22:16:18.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.773+0000 7f34fb7fe700 1 -- 172.21.15.57:0/782263494 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc003e90 con 0x7f350c073280 2023-10-06T22:16:18.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.773+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3504009d00 con 0x7f350c073280 2023-10-06T22:16:18.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.773+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c1a5be0 con 0x7f350c073280 2023-10-06T22:16:18.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.774+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c107e70 con 0x7f350c073280 2023-10-06T22:16:18.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.777+0000 7f34fb7fe700 1 -- 172.21.15.57:0/782263494 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f34fc004000 con 0x7f350c073280 2023-10-06T22:16:18.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.778+0000 7f34fb7fe700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f34f40a1760 0x7f34f40a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:18.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.778+0000 7f34fb7fe700 1 -- 172.21.15.57:0/782263494 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f34fc014070 con 0x7f350c073280 2023-10-06T22:16:18.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.778+0000 7f3509d9b700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f34f40a1760 0x7f34f40a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:18.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.779+0000 7f3509d9b700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f34f40a1760 0x7f34f40a3c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f350c19a5a0 tx=0x7f3500009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:18.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:18.780+0000 7f34fb7fe700 1 -- 172.21.15.57:0/782263494 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f34fc0969b0 con 0x7f350c073280 2023-10-06T22:16:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:18 smithi057 ceph-mon[98492]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1014 KiB/s wr, 57 op/s 2023-10-06T22:16:18.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:18 smithi057 ceph-mon[98492]: mgrmap e27: x(active, since 92s), standbys: y, z 2023-10-06T22:16:19.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.237+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f350c19a390 con 0x7f350c073280 2023-10-06T22:16:19.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.238+0000 7f34fb7fe700 1 -- 172.21.15.57:0/782263494 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11371 (secure 0 0 0) 0x7f34fc099fd0 con 0x7f350c073280 2023-10-06T22:16:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:16:19.240 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:16:13.362401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25784":{"gid":25784,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/1662104434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1662104434},{"type":"v1","addr":"172.21.15.156:6835","nonce":1662104434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f34f40a1760 msgr2=0x7f34f40a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f34f40a1760 0x7f34f40a3c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f350c19a5a0 tx=0x7f3500009260 comp rx=0 tx=0).stop 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c073280 msgr2=0x7f350c198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c073280 0x7f350c198f90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00eb30 tx=0x7f34fc00ee40 comp rx=0 tx=0).stop 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 shutdown_connections 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f34f40a1760 0x7f34f40a3c20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f350c10d550 0x7f350c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f350c073280 0x7f350c198f90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 --2- 172.21.15.57:0/782263494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f350c073ba0 0x7f350c1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 >> 172.21.15.57:0/782263494 conn(0x7f350c0fc7a0 msgr2=0x7f350c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 shutdown_connections 2023-10-06T22:16:19.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.240+0000 7f350b59e700 1 -- 172.21.15.57:0/782263494 wait complete. 2023-10-06T22:16:19.244 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 30 2023-10-06T22:16:19.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:18 smithi156 ceph-mon[134358]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1014 KiB/s wr, 57 op/s 2023-10-06T22:16:19.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:18 smithi156 ceph-mon[134358]: mgrmap e27: x(active, since 92s), standbys: y, z 2023-10-06T22:16:19.262 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T22:16:19.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:18 smithi190 ceph-mon[134891]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1014 KiB/s wr, 57 op/s 2023-10-06T22:16:19.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:18 smithi190 ceph-mon[134891]: mgrmap e27: x(active, since 92s), standbys: y, z 2023-10-06T22:16:19.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.477+0000 7f2d6ddff700 1 -- 172.21.15.57:0/3339694244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d681061d0 msgr2=0x7f2d6810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.477+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/3339694244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d681061d0 0x7f2d6810dd50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b0a0 tx=0x7f2d6000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.478+0000 7f2d6ddff700 1 -- 172.21.15.57:0/3339694244 shutdown_connections 2023-10-06T22:16:19.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.478+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/3339694244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d681061d0 0x7f2d6810dd50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.478+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/3339694244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 0x7f2d68101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.478+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/3339694244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d68101a60 0x7f2d68105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.478+0000 7f2d6ddff700 1 -- 172.21.15.57:0/3339694244 >> 172.21.15.57:0/3339694244 conn(0x7f2d680fc930 msgr2=0x7f2d680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.478+0000 7f2d6ddff700 1 -- 172.21.15.57:0/3339694244 shutdown_connections 2023-10-06T22:16:19.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.478+0000 7f2d6ddff700 1 -- 172.21.15.57:0/3339694244 wait complete. 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 Processor -- start 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 -- start start 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 0x7f2d68199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d681061d0 0x7f2d681995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d6819da00 0x7f2d6819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819e480 con 0x7f2d681010b0 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819e5f0 con 0x7f2d6819da00 2023-10-06T22:16:19.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.479+0000 7f2d6ddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819e7b0 con 0x7f2d681061d0 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 0x7f2d68199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 0x7f2d68199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58302/0 (socket says 172.21.15.57:58302) 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 -- 172.21.15.57:0/4158469129 learned_addr learned my addr 172.21.15.57:0/4158469129 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d67fff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d6819da00 0x7f2d6819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d681061d0 0x7f2d681995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 -- 172.21.15.57:0/4158469129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d681061d0 msgr2=0x7f2d681995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d681061d0 0x7f2d681995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 -- 172.21.15.57:0/4158469129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d6819da00 msgr2=0x7f2d6819de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d6819da00 0x7f2d6819de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 -- 172.21.15.57:0/4158469129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60009d00 con 0x7f2d681010b0 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.480+0000 7f2d677fe700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 0x7f2d68199060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9a0 tx=0x7f2d5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.481+0000 7f2d64ff9700 1 -- 172.21.15.57:0/4158469129 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58009790 con 0x7f2d681010b0 2023-10-06T22:16:19.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.481+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d681a5d10 con 0x7f2d681010b0 2023-10-06T22:16:19.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.481+0000 7f2d64ff9700 1 -- 172.21.15.57:0/4158469129 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d58010430 con 0x7f2d681010b0 2023-10-06T22:16:19.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.481+0000 7f2d64ff9700 1 -- 172.21.15.57:0/4158469129 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58018680 con 0x7f2d681010b0 2023-10-06T22:16:19.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.481+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d681a61e0 con 0x7f2d681010b0 2023-10-06T22:16:19.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.485+0000 7f2d64ff9700 1 -- 172.21.15.57:0/4158469129 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2d580105a0 con 0x7f2d681010b0 2023-10-06T22:16:19.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.486+0000 7f2d81788b80 1 Processor -- start 2023-10-06T22:16:19.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.486+0000 7f2d81788b80 1 -- start start 2023-10-06T22:16:19.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d64ff9700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2d500a15f0 0x7f2d500a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d64ff9700 1 -- 172.21.15.57:0/4158469129 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2d58014070 con 0x7f2d681010b0 2023-10-06T22:16:19.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95518d0 0x556ab9551cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab955f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d81788b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab94b2460 con 0x556ab95521f0 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d81788b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab94a3c20 con 0x556ab95518d0 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d81788b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab94a1540 con 0x556ab955d0a0 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d67fff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2d500a15f0 0x7f2d500a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95518d0 0x556ab9551cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95518d0 0x556ab9551cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55178/0 (socket says 172.21.15.57:55178) 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d677fe700 1 -- 172.21.15.57:0/2156737263 learned_addr learned my addr 172.21.15.57:0/2156737263 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d67fff700 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab955f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d677fe700 1 -- 172.21.15.57:0/2156737263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 msgr2=0x556ab955f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.487+0000 7f2d677fe700 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab955f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d677fe700 1 -- 172.21.15.57:0/2156737263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 msgr2=0x556ab955cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d677fe700 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d677fe700 1 -- 172.21.15.57:0/2156737263 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ab92c7de0 con 0x556ab95518d0 2023-10-06T22:16:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d67fff700 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab955f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d677fe700 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95518d0 0x556ab9551cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d58096cc0 tx=0x7f2d5809a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6eea9ac01d2d88d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d67fff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2d500a15f0 0x7f2d500a3ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009cd0 tx=0x7f2d60009c60 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d4dffb700 1 -- 172.21.15.57:0/2156737263 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5809e3a0 con 0x556ab95518d0 2023-10-06T22:16:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d4dffb700 1 -- 172.21.15.57:0/2156737263 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d580a25e0 con 0x556ab95518d0 2023-10-06T22:16:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.488+0000 7f2d4dffb700 1 -- 172.21.15.57:0/2156737263 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58099740 con 0x556ab95518d0 2023-10-06T22:16:19.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.489+0000 7f2d81788b80 1 -- 172.21.15.57:0/2156737263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95518d0 msgr2=0x556ab9551cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.489+0000 7f2d81788b80 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95518d0 0x556ab9551cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d58096cc0 tx=0x7f2d5809a950 comp rx=0 tx=0).stop 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.489+0000 7f2d81788b80 1 -- 172.21.15.57:0/2156737263 shutdown_connections 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.489+0000 7f2d81788b80 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab955f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.489+0000 7f2d81788b80 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95518d0 0x556ab9551cb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.489+0000 7f2d81788b80 1 --2- 172.21.15.57:0/2156737263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.489+0000 7f2d81788b80 1 -- 172.21.15.57:0/2156737263 >> 172.21.15.57:0/2156737263 conn(0x556ab94be260 msgr2=0x556ab9545450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.490+0000 7f2d81788b80 1 -- 172.21.15.57:0/2156737263 shutdown_connections 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.490+0000 7f2d81788b80 1 -- 172.21.15.57:0/2156737263 wait complete. 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.490+0000 7f2d81788b80 1 -- start start 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 1 Processor -- start 2023-10-06T22:16:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9559df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab955a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab949ce30 con 0x556ab95521f0 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab9439710 con 0x556ab955d0a0 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab94393b0 con 0x556ab95518d0 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.491+0000 7f2d81788b80 10 client.0 _ll_register_callbacks cb 0x556ab9296800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9559df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9559df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48738/0 (socket says 172.21.15.57:48738) 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 -- 172.21.15.57:0/4269065371 learned_addr learned my addr 172.21.15.57:0/4269065371 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d67fff700 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab955a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 -- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 msgr2=0x556ab955a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab955a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 -- 172.21.15.57:0/4269065371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 msgr2=0x556ab955a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 -- 172.21.15.57:0/4269065371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d681a61e0 con 0x556ab95518d0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d677fe700 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9559df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d5801e000 tx=0x7f2d58096590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02c139c7ac982a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d46ffd700 1 -- 172.21.15.57:0/4269065371 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5809e420 con 0x556ab95518d0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ab953cf10 con 0x556ab95518d0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.492+0000 7f2d46ffd700 1 -- 172.21.15.57:0/4269065371 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2d5809a4b0 con 0x556ab95518d0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d46ffd700 1 -- 172.21.15.57:0/4269065371 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d5809a670 con 0x556ab95518d0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 10 client.26029 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556ab92b0c60 con 0x556ab95518d0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d46ffd700 1 -- 172.21.15.57:0/4269065371 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ab953cf10 con 0x556ab95518d0 2023-10-06T22:16:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d46ffd700 1 -- 172.21.15.57:0/4269065371 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d580b4d20 con 0x556ab95518d0 2023-10-06T22:16:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 10 client.26029 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 10 client.26029 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:16:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 20 client.26029 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab956c690 0x556ab956ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 4 client.26029 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.493+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ab9495e10 con 0x556ab956c690 2023-10-06T22:16:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.494+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab956c690 0x556ab956ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.495+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab956c690 0x556ab956ea80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7050701a209f70f7 server_cookie=7805dfc03333f281 in_seq=0 out_seq=0 2023-10-06T22:16:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.495+0000 7f2d46ffd700 10 client.26029 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.495+0000 7f2d46ffd700 1 -- 172.21.15.57:0/4269065371 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ab9495e10 con 0x556ab956c690 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.495+0000 7f2d46ffd700 10 client.26029 handle_command_reply: tid=0 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.495+0000 7f2d81788b80 1 client.26029 shutdown 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 msgr2=0x556ab9559df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9559df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d5801e000 tx=0x7f2d58096590 comp rx=0 tx=0).stop 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 shutdown_connections 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab956c690 0x556ab956ea80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab955a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9559df0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 --2- 172.21.15.57:0/4269065371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab955a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 >> 172.21.15.57:0/4269065371 conn(0x556ab94be260 msgr2=0x556ab9544fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d46ffd700 0 client.26029 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 shutdown_connections 2023-10-06T22:16:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 1 -- 172.21.15.57:0/4269065371 wait complete. 2023-10-06T22:16:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.496+0000 7f2d81788b80 20 client.26029 trim_cache size 0 max 16384 2023-10-06T22:16:19.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.510+0000 7f2d81788b80 1 Processor -- start 2023-10-06T22:16:19.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.510+0000 7f2d81788b80 1 -- start start 2023-10-06T22:16:19.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.510+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95518d0 0x556ab955a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.510+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95521f0 0x556ab9553fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.510+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab9557060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.511+0000 7f2d81788b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab931d920 con 0x556ab95518d0 2023-10-06T22:16:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.511+0000 7f2d81788b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab9555c40 con 0x556ab95521f0 2023-10-06T22:16:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.511+0000 7f2d81788b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab9555990 con 0x556ab955d0a0 2023-10-06T22:16:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.511+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95518d0 0x556ab955a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.511+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95518d0 0x556ab955a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58318/0 (socket says 172.21.15.57:58318) 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.511+0000 7f2d677fe700 1 -- 172.21.15.57:0/3848364201 learned_addr learned my addr 172.21.15.57:0/3848364201 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.511+0000 7f2d67fff700 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab9557060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95521f0 0x556ab9553fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d677fe700 1 -- 172.21.15.57:0/3848364201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 msgr2=0x556ab9557060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d677fe700 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab9557060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d677fe700 1 -- 172.21.15.57:0/3848364201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95521f0 msgr2=0x556ab9553fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d677fe700 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95521f0 0x556ab9553fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d677fe700 1 -- 172.21.15.57:0/3848364201 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ab92ee250 con 0x556ab95518d0 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d677fe700 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95518d0 0x556ab955a6a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2d58099b40 tx=0x7f2d58098800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997778b47f4faab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d4e7fc700 1 -- 172.21.15.57:0/3848364201 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580991b0 con 0x556ab95518d0 2023-10-06T22:16:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d4e7fc700 1 -- 172.21.15.57:0/3848364201 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d5809e3a0 con 0x556ab95518d0 2023-10-06T22:16:19.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.512+0000 7f2d4e7fc700 1 -- 172.21.15.57:0/3848364201 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5809a710 con 0x556ab95518d0 2023-10-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 -- 172.21.15.57:0/3848364201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95518d0 msgr2=0x556ab955a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95518d0 0x556ab955a6a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2d58099b40 tx=0x7f2d58098800 comp rx=0 tx=0).stop 2023-10-06T22:16:19.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 -- 172.21.15.57:0/3848364201 shutdown_connections 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab955d0a0 0x556ab9557060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95518d0 0x556ab955a6a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 --2- 172.21.15.57:0/3848364201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab95521f0 0x556ab9553fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 -- 172.21.15.57:0/3848364201 >> 172.21.15.57:0/3848364201 conn(0x556ab94be260 msgr2=0x556ab95538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 -- 172.21.15.57:0/3848364201 shutdown_connections 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 -- 172.21.15.57:0/3848364201 wait complete. 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.513+0000 7f2d81788b80 1 -- start start 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 1 Processor -- start 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9561490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab95619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab95bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab95621c0 con 0x556ab95521f0 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab9562330 con 0x556ab955d0a0 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab95bfa40 con 0x556ab95518d0 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.514+0000 7f2d81788b80 10 client.0 _ll_register_callbacks cb 0x7f2d5c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d66ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab95619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab95bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d677fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9561490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d66ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab95619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58326/0 (socket says 172.21.15.57:58326) 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d66ffd700 1 -- 172.21.15.57:0/415085507 learned_addr learned my addr 172.21.15.57:0/415085507 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d677fe700 1 -- 172.21.15.57:0/415085507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 msgr2=0x556ab95bf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d677fe700 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab95bf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d677fe700 1 -- 172.21.15.57:0/415085507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 msgr2=0x556ab95619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d677fe700 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab95619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d677fe700 1 -- 172.21.15.57:0/415085507 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ab92b0c60 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d677fe700 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9561490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d580985a0 tx=0x7f2d58010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b80d59cc08f2f60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d46ffd700 1 -- 172.21.15.57:0/415085507 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580991b0 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d46ffd700 1 -- 172.21.15.57:0/415085507 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2d58083570 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.515+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d580a25e0 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d46ffd700 1 -- 172.21.15.57:0/415085507 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d5809f160 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 10 client.26035 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556ab92c62b0 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d46ffd700 1 -- 172.21.15.57:0/415085507 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d580a25e0 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d46ffd700 1 -- 172.21.15.57:0/415085507 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d5800ca10 con 0x556ab95518d0 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 10 client.26035 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:19.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 10 client.26035 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 20 client.26035 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab95c0d10 0x556ab95c3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 4 client.26035 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ab9495b30 con 0x556ab95c0d10 2023-10-06T22:16:19.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.516+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab95c0d10 0x556ab95c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d66ffd700 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab95c0d10 0x556ab95c3100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=159277872f813b73 server_cookie=9e675292292143e0 in_seq=0 out_seq=0 2023-10-06T22:16:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d46ffd700 10 client.26035 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d46ffd700 1 -- 172.21.15.57:0/415085507 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ab9495b30 con 0x556ab95c0d10 2023-10-06T22:16:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d46ffd700 10 client.26035 handle_command_reply: tid=0 2023-10-06T22:16:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 client.26035 shutdown 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 msgr2=0x556ab9561490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9561490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d580985a0 tx=0x7f2d58010a90 comp rx=0 tx=0).stop 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 shutdown_connections 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x556ab95c0d10 0x556ab95c3100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab955d0a0 0x556ab95bf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556ab95518d0 0x556ab9561490 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 --2- 172.21.15.57:0/415085507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ab95521f0 0x556ab95619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 >> 172.21.15.57:0/415085507 conn(0x556ab94be260 msgr2=0x556ab9554160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d46ffd700 0 client.26035 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.517+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 shutdown_connections 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d81788b80 1 -- 172.21.15.57:0/415085507 wait complete. 2023-10-06T22:16:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d81788b80 20 client.26035 trim_cache size 0 max 16384 2023-10-06T22:16:19.525 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2d500a15f0 msgr2=0x7f2d500a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2d500a15f0 0x7f2d500a3ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009cd0 tx=0x7f2d60009c60 comp rx=0 tx=0).stop 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 msgr2=0x7f2d68199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 0x7f2d68199060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9a0 tx=0x7f2d5800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 shutdown_connections 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2d500a15f0 0x7f2d500a3ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d6819da00 0x7f2d6819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d681010b0 0x7f2d68199060 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 --2- 172.21.15.57:0/4158469129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d681061d0 0x7f2d681995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 >> 172.21.15.57:0/4158469129 conn(0x7f2d680fc930 msgr2=0x7f2d680fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.518+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 shutdown_connections 2023-10-06T22:16:19.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.519+0000 7f2d6ddff700 1 -- 172.21.15.57:0/4158469129 wait complete. 2023-10-06T22:16:19.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 -- 172.21.15.57:0/3824762954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81010b0 msgr2=0x7f63e8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 --2- 172.21.15.57:0/3824762954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81010b0 0x7f63e8101490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009990 tx=0x7f63d8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 -- 172.21.15.57:0/3824762954 shutdown_connections 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 --2- 172.21.15.57:0/3824762954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81061d0 0x7f63e810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 --2- 172.21.15.57:0/3824762954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81010b0 0x7f63e8101490 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 --2- 172.21.15.57:0/3824762954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e8101a60 0x7f63e8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 -- 172.21.15.57:0/3824762954 >> 172.21.15.57:0/3824762954 conn(0x7f63e80fc950 msgr2=0x7f63e80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 -- 172.21.15.57:0/3824762954 shutdown_connections 2023-10-06T22:16:19.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.766+0000 7f63edfff700 1 -- 172.21.15.57:0/3824762954 wait complete. 2023-10-06T22:16:19.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.767+0000 7f63edfff700 1 Processor -- start 2023-10-06T22:16:19.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63edfff700 1 -- start start 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63edfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81010b0 0x7f63e80731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63edfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81061d0 0x7f63e80736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63edfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e8073c30 0x7f63e81a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63edfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8075100 con 0x7f63e81061d0 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63edfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e81a9ed0 con 0x7f63e81010b0 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63edfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e81aa010 con 0x7f63e8073c30 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81010b0 0x7f63e80731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63e6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81061d0 0x7f63e80736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81010b0 0x7f63e80731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55218/0 (socket says 172.21.15.57:55218) 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63e77fe700 1 -- 172.21.15.57:0/283370078 learned_addr learned my addr 172.21.15.57:0/283370078 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.768+0000 7f63e7fff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e8073c30 0x7f63e81a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e77fe700 1 -- 172.21.15.57:0/283370078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e8073c30 msgr2=0x7f63e81a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e77fe700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e8073c30 0x7f63e81a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e77fe700 1 -- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81061d0 msgr2=0x7f63e80736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e77fe700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81061d0 0x7f63e80736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e77fe700 1 -- 172.21.15.57:0/283370078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0009c40 con 0x7f63e81010b0 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e7fff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e8073c30 0x7f63e81a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e6ffd700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81061d0 0x7f63e80736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e77fe700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81010b0 0x7f63e80731b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005910 tx=0x7f63d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e4ff9700 1 -- 172.21.15.57:0/283370078 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d801d070 con 0x7f63e81010b0 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d80095f0 con 0x7f63e81010b0 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e4ff9700 1 -- 172.21.15.57:0/283370078 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63d8022470 con 0x7f63e81010b0 2023-10-06T22:16:19.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63e4ff9700 1 -- 172.21.15.57:0/283370078 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8017650 con 0x7f63e81010b0 2023-10-06T22:16:19.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.769+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e81aa520 con 0x7f63e81010b0 2023-10-06T22:16:19.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.774+0000 7f63e4ff9700 1 -- 172.21.15.57:0/283370078 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f63d80225e0 con 0x7f63e81010b0 2023-10-06T22:16:19.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.775+0000 7f64018d2b80 1 Processor -- start 2023-10-06T22:16:19.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.775+0000 7f64018d2b80 1 -- start start 2023-10-06T22:16:19.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.775+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f70c0 0x5560371f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.775+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037202350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.775+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x556037204c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f64018d2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556037160460 con 0x556037202890 2023-10-06T22:16:19.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f64018d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556037151c20 con 0x5560371f70c0 2023-10-06T22:16:19.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f64018d2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603714f540 con 0x5560371f79e0 2023-10-06T22:16:19.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e4ff9700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f63d00a15f0 0x7f63d00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037202350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f70c0 0x5560371f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x556037204c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037202350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48772/0 (socket says 172.21.15.57:48772) 2023-10-06T22:16:19.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 -- 172.21.15.57:0/530146956 learned_addr learned my addr 172.21.15.57:0/530146956 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e4ff9700 1 -- 172.21.15.57:0/283370078 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f63d80ce340 con 0x7f63e81010b0 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e7fff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f63d00a15f0 0x7f63d00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 -- 172.21.15.57:0/530146956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f70c0 msgr2=0x5560371f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 --2- 172.21.15.57:0/530146956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f70c0 0x5560371f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 -- 172.21.15.57:0/530146956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 msgr2=0x556037204c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 --2- 172.21.15.57:0/530146956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x556037204c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.776+0000 7f63e77fe700 1 -- 172.21.15.57:0/530146956 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556036f75de0 con 0x5560371f79e0 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.777+0000 7f63e77fe700 1 --2- 172.21.15.57:0/530146956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037202350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f63d8097e30 tx=0x7f63d806d280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913a91c1f386c897 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.777+0000 7f63cdffb700 1 -- 172.21.15.57:0/530146956 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d809ad20 con 0x5560371f79e0 2023-10-06T22:16:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.777+0000 7f63cdffb700 1 -- 172.21.15.57:0/530146956 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63d809bac0 con 0x5560371f79e0 2023-10-06T22:16:19.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.777+0000 7f63cdffb700 1 -- 172.21.15.57:0/530146956 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80997f0 con 0x5560371f79e0 2023-10-06T22:16:19.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.777+0000 7f64018d2b80 1 -- 172.21.15.57:0/530146956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 msgr2=0x556037202350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.777+0000 7f63e7fff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f63d00a15f0 0x7f63d00a3ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009fd0 tx=0x7f63e000b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.777+0000 7f64018d2b80 1 --2- 172.21.15.57:0/530146956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037202350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f63d8097e30 tx=0x7f63d806d280 comp rx=0 tx=0).stop 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 -- 172.21.15.57:0/530146956 shutdown_connections 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 --2- 172.21.15.57:0/530146956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x556037204c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 --2- 172.21.15.57:0/530146956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f70c0 0x5560371f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 --2- 172.21.15.57:0/530146956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037202350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 -- 172.21.15.57:0/530146956 >> 172.21.15.57:0/530146956 conn(0x55603716c260 msgr2=0x5560371f3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 -- 172.21.15.57:0/530146956 shutdown_connections 2023-10-06T22:16:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 -- 172.21.15.57:0/530146956 wait complete. 2023-10-06T22:16:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 -- start start 2023-10-06T22:16:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.778+0000 7f64018d2b80 1 Processor -- start 2023-10-06T22:16:19.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f70c0 0x556037202c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f79e0 0x5560372031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x55603720dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f64018d2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603714ae30 con 0x556037202890 2023-10-06T22:16:19.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f64018d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560370e7710 con 0x5560371f79e0 2023-10-06T22:16:19.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f64018d2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560370e73b0 con 0x5560371f70c0 2023-10-06T22:16:19.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f64018d2b80 10 client.0 _ll_register_callbacks cb 0x556036f44800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f63e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f79e0 0x5560372031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f70c0 0x556037202c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f63e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f79e0 0x5560372031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55238/0 (socket says 172.21.15.57:55238) 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f63e6ffd700 1 -- 172.21.15.57:0/2501853400 learned_addr learned my addr 172.21.15.57:0/2501853400 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.779+0000 7f63e7fff700 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x55603720dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63e6ffd700 1 -- 172.21.15.57:0/2501853400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f70c0 msgr2=0x556037202c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63e6ffd700 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f70c0 0x556037202c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63e6ffd700 1 -- 172.21.15.57:0/2501853400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 msgr2=0x55603720dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63e6ffd700 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x55603720dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63e6ffd700 1 -- 172.21.15.57:0/2501853400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e81aa520 con 0x5560371f79e0 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63e6ffd700 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f79e0 0x5560372031b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63dc012d20 tx=0x7f63dc00f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f842b5d76659a795 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63c6ffd700 1 -- 172.21.15.57:0/2501853400 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc017070 con 0x5560371f79e0 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560371eaf10 con 0x5560371f79e0 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.780+0000 7f63c6ffd700 1 -- 172.21.15.57:0/2501853400 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f63dc009e00 con 0x5560371f79e0 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f63c6ffd700 1 -- 172.21.15.57:0/2501853400 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63dc00fc30 con 0x5560371f79e0 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 10 client.26042 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:19.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556036f5ec60 con 0x5560371f79e0 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f63c6ffd700 1 -- 172.21.15.57:0/2501853400 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560371eaf10 con 0x5560371f79e0 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f63c6ffd700 1 -- 172.21.15.57:0/2501853400 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63dc003a50 con 0x5560371f79e0 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 10 client.26042 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 10 client.26042 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 20 client.26042 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x556037212080 0x5560372144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 4 client.26042 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.781+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556037143e10 con 0x556037212080 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.782+0000 7f63e77fe700 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x556037212080 0x5560372144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.782+0000 7f63e77fe700 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x556037212080 0x5560372144a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3a51a82724a7910a server_cookie=931cb97b393a32a5 in_seq=0 out_seq=0 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.782+0000 7f63c6ffd700 10 client.26042 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f63c6ffd700 1 -- 172.21.15.57:0/2501853400 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556037143e10 con 0x556037212080 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f63c6ffd700 10 client.26042 handle_command_reply: tid=0 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 client.26042 shutdown 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f79e0 msgr2=0x5560372031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f79e0 0x5560372031b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63dc012d20 tx=0x7f63dc00f6a0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 shutdown_connections 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x556037212080 0x5560372144a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556037202890 0x55603720dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f70c0 0x556037202c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f63c6ffd700 0 client.26042 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 --2- 172.21.15.57:0/2501853400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5560371f79e0 0x5560372031b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 >> 172.21.15.57:0/2501853400 conn(0x55603716c260 msgr2=0x5560371f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 shutdown_connections 2023-10-06T22:16:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.783+0000 7f64018d2b80 1 -- 172.21.15.57:0/2501853400 wait complete. 2023-10-06T22:16:19.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.784+0000 7f64018d2b80 20 client.26042 trim_cache size 0 max 16384 2023-10-06T22:16:19.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 Processor -- start 2023-10-06T22:16:19.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 -- start start 2023-10-06T22:16:19.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x5560371fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x5560371f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556036fcb920 con 0x5560371f70c0 2023-10-06T22:16:19.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560372081f0 con 0x556037202890 2023-10-06T22:16:19.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.799+0000 7f64018d2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560371ffeb0 con 0x5560371f79e0 2023-10-06T22:16:19.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x5560371f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x5560371f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55244/0 (socket says 172.21.15.57:55244) 2023-10-06T22:16:19.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 -- 172.21.15.57:0/3942105928 learned_addr learned my addr 172.21.15.57:0/3942105928 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 -- 172.21.15.57:0/3942105928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 msgr2=0x5560371fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 --2- 172.21.15.57:0/3942105928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x5560371fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 -- 172.21.15.57:0/3942105928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 msgr2=0x556037207cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 --2- 172.21.15.57:0/3942105928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 -- 172.21.15.57:0/3942105928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556036f9c250 con 0x556037202890 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.800+0000 7f63e7fff700 1 --2- 172.21.15.57:0/3942105928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x5560371f2d70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009470 tx=0x7f63e0009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e648a9b3c2167d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f63ce7fc700 1 -- 172.21.15.57:0/3942105928 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e001f050 con 0x556037202890 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f63ce7fc700 1 -- 172.21.15.57:0/3942105928 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e0020040 con 0x556037202890 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f63ce7fc700 1 -- 172.21.15.57:0/3942105928 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e000e040 con 0x556037202890 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f64018d2b80 1 -- 172.21.15.57:0/3942105928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 msgr2=0x5560371f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f64018d2b80 1 --2- 172.21.15.57:0/3942105928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x5560371f2d70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009470 tx=0x7f63e0009040 comp rx=0 tx=0).stop 2023-10-06T22:16:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f64018d2b80 1 -- 172.21.15.57:0/3942105928 shutdown_connections 2023-10-06T22:16:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f64018d2b80 1 --2- 172.21.15.57:0/3942105928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x5560371f2d70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f64018d2b80 1 --2- 172.21.15.57:0/3942105928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f64018d2b80 1 --2- 172.21.15.57:0/3942105928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x5560371fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.801+0000 7f64018d2b80 1 -- 172.21.15.57:0/3942105928 >> 172.21.15.57:0/3942105928 conn(0x55603716c260 msgr2=0x5560371f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.802+0000 7f64018d2b80 1 -- 172.21.15.57:0/3942105928 shutdown_connections 2023-10-06T22:16:19.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.802+0000 7f64018d2b80 1 -- 172.21.15.57:0/3942105928 wait complete. 2023-10-06T22:16:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.802+0000 7f64018d2b80 1 -- start start 2023-10-06T22:16:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.802+0000 7f64018d2b80 1 Processor -- start 2023-10-06T22:16:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037205de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f64018d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x556037206320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f64018d2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556037206a30 con 0x5560371f70c0 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f64018d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556037206ba0 con 0x556037202890 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f64018d2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556037265ec0 con 0x5560371f79e0 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f64018d2b80 10 client.0 _ll_register_callbacks cb 0x7f63dc0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x556037206320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037205de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58378/0 (socket says 172.21.15.57:58378) 2023-10-06T22:16:19.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 -- 172.21.15.57:0/1259659140 learned_addr learned my addr 172.21.15.57:0/1259659140 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 -- 172.21.15.57:0/1259659140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 msgr2=0x556037205de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037205de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 -- 172.21.15.57:0/1259659140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 msgr2=0x556037206320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x556037206320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.803+0000 7f63e77fe700 1 -- 172.21.15.57:0/1259659140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e0009150 con 0x5560371f70c0 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f63e77fe700 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207b20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f63d8022c70 tx=0x7f63d8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b5d8ecea9306f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f63c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f63c6ffd700 1 -- 172.21.15.57:0/1259659140 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d809b3c0 con 0x5560371f70c0 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f63c6ffd700 1 -- 172.21.15.57:0/1259659140 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f63d80a36c0 con 0x5560371f70c0 2023-10-06T22:16:19.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63d809bac0 con 0x5560371f70c0 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f63c6ffd700 1 -- 172.21.15.57:0/1259659140 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63d80228c0 con 0x5560371f70c0 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f64018d2b80 10 client.16050 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556036f742b0 con 0x5560371f70c0 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.804+0000 7f63c6ffd700 1 -- 172.21.15.57:0/1259659140 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63d809bac0 con 0x5560371f70c0 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f63c6ffd700 1 -- 172.21.15.57:0/1259659140 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63d809b620 con 0x5560371f70c0 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f64018d2b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f64018d2b80 10 client.16050 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f64018d2b80 20 client.16050 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f64018d2b80 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x5560372671d0 0x5560372695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:19.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f64018d2b80 4 client.16050 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:19.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556037143b30 con 0x5560372671d0 2023-10-06T22:16:19.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.805+0000 7f63e6ffd700 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x5560372671d0 0x5560372695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:19.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f63e6ffd700 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x5560372671d0 0x5560372695c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=288629e55e3939b5 server_cookie=68ec357d33a82100 in_seq=0 out_seq=0 2023-10-06T22:16:19.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f63c6ffd700 10 client.16050 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:19.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f63c6ffd700 1 -- 172.21.15.57:0/1259659140 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556037143b30 con 0x5560372671d0 2023-10-06T22:16:19.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f63c6ffd700 10 client.16050 handle_command_reply: tid=0 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 client.16050 shutdown 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 msgr2=0x556037207b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207b20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f63d8022c70 tx=0x7f63d8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 shutdown_connections 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x5560372671d0 0x5560372695c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556037202890 0x556037206320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560371f70c0 0x556037207b20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 --2- 172.21.15.57:0/1259659140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560371f79e0 0x556037205de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 >> 172.21.15.57:0/1259659140 conn(0x55603716c260 msgr2=0x5560371f2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f63c6ffd700 0 client.16050 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:19.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 shutdown_connections 2023-10-06T22:16:19.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 1 -- 172.21.15.57:0/1259659140 wait complete. 2023-10-06T22:16:19.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.806+0000 7f64018d2b80 20 client.16050 trim_cache size 0 max 16384 2023-10-06T22:16:19.817 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:19.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f63d00a15f0 msgr2=0x7f63d00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f63d00a15f0 0x7f63d00a3ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009fd0 tx=0x7f63e000b040 comp rx=0 tx=0).stop 2023-10-06T22:16:19.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81010b0 msgr2=0x7f63e80731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:19.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81010b0 0x7f63e80731b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005910 tx=0x7f63d8004730 comp rx=0 tx=0).stop 2023-10-06T22:16:19.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 shutdown_connections 2023-10-06T22:16:19.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f63d00a15f0 0x7f63d00a3ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e8073c30 0x7f63e81a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63e81010b0 0x7f63e80731b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 --2- 172.21.15.57:0/283370078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63e81061d0 0x7f63e80736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:19.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 >> 172.21.15.57:0/283370078 conn(0x7f63e80fc950 msgr2=0x7f63e8104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:19.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 shutdown_connections 2023-10-06T22:16:19.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:19.807+0000 7f63edfff700 1 -- 172.21.15.57:0/283370078 wait complete. 2023-10-06T22:16:19.828 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T22:16:20.130 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:19 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/782263494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:16:20.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:19 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/782263494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:16:20.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:19 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/782263494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:16:20.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.423+0000 7f6bb75ff700 1 -- 172.21.15.57:0/1489469313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bb01083f0 msgr2=0x7f6bb0108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.423+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/1489469313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bb01083f0 0x7f6bb0108870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009990 tx=0x7f6ba0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.423+0000 7f6bb75ff700 1 -- 172.21.15.57:0/1489469313 shutdown_connections 2023-10-06T22:16:20.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.423+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/1489469313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb011a690 0x7f6bb011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.424+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/1489469313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb010f640 0x7f6bb0107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.424+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/1489469313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bb01083f0 0x7f6bb0108870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.424+0000 7f6bb75ff700 1 -- 172.21.15.57:0/1489469313 >> 172.21.15.57:0/1489469313 conn(0x7f6bb006d1a0 msgr2=0x7f6bb006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.424+0000 7f6bb75ff700 1 -- 172.21.15.57:0/1489469313 shutdown_connections 2023-10-06T22:16:20.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.424+0000 7f6bb75ff700 1 -- 172.21.15.57:0/1489469313 wait complete. 2023-10-06T22:16:20.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.426+0000 7f6bb75ff700 1 Processor -- start 2023-10-06T22:16:20.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.427+0000 7f6bb75ff700 1 -- start start 2023-10-06T22:16:20.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.427+0000 7f6bb75ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bb01083f0 0x7f6bb0110950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.427+0000 7f6bb75ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb010f640 0x7f6bb0110e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.427+0000 7f6bb75ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb011a690 0x7f6bb01ba360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.427+0000 7f6bb75ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01ba930 con 0x7f6bb011a690 2023-10-06T22:16:20.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.427+0000 7f6bb75ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01baa70 con 0x7f6bb010f640 2023-10-06T22:16:20.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.427+0000 7f6bb75ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01bac30 con 0x7f6bb01083f0 2023-10-06T22:16:20.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.429+0000 7f6bb5b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb011a690 0x7f6bb01ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.429+0000 7f6bb5b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb011a690 0x7f6bb01ba360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58398/0 (socket says 172.21.15.57:58398) 2023-10-06T22:16:20.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.429+0000 7f6bb5b9c700 1 -- 172.21.15.57:0/3420783776 learned_addr learned my addr 172.21.15.57:0/3420783776 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.429+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb010f640 0x7f6bb0110e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.433+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/3420783776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bb01083f0 msgr2=0x7f6bb0110950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:16:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.433+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bb01083f0 0x7f6bb0110950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.433+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb011a690 msgr2=0x7f6bb01ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.433+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb011a690 0x7f6bb01ba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.433+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/3420783776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba00095f0 con 0x7f6bb010f640 2023-10-06T22:16:20.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.443+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb010f640 0x7f6bb0110e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009990 tx=0x7f6ba0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.454+0000 7f6ba67fc700 1 -- 172.21.15.57:0/3420783776 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba001d070 con 0x7f6bb010f640 2023-10-06T22:16:20.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.454+0000 7f6ba67fc700 1 -- 172.21.15.57:0/3420783776 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba0021410 con 0x7f6bb010f640 2023-10-06T22:16:20.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.454+0000 7f6ba67fc700 1 -- 172.21.15.57:0/3420783776 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0017650 con 0x7f6bb010f640 2023-10-06T22:16:20.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.454+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb01baeb0 con 0x7f6bb010f640 2023-10-06T22:16:20.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.455+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb01bb220 con 0x7f6bb010f640 2023-10-06T22:16:20.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.462+0000 7f6bcaff0b80 1 Processor -- start 2023-10-06T22:16:20.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.463+0000 7f6bcaff0b80 1 -- start start 2023-10-06T22:16:20.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.464+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8070c0 0x55936f8074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.464+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8079e0 0x55936f812350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.464+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f812890 0x55936f814c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.464+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f770460 con 0x55936f8079e0 2023-10-06T22:16:20.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.464+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f761c20 con 0x55936f812890 2023-10-06T22:16:20.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.464+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f75f540 con 0x55936f8070c0 2023-10-06T22:16:20.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.468+0000 7f6ba67fc700 1 -- 172.21.15.57:0/3420783776 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6ba000fa90 con 0x7f6bb010f640 2023-10-06T22:16:20.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.471+0000 7f6ba67fc700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6b9c0a1800 0x7f6b9c0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.471+0000 7f6ba67fc700 1 -- 172.21.15.57:0/3420783776 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f6ba00cdfa0 con 0x7f6bb010f640 2023-10-06T22:16:20.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.472+0000 7f6bb5b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8070c0 0x55936f8074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.472+0000 7f6bb539b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8079e0 0x55936f812350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.472+0000 7f6bb5b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8070c0 0x55936f8074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48820/0 (socket says 172.21.15.57:48820) 2023-10-06T22:16:20.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.472+0000 7f6bb5b9c700 1 -- 172.21.15.57:0/3342205417 learned_addr learned my addr 172.21.15.57:0/3342205417 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.472+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f812890 0x55936f814c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.475+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6b9c0a1800 0x7f6b9c0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.475+0000 7f6bb5b9c700 1 -- 172.21.15.57:0/3342205417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f812890 msgr2=0x55936f814c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.475+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f812890 0x55936f814c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.475+0000 7f6bb5b9c700 1 -- 172.21.15.57:0/3342205417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8079e0 msgr2=0x55936f812350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.475+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8079e0 0x55936f812350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.475+0000 7f6bb5b9c700 1 -- 172.21.15.57:0/3342205417 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55936f585de0 con 0x55936f8070c0 2023-10-06T22:16:20.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.477+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8070c0 0x55936f8074a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8011cf0 tx=0x7f6ba8018360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172c656beabbeea4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.477+0000 7f6b9b7fe700 1 -- 172.21.15.57:0/3342205417 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800e040 con 0x55936f8070c0 2023-10-06T22:16:20.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.478+0000 7f6b9b7fe700 1 -- 172.21.15.57:0/3342205417 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba800f040 con 0x55936f8070c0 2023-10-06T22:16:20.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.478+0000 7f6b9b7fe700 1 -- 172.21.15.57:0/3342205417 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8007500 con 0x55936f8070c0 2023-10-06T22:16:20.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.478+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6b9c0a1800 0x7f6b9c0a3cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80186b0 tx=0x7f6ba801e000 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/3342205417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8070c0 msgr2=0x55936f8074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8070c0 0x55936f8074a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8011cf0 tx=0x7f6ba8018360 comp rx=0 tx=0).stop 2023-10-06T22:16:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/3342205417 shutdown_connections 2023-10-06T22:16:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f812890 0x55936f814c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8070c0 0x55936f8074a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/3342205417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8079e0 0x55936f812350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/3342205417 >> 172.21.15.57:0/3342205417 conn(0x55936f77c260 msgr2=0x55936f803450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/3342205417 shutdown_connections 2023-10-06T22:16:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.480+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/3342205417 wait complete. 2023-10-06T22:16:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.481+0000 7f6bcaff0b80 1 -- start start 2023-10-06T22:16:20.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.481+0000 7f6bcaff0b80 1 Processor -- start 2023-10-06T22:16:20.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.484+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8070c0 0x55936f8122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.484+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8079e0 0x55936f80f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.484+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f812890 0x55936f80fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.484+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f75ae30 con 0x55936f812890 2023-10-06T22:16:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.484+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f6f7710 con 0x55936f8070c0 2023-10-06T22:16:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.484+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f6f73b0 con 0x55936f8079e0 2023-10-06T22:16:20.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.484+0000 7f6bcaff0b80 10 client.0 _ll_register_callbacks cb 0x55936f554800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:20.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8070c0 0x55936f8122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8070c0 0x55936f8122e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55296/0 (socket says 172.21.15.57:55296) 2023-10-06T22:16:20.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 -- 172.21.15.57:0/1904832001 learned_addr learned my addr 172.21.15.57:0/1904832001 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f812890 0x55936f80fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8079e0 0x55936f80f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 -- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8079e0 msgr2=0x55936f80f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8079e0 0x55936f80f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 -- 172.21.15.57:0/1904832001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f812890 msgr2=0x55936f80fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f812890 0x55936f80fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.485+0000 7f6bb539b700 1 -- 172.21.15.57:0/1904832001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba80118a0 con 0x55936f8070c0 2023-10-06T22:16:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.487+0000 7f6bb539b700 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8070c0 0x55936f8122e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00f700 tx=0x7f6bac00fac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5584cbc02ca7c577 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.487+0000 7f6b98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.487+0000 7f6b98ff9700 1 -- 172.21.15.57:0/1904832001 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00a6f0 con 0x55936f8070c0 2023-10-06T22:16:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.488+0000 7f6b98ff9700 1 -- 172.21.15.57:0/1904832001 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f6bac010960 con 0x55936f8070c0 2023-10-06T22:16:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.488+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55936f7faf10 con 0x55936f8070c0 2023-10-06T22:16:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.488+0000 7f6b98ff9700 1 -- 172.21.15.57:0/1904832001 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bac010b20 con 0x55936f8070c0 2023-10-06T22:16:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.488+0000 7f6bcaff0b80 10 client.26060 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.488+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55936f56ec60 con 0x55936f8070c0 2023-10-06T22:16:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.489+0000 7f6b98ff9700 1 -- 172.21.15.57:0/1904832001 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55936f7faf10 con 0x55936f8070c0 2023-10-06T22:16:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.489+0000 7f6b98ff9700 1 -- 172.21.15.57:0/1904832001 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6bac020070 con 0x55936f8070c0 2023-10-06T22:16:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.490+0000 7f6bcaff0b80 10 client.26060 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.490+0000 7f6bcaff0b80 10 client.26060 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:16:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.490+0000 7f6bcaff0b80 20 client.26060 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.490+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8222a0 0x55936f8246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.490+0000 7f6bcaff0b80 4 client.26060 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.490+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55936f753e10 con 0x55936f8222a0 2023-10-06T22:16:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.491+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8222a0 0x55936f8246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.492+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8222a0 0x55936f8246c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9787b7f1405e2749 server_cookie=2af6477571c277d5 in_seq=0 out_seq=0 2023-10-06T22:16:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.492+0000 7f6b98ff9700 10 client.26060 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.493+0000 7f6b98ff9700 1 -- 172.21.15.57:0/1904832001 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55936f753e10 con 0x55936f8222a0 2023-10-06T22:16:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.493+0000 7f6b98ff9700 10 client.26060 handle_command_reply: tid=0 2023-10-06T22:16:20.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.493+0000 7f6bcaff0b80 1 client.26060 shutdown 2023-10-06T22:16:20.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.493+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8070c0 msgr2=0x55936f8122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.493+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8070c0 0x55936f8122e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00f700 tx=0x7f6bac00fac0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 shutdown_connections 2023-10-06T22:16:20.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8222a0 0x55936f8246c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f812890 0x55936f80fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8070c0 0x55936f8122e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/1904832001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f8079e0 0x55936f80f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 >> 172.21.15.57:0/1904832001 conn(0x55936f77c260 msgr2=0x55936f803040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6b98ff9700 0 client.26060 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 shutdown_connections 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/1904832001 wait complete. 2023-10-06T22:16:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.494+0000 7f6bcaff0b80 20 client.26060 trim_cache size 0 max 16384 2023-10-06T22:16:20.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.511+0000 7f6bcaff0b80 1 Processor -- start 2023-10-06T22:16:20.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.511+0000 7f6bcaff0b80 1 -- start start 2023-10-06T22:16:20.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.511+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f811e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.511+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f802f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.511+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f8034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f5db920 con 0x55936f8070c0 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f80b410 con 0x55936f8079e0 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f804070 con 0x55936f812890 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f802f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb539b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f811e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f802f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55304/0 (socket says 172.21.15.57:55304) 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2106875326 learned_addr learned my addr 172.21.15.57:0/2106875326 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f8034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2106875326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 msgr2=0x55936f8034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f8034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2106875326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 msgr2=0x55936f811e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f811e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2106875326 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55936f5ac250 con 0x55936f8079e0 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.512+0000 7f6bb539b700 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f811e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.513+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f802f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ba009afb0 tx=0x7f6ba001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719a26ff38857f7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.513+0000 7f6b9bfff700 1 -- 172.21.15.57:0/2106875326 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba009b020 con 0x55936f8079e0 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.513+0000 7f6b9bfff700 1 -- 172.21.15.57:0/2106875326 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba009f170 con 0x55936f8079e0 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.513+0000 7f6b9bfff700 1 -- 172.21.15.57:0/2106875326 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba009b280 con 0x55936f8079e0 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2106875326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 msgr2=0x55936f802f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f802f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ba009afb0 tx=0x7f6ba001a040 comp rx=0 tx=0).stop 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2106875326 shutdown_connections 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f8034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f811e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2106875326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f802f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2106875326 >> 172.21.15.57:0/2106875326 conn(0x55936f77c260 msgr2=0x55936f80a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2106875326 shutdown_connections 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2106875326 wait complete. 2023-10-06T22:16:20.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 -- start start 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.514+0000 7f6bcaff0b80 1 Processor -- start 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f80bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f80c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bcaff0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f80c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f8199c0 con 0x55936f8070c0 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f876090 con 0x55936f8079e0 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bcaff0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936f876220 con 0x55936f812890 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bcaff0b80 10 client.0 _ll_register_callbacks cb 0x55936f554800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bb4b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f80c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bb539b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f80bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bb4b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f80c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55314/0 (socket says 172.21.15.57:55314) 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2457674380 learned_addr learned my addr 172.21.15.57:0/2457674380 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.515+0000 7f6bb5b9c700 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f80c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 msgr2=0x55936f80c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f80c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2457674380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 msgr2=0x55936f80bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f80bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6bb4b9a700 1 -- 172.21.15.57:0/2457674380 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55936f56ec60 con 0x55936f8079e0 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6bb4b9a700 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f80c2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0021d10 tx=0x7f6ba009adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c819d3e7125e08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6b98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6b98ff9700 1 -- 172.21.15.57:0/2457674380 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00a3320 con 0x55936f8079e0 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6b98ff9700 1 -- 172.21.15.57:0/2457674380 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f6ba009b020 con 0x55936f8079e0 2023-10-06T22:16:20.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ba800f040 con 0x55936f8079e0 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.516+0000 7f6b98ff9700 1 -- 172.21.15.57:0/2457674380 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba00cb980 con 0x55936f8079e0 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 10 client.26066 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f6bb01bb220 con 0x55936f8079e0 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6b98ff9700 1 -- 172.21.15.57:0/2457674380 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba800f040 con 0x55936f8079e0 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6b98ff9700 1 -- 172.21.15.57:0/2457674380 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ba009a660 con 0x55936f8079e0 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 10 client.26066 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 10 client.26066 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 20 client.26066 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8777a0 0x55936f879bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 4 client.26066 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55936f753b30 con 0x55936f8777a0 2023-10-06T22:16:20.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.517+0000 7f6bb539b700 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8777a0 0x55936f879bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6bb539b700 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8777a0 0x55936f879bc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bde13bf724cdcc01 server_cookie=c8f20d22f2b413ba in_seq=0 out_seq=0 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6b98ff9700 10 client.26066 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6b98ff9700 1 -- 172.21.15.57:0/2457674380 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55936f753b30 con 0x55936f8777a0 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6b98ff9700 10 client.26066 handle_command_reply: tid=0 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6bcaff0b80 1 client.26066 shutdown 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 msgr2=0x55936f80c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f80c2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0021d10 tx=0x7f6ba009adf0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 shutdown_connections 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x55936f8777a0 0x55936f879bc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.518+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55936f812890 0x55936f80c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.519+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55936f8070c0 0x55936f80bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.519+0000 7f6bcaff0b80 1 --2- 172.21.15.57:0/2457674380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55936f8079e0 0x55936f80c2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.519+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 >> 172.21.15.57:0/2457674380 conn(0x55936f77c260 msgr2=0x55936f802f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.519+0000 7f6b98ff9700 0 client.26066 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.519+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 shutdown_connections 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.519+0000 7f6bcaff0b80 1 -- 172.21.15.57:0/2457674380 wait complete. 2023-10-06T22:16:20.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.519+0000 7f6bcaff0b80 20 client.26066 trim_cache size 0 max 16384 2023-10-06T22:16:20.531 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:20.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6b9c0a1800 msgr2=0x7f6b9c0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6b9c0a1800 0x7f6b9c0a3cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80186b0 tx=0x7f6ba801e000 comp rx=0 tx=0).stop 2023-10-06T22:16:20.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb010f640 msgr2=0x7f6bb0110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb010f640 0x7f6bb0110e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009990 tx=0x7f6ba0009580 comp rx=0 tx=0).stop 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 shutdown_connections 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6b9c0a1800 0x7f6b9c0a3cc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6bb011a690 0x7f6bb01ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bb01083f0 0x7f6bb0110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 --2- 172.21.15.57:0/3420783776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb010f640 0x7f6bb0110e90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 >> 172.21.15.57:0/3420783776 conn(0x7f6bb006d1a0 msgr2=0x7f6bb01190f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 shutdown_connections 2023-10-06T22:16:20.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.520+0000 7f6bb75ff700 1 -- 172.21.15.57:0/3420783776 wait complete. 2023-10-06T22:16:20.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T22:16:20.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 -- 172.21.15.57:0/93401075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c102050 msgr2=0x7f755c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 --2- 172.21.15.57:0/93401075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c102050 0x7f755c102430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f754c009990 tx=0x7f754c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 -- 172.21.15.57:0/93401075 shutdown_connections 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 --2- 172.21.15.57:0/93401075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f755c10b470 0x7f755c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 --2- 172.21.15.57:0/93401075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c102050 0x7f755c102430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 --2- 172.21.15.57:0/93401075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f755c102970 0x7f755c10ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 -- 172.21.15.57:0/93401075 >> 172.21.15.57:0/93401075 conn(0x7f755c0fb890 msgr2=0x7f755c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 -- 172.21.15.57:0/93401075 shutdown_connections 2023-10-06T22:16:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.780+0000 7f7560dff700 1 -- 172.21.15.57:0/93401075 wait complete. 2023-10-06T22:16:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.781+0000 7f7560dff700 1 Processor -- start 2023-10-06T22:16:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.781+0000 7f7560dff700 1 -- start start 2023-10-06T22:16:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f7560dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f755c102050 0x7f755c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f7560dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f755c102970 0x7f755c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f7560dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c10b470 0x7f755c1a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f7560dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c100a10 con 0x7f755c102970 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f7560dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1a5680 con 0x7f755c10b470 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f7560dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1a57c0 con 0x7f755c102050 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f7559d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f755c102970 0x7f755c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c10b470 0x7f755c1a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c10b470 0x7f755c1a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55342/0 (socket says 172.21.15.57:55342) 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f755ad9d700 1 -- 172.21.15.57:0/4089177574 learned_addr learned my addr 172.21.15.57:0/4089177574 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.782+0000 7f755a59c700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f755c102050 0x7f755c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f755ad9d700 1 -- 172.21.15.57:0/4089177574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f755c102050 msgr2=0x7f755c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f755ad9d700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f755c102050 0x7f755c0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f755ad9d700 1 -- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f755c102970 msgr2=0x7f755c1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f755ad9d700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f755c102970 0x7f755c1001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f755ad9d700 1 -- 172.21.15.57:0/4089177574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0095f0 con 0x7f755c10b470 2023-10-06T22:16:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f755ad9d700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c10b470 0x7f755c1a5140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7554007ee0 tx=0x7f755400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f754b7fe700 1 -- 172.21.15.57:0/4089177574 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755400dd20 con 0x7f755c10b470 2023-10-06T22:16:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f754b7fe700 1 -- 172.21.15.57:0/4089177574 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f755400f040 con 0x7f755c10b470 2023-10-06T22:16:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c1a5a20 con 0x7f755c10b470 2023-10-06T22:16:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.783+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1a5eb0 con 0x7f755c10b470 2023-10-06T22:16:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.784+0000 7f754b7fe700 1 -- 172.21.15.57:0/4089177574 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540048a0 con 0x7f755c10b470 2023-10-06T22:16:20.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.788+0000 7f754b7fe700 1 -- 172.21.15.57:0/4089177574 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7554004020 con 0x7f755c10b470 2023-10-06T22:16:20.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.789+0000 7f757471fb80 1 Processor -- start 2023-10-06T22:16:20.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.789+0000 7f757471fb80 1 -- start start 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f757471fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f754b7fe700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f75440a15f0 0x7f75440a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f757471fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 0x5575ee100350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f757471fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f757471fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee05e460 con 0x5575ee0f50c0 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f757471fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee04fc20 con 0x5575ee100890 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f757471fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee04d540 con 0x5575ee0f59e0 2023-10-06T22:16:20.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f754b7fe700 1 -- 172.21.15.57:0/4089177574 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f75540cc790 con 0x7f755c10b470 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55346/0 (socket says 172.21.15.57:55346) 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f7559d9b700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f75440a15f0 0x7f75440a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 -- 172.21.15.57:0/396259903 learned_addr learned my addr 172.21.15.57:0/396259903 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 -- 172.21.15.57:0/396259903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 msgr2=0x5575ee100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 --2- 172.21.15.57:0/396259903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 0x5575ee100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 -- 172.21.15.57:0/396259903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 msgr2=0x5575ee0f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 --2- 172.21.15.57:0/396259903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.790+0000 7f755ad9d700 1 -- 172.21.15.57:0/396259903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ede73de0 con 0x5575ee100890 2023-10-06T22:16:20.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.791+0000 7f755ad9d700 1 --2- 172.21.15.57:0/396259903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee102c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7554096230 tx=0x7f75540996a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2a1b3cae1d8f42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.791+0000 7f7559d9b700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f75440a15f0 0x7f75440a3ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f755c101490 tx=0x7f7550007840 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.791+0000 7f7548ff9700 1 -- 172.21.15.57:0/396259903 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554076e80 con 0x5575ee100890 2023-10-06T22:16:20.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.791+0000 7f7548ff9700 1 -- 172.21.15.57:0/396259903 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7554076ca0 con 0x5575ee100890 2023-10-06T22:16:20.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.791+0000 7f7548ff9700 1 -- 172.21.15.57:0/396259903 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554099830 con 0x5575ee100890 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 -- 172.21.15.57:0/396259903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 msgr2=0x5575ee102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 --2- 172.21.15.57:0/396259903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee102c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7554096230 tx=0x7f75540996a0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 -- 172.21.15.57:0/396259903 shutdown_connections 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 --2- 172.21.15.57:0/396259903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee102c80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 --2- 172.21.15.57:0/396259903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 --2- 172.21.15.57:0/396259903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 0x5575ee100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 -- 172.21.15.57:0/396259903 >> 172.21.15.57:0/396259903 conn(0x5575ee06a260 msgr2=0x5575ee0f1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 -- 172.21.15.57:0/396259903 shutdown_connections 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 -- 172.21.15.57:0/396259903 wait complete. 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.792+0000 7f757471fb80 1 -- start start 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 1 Processor -- start 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee100350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 0x5575ee0fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee0fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee048e30 con 0x5575ee0f50c0 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575edfe5710 con 0x5575ee100890 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575edfe53b0 con 0x5575ee0f59e0 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f757471fb80 10 client.0 _ll_register_callbacks cb 0x5575ede42800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee0fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f7559d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 0x5575ee0fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee0fdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55350/0 (socket says 172.21.15.57:55350) 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f755ad9d700 1 -- 172.21.15.57:0/2508426192 learned_addr learned my addr 172.21.15.57:0/2508426192 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.793+0000 7f755a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f755ad9d700 1 -- 172.21.15.57:0/2508426192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 msgr2=0x5575ee0fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f755ad9d700 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 0x5575ee0fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f755ad9d700 1 -- 172.21.15.57:0/2508426192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 msgr2=0x5575ee100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f755ad9d700 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f755ad9d700 1 -- 172.21.15.57:0/2508426192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1a5eb0 con 0x5575ee100890 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f755a59c700 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f755ad9d700 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee0fdd10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7554099750 tx=0x7f7554009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8e73cc5c729050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f7541ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f7541ffb700 1 -- 172.21.15.57:0/2508426192 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554076f00 con 0x5575ee100890 2023-10-06T22:16:20.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f7541ffb700 1 -- 172.21.15.57:0/2508426192 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7554099ec0 con 0x5575ee100890 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575ee0e8f10 con 0x5575ee100890 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f7541ffb700 1 -- 172.21.15.57:0/2508426192 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755409d910 con 0x5575ee100890 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f757471fb80 10 client.26078 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.794+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5575ede5cc60 con 0x5575ee100890 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f7541ffb700 1 -- 172.21.15.57:0/2508426192 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575ee0e8f10 con 0x5575ee100890 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f7541ffb700 1 -- 172.21.15.57:0/2508426192 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75540ca1b0 con 0x5575ee100890 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f757471fb80 10 client.26078 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f757471fb80 10 client.26078 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f757471fb80 20 client.26078 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f757471fb80 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee110060 0x5575ee112480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f757471fb80 4 client.26078 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:20.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575ee041e10 con 0x5575ee110060 2023-10-06T22:16:20.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.795+0000 7f755a59c700 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee110060 0x5575ee112480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.796+0000 7f755a59c700 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee110060 0x5575ee112480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eaa57cf702cc4739 server_cookie=8bad2c49fdbb0635 in_seq=0 out_seq=0 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.796+0000 7f7541ffb700 10 client.26078 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f7541ffb700 1 -- 172.21.15.57:0/2508426192 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5575ee041e10 con 0x5575ee110060 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f7541ffb700 10 client.26078 handle_command_reply: tid=0 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 client.26078 shutdown 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 msgr2=0x5575ee0fdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee0fdd10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7554099750 tx=0x7f7554009480 comp rx=0 tx=0).stop 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 shutdown_connections 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee110060 0x5575ee112480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee0fdd10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f7541ffb700 0 client.26078 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 --2- 172.21.15.57:0/2508426192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f59e0 0x5575ee0fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.797+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 >> 172.21.15.57:0/2508426192 conn(0x5575ee06a260 msgr2=0x5575ee0f0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.798+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 shutdown_connections 2023-10-06T22:16:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.798+0000 7f757471fb80 1 -- 172.21.15.57:0/2508426192 wait complete. 2023-10-06T22:16:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.798+0000 7f757471fb80 20 client.26078 trim_cache size 0 max 16384 2023-10-06T22:16:20.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.811+0000 7f757471fb80 1 Processor -- start 2023-10-06T22:16:20.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.811+0000 7f757471fb80 1 -- start start 2023-10-06T22:16:20.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f757471fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f757471fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee0f59e0 0x5575ee0f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f757471fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee100890 0x5575ee0fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f757471fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575edec9920 con 0x5575ee0f50c0 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f757471fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee0f84e0 con 0x5575ee0f59e0 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f757471fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee0f56f0 con 0x5575ee100890 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58468/0 (socket says 172.21.15.57:58468) 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 -- 172.21.15.57:0/3215662711 learned_addr learned my addr 172.21.15.57:0/3215662711 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f7559d9b700 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee0f59e0 0x5575ee0f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755ad9d700 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee100890 0x5575ee0fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 -- 172.21.15.57:0/3215662711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee100890 msgr2=0x5575ee0fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee100890 0x5575ee0fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 -- 172.21.15.57:0/3215662711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee0f59e0 msgr2=0x5575ee0f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee0f59e0 0x5575ee0f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f755a59c700 1 -- 172.21.15.57:0/3215662711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ede9a250 con 0x5575ee0f50c0 2023-10-06T22:16:20.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.812+0000 7f7559d9b700 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee0f59e0 0x5575ee0f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.813+0000 7f755a59c700 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f8990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f754c015000 tx=0x7f754c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd046e6667ca32cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.813+0000 7f75497fa700 1 -- 172.21.15.57:0/3215662711 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c01d040 con 0x5575ee0f50c0 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.813+0000 7f75497fa700 1 -- 172.21.15.57:0/3215662711 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754c01e040 con 0x5575ee0f50c0 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.813+0000 7f75497fa700 1 -- 172.21.15.57:0/3215662711 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c003b80 con 0x5575ee0f50c0 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 -- 172.21.15.57:0/3215662711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 msgr2=0x5575ee0f8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f8990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f754c015000 tx=0x7f754c005c70 comp rx=0 tx=0).stop 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 -- 172.21.15.57:0/3215662711 shutdown_connections 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee100890 0x5575ee0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f50c0 0x5575ee0f8990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 --2- 172.21.15.57:0/3215662711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee0f59e0 0x5575ee0f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 -- 172.21.15.57:0/3215662711 >> 172.21.15.57:0/3215662711 conn(0x5575ee06a260 msgr2=0x5575ee10c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 -- 172.21.15.57:0/3215662711 shutdown_connections 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 -- 172.21.15.57:0/3215662711 wait complete. 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 -- start start 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.814+0000 7f757471fb80 1 Processor -- start 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f757471fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f50c0 0x5575ee1055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f757471fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f59e0 0x5575ee105b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f757471fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f757471fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee163390 con 0x5575ee0f59e0 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f757471fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee1634d0 con 0x5575ee100890 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f757471fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee163640 con 0x5575ee0f50c0 2023-10-06T22:16:20.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f757471fb80 10 client.0 _ll_register_callbacks cb 0x7f754c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f7559d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f59e0 0x5575ee105b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f7559d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f59e0 0x5575ee105b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58476/0 (socket says 172.21.15.57:58476) 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f7559d9b700 1 -- 172.21.15.57:0/557113185 learned_addr learned my addr 172.21.15.57:0/557113185 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.815+0000 7f755a59c700 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f50c0 0x5575ee1055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f755ad9d700 1 -- 172.21.15.57:0/557113185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f50c0 msgr2=0x5575ee1055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f755ad9d700 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f50c0 0x5575ee1055e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f755ad9d700 1 -- 172.21.15.57:0/557113185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f59e0 msgr2=0x5575ee105b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f755ad9d700 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f59e0 0x5575ee105b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f755ad9d700 1 -- 172.21.15.57:0/557113185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550012b10 con 0x5575ee100890 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f755ad9d700 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee162e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f755400d700 tx=0x7f7554099220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e992eca3b3a8dab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f7541ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:20.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f7541ffb700 1 -- 172.21.15.57:0/557113185 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755405f1b0 con 0x5575ee100890 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7554076ca0 con 0x5575ee100890 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f7541ffb700 1 -- 172.21.15.57:0/557113185 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f7554099ec0 con 0x5575ee100890 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f7541ffb700 1 -- 172.21.15.57:0/557113185 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7554076e80 con 0x5575ee100890 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f757471fb80 10 client.26084 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f755c10e5f0 con 0x5575ee100890 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.816+0000 7f7541ffb700 1 -- 172.21.15.57:0/557113185 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7554076ca0 con 0x5575ee100890 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f7541ffb700 1 -- 172.21.15.57:0/557113185 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75540a16c0 con 0x5575ee100890 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f757471fb80 10 client.26084 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f757471fb80 10 client.26084 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f757471fb80 20 client.26084 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f757471fb80 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee164940 0x5575ee166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:20.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f757471fb80 4 client.26084 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:20.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575ee041b30 con 0x5575ee164940 2023-10-06T22:16:20.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f755a59c700 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee164940 0x5575ee166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:20.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f755a59c700 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee164940 0x5575ee166d60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6cc1339c4d84dc server_cookie=e47f6f1842b32174 in_seq=0 out_seq=0 2023-10-06T22:16:20.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.817+0000 7f7541ffb700 10 client.26084 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:20.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.819+0000 7f7541ffb700 1 -- 172.21.15.57:0/557113185 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575ee041b30 con 0x5575ee164940 2023-10-06T22:16:20.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.819+0000 7f7541ffb700 10 client.26084 handle_command_reply: tid=0 2023-10-06T22:16:20.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.819+0000 7f757471fb80 1 client.26084 shutdown 2023-10-06T22:16:20.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 msgr2=0x5575ee162e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee162e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f755400d700 tx=0x7f7554099220 comp rx=0 tx=0).stop 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 shutdown_connections 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5575ee164940 0x5575ee166d60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575ee100890 0x5575ee162e50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575ee0f50c0 0x5575ee1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f7541ffb700 0 client.26084 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 --2- 172.21.15.57:0/557113185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575ee0f59e0 0x5575ee105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 >> 172.21.15.57:0/557113185 conn(0x5575ee06a260 msgr2=0x5575ee0f8990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 shutdown_connections 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 1 -- 172.21.15.57:0/557113185 wait complete. 2023-10-06T22:16:20.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f757471fb80 20 client.26084 trim_cache size 0 max 16384 2023-10-06T22:16:20.826 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:20.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f75440a15f0 msgr2=0x7f75440a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.820+0000 7f7560dff700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f75440a15f0 0x7f75440a3ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f755c101490 tx=0x7f7550007840 comp rx=0 tx=0).stop 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c10b470 msgr2=0x7f755c1a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c10b470 0x7f755c1a5140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7554007ee0 tx=0x7f755400d3b0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 shutdown_connections 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f75440a15f0 0x7f75440a3ab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f755c10b470 0x7f755c1a5140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f755c102050 0x7f755c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 --2- 172.21.15.57:0/4089177574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f755c102970 0x7f755c1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 >> 172.21.15.57:0/4089177574 conn(0x7f755c0fb890 msgr2=0x7f755c10d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 shutdown_connections 2023-10-06T22:16:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:20.821+0000 7f7560dff700 1 -- 172.21.15.57:0/4089177574 wait complete. 2023-10-06T22:16:20.842 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T22:16:21.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.134+0000 7fd0dddff700 1 -- 172.21.15.57:0/1016840412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8073b70 msgr2=0x7fd0d8111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.134+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1016840412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8073b70 0x7fd0d8111910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0098f0 tx=0x7fd0cc009c00 comp rx=0 tx=0).stop 2023-10-06T22:16:21.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.134+0000 7fd0dddff700 1 -- 172.21.15.57:0/1016840412 shutdown_connections 2023-10-06T22:16:21.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.134+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1016840412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d8111e50 0x7fd0d8114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.134+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1016840412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 0x7fd0d8073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.134+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1016840412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8073b70 0x7fd0d8111910 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.134+0000 7fd0dddff700 1 -- 172.21.15.57:0/1016840412 >> 172.21.15.57:0/1016840412 conn(0x7fd0d80fc950 msgr2=0x7fd0d80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.135+0000 7fd0dddff700 1 -- 172.21.15.57:0/1016840412 shutdown_connections 2023-10-06T22:16:21.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.135+0000 7fd0dddff700 1 -- 172.21.15.57:0/1016840412 wait complete. 2023-10-06T22:16:21.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 Processor -- start 2023-10-06T22:16:21.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 -- start start 2023-10-06T22:16:21.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 0x7fd0d819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d8073b70 0x7fd0d819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8111e50 0x7fd0d81a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d81a25f0 con 0x7fd0d8073b70 2023-10-06T22:16:21.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d81a2760 con 0x7fd0d8073250 2023-10-06T22:16:21.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0dddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d81a2920 con 0x7fd0d8111e50 2023-10-06T22:16:21.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 0x7fd0d819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0d7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8111e50 0x7fd0d81a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 0x7fd0d819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55392/0 (socket says 172.21.15.57:55392) 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.136+0000 7fd0d77fe700 1 -- 172.21.15.57:0/1281099410 learned_addr learned my addr 172.21.15.57:0/1281099410 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d8073b70 0x7fd0d819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d7fff700 1 -- 172.21.15.57:0/1281099410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 msgr2=0x7fd0d819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 0x7fd0d819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d7fff700 1 -- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d8073b70 msgr2=0x7fd0d819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d8073b70 0x7fd0d819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d7fff700 1 -- 172.21.15.57:0/1281099410 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80095f0 con 0x7fd0d8111e50 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 0x7fd0d819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8111e50 0x7fd0d81a1fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0d806a470 tx=0x7fd0d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d4ff9700 1 -- 172.21.15.57:0/1281099410 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0003a00 con 0x7fd0d8111e50 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d4ff9700 1 -- 172.21.15.57:0/1281099410 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0d000f040 con 0x7fd0d8111e50 2023-10-06T22:16:21.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0d4ff9700 1 -- 172.21.15.57:0/1281099410 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0004570 con 0x7fd0d8111e50 2023-10-06T22:16:21.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.137+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d0009c40 con 0x7fd0d8111e50 2023-10-06T22:16:21.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.138+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc009520 con 0x7fd0d8111e50 2023-10-06T22:16:21.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.142+0000 7fd0f171ab80 1 Processor -- start 2023-10-06T22:16:21.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.142+0000 7fd0d4ff9700 1 -- 172.21.15.57:0/1281099410 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd0d0003b60 con 0x7fd0d8111e50 2023-10-06T22:16:21.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0f171ab80 1 -- start start 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced0c0 0x562a84ced4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cf8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cfac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84c56460 con 0x562a84cf8890 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84c47c20 con 0x562a84ced9e0 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84c45540 con 0x562a84ced0c0 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cf8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cf8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55396/0 (socket says 172.21.15.57:55396) 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0d77fe700 1 -- 172.21.15.57:0/2078889006 learned_addr learned my addr 172.21.15.57:0/2078889006 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced0c0 0x562a84ced4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.143+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cfac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0d6ffd700 1 -- 172.21.15.57:0/2078889006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 msgr2=0x562a84cf8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cf8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0d6ffd700 1 -- 172.21.15.57:0/2078889006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 msgr2=0x562a84cfac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cfac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0d6ffd700 1 -- 172.21.15.57:0/2078889006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a84a6bde0 con 0x562a84ced0c0 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced0c0 0x562a84ced4a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009810 tx=0x7fd0cc019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f1b2b07a4e4252 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0bdffb700 1 -- 172.21.15.57:0/2078889006 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00f510 con 0x562a84ced0c0 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0bdffb700 1 -- 172.21.15.57:0/2078889006 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0cc00ba20 con 0x562a84ced0c0 2023-10-06T22:16:21.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.144+0000 7fd0bdffb700 1 -- 172.21.15.57:0/2078889006 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc005290 con 0x562a84ced0c0 2023-10-06T22:16:21.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.145+0000 7fd0d4ff9700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd0c00a1670 0x7fd0c00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.145+0000 7fd0d4ff9700 1 -- 172.21.15.57:0/1281099410 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd0d00ccba0 con 0x7fd0d8111e50 2023-10-06T22:16:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.145+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd0c00a1670 0x7fd0c00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.145+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2078889006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced0c0 msgr2=0x562a84ced4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.145+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced0c0 0x562a84ced4a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009810 tx=0x7fd0cc019040 comp rx=0 tx=0).stop 2023-10-06T22:16:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2078889006 shutdown_connections 2023-10-06T22:16:21.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cfac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced0c0 0x562a84ced4a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2078889006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cf8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2078889006 >> 172.21.15.57:0/2078889006 conn(0x562a84c62260 msgr2=0x562a84ce9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2078889006 shutdown_connections 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd0c00a1670 0x7fd0c00a3b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8012840 tx=0x7fd0c8009310 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2078889006 wait complete. 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.146+0000 7fd0f171ab80 1 -- start start 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.147+0000 7fd0f171ab80 1 Processor -- start 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced0c0 0x562a84cf5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cf6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84c40e30 con 0x562a84cf8890 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84bdd710 con 0x562a84ced0c0 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84bdd3b0 con 0x562a84ced9e0 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0f171ab80 10 client.0 _ll_register_callbacks cb 0x562a84a3a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.148+0000 7fd0d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced0c0 0x562a84cf5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced0c0 0x562a84cf5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55404/0 (socket says 172.21.15.57:55404) 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 -- 172.21.15.57:0/3501019280 learned_addr learned my addr 172.21.15.57:0/3501019280 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cf6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 -- 172.21.15.57:0/3501019280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 msgr2=0x562a84cf5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 -- 172.21.15.57:0/3501019280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 msgr2=0x562a84cf6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:21.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cf6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 -- 172.21.15.57:0/3501019280 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc01f040 con 0x562a84ced0c0 2023-10-06T22:16:21.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced0c0 0x562a84cf5610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800f790 tx=0x7fd0c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ec8396abb6cfac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:21.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.149+0000 7fd0b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:21.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.150+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3501019280 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8017070 con 0x562a84ced0c0 2023-10-06T22:16:21.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.150+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3501019280 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd0c8011710 con 0x562a84ced0c0 2023-10-06T22:16:21.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.150+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a84ce0f10 con 0x562a84ced0c0 2023-10-06T22:16:21.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.150+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3501019280 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0c80118d0 con 0x562a84ced0c0 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.150+0000 7fd0f171ab80 10 client.26096 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.150+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562a84a54c60 con 0x562a84ced0c0 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.150+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3501019280 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a84ce0f10 con 0x562a84ced0c0 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3501019280 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd0c8029af0 con 0x562a84ced0c0 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0f171ab80 10 client.26096 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0f171ab80 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0f171ab80 20 client.26096 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d08560 0x562a84d0a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0f171ab80 4 client.26096 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a84c39e10 con 0x562a84d08560 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.151+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d08560 0x562a84d0a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.152+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d08560 0x562a84d0a980 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e1a0467e3cbef3 server_cookie=55bbfbe8fa406ae4 in_seq=0 out_seq=0 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.152+0000 7fd0b6ffd700 10 client.26096 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.153+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3501019280 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562a84c39e10 con 0x562a84d08560 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.153+0000 7fd0b6ffd700 10 client.26096 handle_command_reply: tid=0 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.153+0000 7fd0f171ab80 1 client.26096 shutdown 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.153+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced0c0 msgr2=0x562a84cf5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.153+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced0c0 0x562a84cf5610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800f790 tx=0x7fd0c80047c0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 shutdown_connections 2023-10-06T22:16:21.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d08560 0x562a84d0a980 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84cf8890 0x562a84cf6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced0c0 0x562a84cf5610 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3501019280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 >> 172.21.15.57:0/3501019280 conn(0x562a84c62260 msgr2=0x562a84ce8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:21.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0b6ffd700 0 client.26096 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:21.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 shutdown_connections 2023-10-06T22:16:21.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3501019280 wait complete. 2023-10-06T22:16:21.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.154+0000 7fd0f171ab80 20 client.26096 trim_cache size 0 max 16384 2023-10-06T22:16:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:20 smithi057 ceph-mon[98492]: pgmap v51: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 983 KiB/s wr, 48 op/s 2023-10-06T22:16:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.177+0000 7fd0f171ab80 1 Processor -- start 2023-10-06T22:16:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.177+0000 7fd0f171ab80 1 -- start start 2023-10-06T22:16:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 0x562a84ceff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84cf8890 0x562a84cf4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84ac1920 con 0x562a84ced0c0 2023-10-06T22:16:21.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84ce0ae0 con 0x562a84cf8890 2023-10-06T22:16:21.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84cf04a0 con 0x562a84ced9e0 2023-10-06T22:16:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0d7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84cf8890 0x562a84cf4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0d7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84cf8890 0x562a84cf4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55418/0 (socket says 172.21.15.57:55418) 2023-10-06T22:16:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0d7fff700 1 -- 172.21.15.57:0/2213155148 learned_addr learned my addr 172.21.15.57:0/2213155148 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.178+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0d7fff700 1 -- 172.21.15.57:0/2213155148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 msgr2=0x562a84cf5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0d7fff700 1 -- 172.21.15.57:0/2213155148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 msgr2=0x562a84ceff60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:16:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 0x562a84ceff60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0d7fff700 1 -- 172.21.15.57:0/2213155148 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a84a92250 con 0x562a84cf8890 2023-10-06T22:16:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84cf8890 0x562a84cf4300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0017000 tx=0x7fd0d0099ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614aa3c851645bda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0be7fc700 1 -- 172.21.15.57:0/2213155148 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d009a320 con 0x562a84cf8890 2023-10-06T22:16:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0be7fc700 1 -- 172.21.15.57:0/2213155148 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0d0099c20 con 0x562a84cf8890 2023-10-06T22:16:21.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.179+0000 7fd0be7fc700 1 -- 172.21.15.57:0/2213155148 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00046d0 con 0x562a84cf8890 2023-10-06T22:16:21.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2213155148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84cf8890 msgr2=0x562a84cf4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84cf8890 0x562a84cf4300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0017000 tx=0x7fd0d0099ae0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2213155148 shutdown_connections 2023-10-06T22:16:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84cf8890 0x562a84cf4300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 0x562a84ceff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/2213155148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84ced9e0 0x562a84cf5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2213155148 >> 172.21.15.57:0/2213155148 conn(0x562a84c62260 msgr2=0x562a84cf3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2213155148 shutdown_connections 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.180+0000 7fd0f171ab80 1 -- 172.21.15.57:0/2213155148 wait complete. 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.181+0000 7fd0f171ab80 1 -- start start 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.181+0000 7fd0f171ab80 1 Processor -- start 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 0x562a84cfd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cfd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0f171ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84cf8890 0x562a84d6c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84d6c900 con 0x562a84ced0c0 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84d6ca40 con 0x562a84ced9e0 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0f171ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a84d6cb80 con 0x562a84cf8890 2023-10-06T22:16:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0f171ab80 10 client.0 _ll_register_callbacks cb 0x7fd0c800f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84cf8890 0x562a84d6c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84cf8890 0x562a84d6c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48940/0 (socket says 172.21.15.57:48940) 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 -- 172.21.15.57:0/3944515446 learned_addr learned my addr 172.21.15.57:0/3944515446 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d6ffd700 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cfd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 -- 172.21.15.57:0/3944515446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 msgr2=0x562a84cfd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cfd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 -- 172.21.15.57:0/3944515446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 msgr2=0x562a84cfd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 0x562a84cfd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.182+0000 7fd0d7fff700 1 -- 172.21.15.57:0/3944515446 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a84a6bde0 con 0x562a84cf8890 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0d7fff700 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84cf8890 0x562a84d6c3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00a19f0 tx=0x7fd0d000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d6f0ce595dbacb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3944515446 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00048d0 con 0x562a84cf8890 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3944515446 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd0d00a1f20 con 0x562a84cf8890 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0cc00ba20 con 0x562a84cf8890 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3944515446 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0d000cbb0 con 0x562a84cf8890 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0f171ab80 10 client.26095 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fd0cc009520 con 0x562a84cf8890 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.183+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3944515446 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0cc00ba20 con 0x562a84cf8890 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3944515446 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd0d0099110 con 0x562a84cf8890 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0f171ab80 10 client.26095 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0f171ab80 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0f171ab80 20 client.26095 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d6e040 0x562a84d70460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0f171ab80 4 client.26095 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a84c39b30 con 0x562a84d6e040 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d6e040 0x562a84d70460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0d77fe700 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d6e040 0x562a84d70460 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea081fb24e9501a5 server_cookie=af5f71a688c1ceb5 in_seq=0 out_seq=0 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.184+0000 7fd0b6ffd700 10 client.26095 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0b6ffd700 1 -- 172.21.15.57:0/3944515446 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a84c39b30 con 0x562a84d6e040 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0b6ffd700 10 client.26095 handle_command_reply: tid=0 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 client.26095 shutdown 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84cf8890 msgr2=0x562a84d6c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84cf8890 0x562a84d6c3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00a19f0 tx=0x7fd0d000c7a0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 shutdown_connections 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562a84d6e040 0x562a84d70460 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a84cf8890 0x562a84d6c3c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a84ced0c0 0x562a84cfd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 --2- 172.21.15.57:0/3944515446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a84ced9e0 0x562a84cfd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 >> 172.21.15.57:0/3944515446 conn(0x562a84c62260 msgr2=0x562a84cf5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0b6ffd700 0 client.26095 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 shutdown_connections 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 1 -- 172.21.15.57:0/3944515446 wait complete. 2023-10-06T22:16:21.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.185+0000 7fd0f171ab80 20 client.26095 trim_cache size 0 max 16384 2023-10-06T22:16:21.195 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:21.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd0c00a1670 msgr2=0x7fd0c00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd0c00a1670 0x7fd0c00a3b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8012840 tx=0x7fd0c8009310 comp rx=0 tx=0).stop 2023-10-06T22:16:21.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8111e50 msgr2=0x7fd0d81a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8111e50 0x7fd0d81a1fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd0d806a470 tx=0x7fd0d000c3a0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 shutdown_connections 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd0c00a1670 0x7fd0c00a3b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d8111e50 0x7fd0d81a1fb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0d8073250 0x7fd0d819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 --2- 172.21.15.57:0/1281099410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd0d8073b70 0x7fd0d819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.186+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 >> 172.21.15.57:0/1281099410 conn(0x7fd0d80fc950 msgr2=0x7fd0d80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.187+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 shutdown_connections 2023-10-06T22:16:21.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:21.187+0000 7fd0dddff700 1 -- 172.21.15.57:0/1281099410 wait complete. 2023-10-06T22:16:21.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:20 smithi156 ceph-mon[134358]: pgmap v51: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 983 KiB/s wr, 48 op/s 2023-10-06T22:16:21.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:20 smithi190 ceph-mon[134891]: pgmap v51: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 983 KiB/s wr, 48 op/s 2023-10-06T22:16:22.022 INFO:tasks.workunit.client.0.smithi057.stdout: 15 179350 755 90083 707 48177 1065 2023-10-06T22:16:22.210 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T22:16:22.211 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T22:16:22.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 -- 172.21.15.57:0/3637307132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0102910 msgr2=0x7fccd010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 --2- 172.21.15.57:0/3637307132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0102910 0x7fccd010adf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098f0 tx=0x7fccc4009c00 comp rx=0 tx=0).stop 2023-10-06T22:16:22.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 -- 172.21.15.57:0/3637307132 shutdown_connections 2023-10-06T22:16:22.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 --2- 172.21.15.57:0/3637307132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd010b410 0x7fccd010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 --2- 172.21.15.57:0/3637307132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccd0101ff0 0x7fccd01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 --2- 172.21.15.57:0/3637307132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0102910 0x7fccd010adf0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098f0 tx=0x7fccc4009c00 comp rx=0 tx=0).stop 2023-10-06T22:16:22.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 -- 172.21.15.57:0/3637307132 >> 172.21.15.57:0/3637307132 conn(0x7fccd00fb830 msgr2=0x7fccd00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 -- 172.21.15.57:0/3637307132 shutdown_connections 2023-10-06T22:16:22.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.537+0000 7fccd67ff700 1 -- 172.21.15.57:0/3637307132 wait complete. 2023-10-06T22:16:22.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.538+0000 7fccd67ff700 1 Processor -- start 2023-10-06T22:16:22.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.538+0000 7fccd67ff700 1 -- start start 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fccd67ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0101ff0 0x7fccd01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fccd67ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccd010b410 0x7fccd01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fccd67ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd019bc40 0x7fccd019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fccd67ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd019c6c0 con 0x7fccd010b410 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fccd67ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd019c830 con 0x7fccd0101ff0 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fccd67ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd019c9f0 con 0x7fccd019bc40 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0101ff0 0x7fccd01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0101ff0 0x7fccd01972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34180/0 (socket says 172.21.15.57:34180) 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 -- 172.21.15.57:0/183596345 learned_addr learned my addr 172.21.15.57:0/183596345 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fccd4d9c700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd019bc40 0x7fccd019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccd010b410 0x7fccd01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 -- 172.21.15.57:0/183596345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd019bc40 msgr2=0x7fccd019c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd019bc40 0x7fccd019c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 -- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccd010b410 msgr2=0x7fccd01977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccd010b410 0x7fccd01977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccffff700 1 -- 172.21.15.57:0/183596345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8009c40 con 0x7fccd0101ff0 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.539+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccd010b410 0x7fccd01977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:16:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.540+0000 7fcccffff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0101ff0 0x7fccd01972a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e9c0 tx=0x7fccc000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.540+0000 7fcccd7fa700 1 -- 172.21.15.57:0/183596345 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000cbd0 con 0x7fccd0101ff0 2023-10-06T22:16:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.540+0000 7fcccd7fa700 1 -- 172.21.15.57:0/183596345 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc000cd30 con 0x7fccd0101ff0 2023-10-06T22:16:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.540+0000 7fcccd7fa700 1 -- 172.21.15.57:0/183596345 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0005580 con 0x7fccd0101ff0 2023-10-06T22:16:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.540+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc4009520 con 0x7fccd0101ff0 2023-10-06T22:16:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.540+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd00fff50 con 0x7fccd0101ff0 2023-10-06T22:16:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.544+0000 7fcccd7fa700 1 -- 172.21.15.57:0/183596345 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fccc0004750 con 0x7fccd0101ff0 2023-10-06T22:16:22.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.546+0000 7fcccd7fa700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fccb80a5a80 0x7fccb80a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.547+0000 7fcccd7fa700 1 -- 172.21.15.57:0/183596345 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fccc0014070 con 0x7fccd0101ff0 2023-10-06T22:16:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.547+0000 7fccea09cb80 1 Processor -- start 2023-10-06T22:16:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccea09cb80 1 -- start start 2023-10-06T22:16:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fccb80a5a80 0x7fccb80a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae5ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccea09cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491adb64e0 con 0x56491ae4da60 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccea09cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ada7ca0 con 0x56491ae4d140 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccea09cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ada55c0 con 0x56491ae58910 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fcccffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae4d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fcccffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae4d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34188/0 (socket says 172.21.15.57:34188) 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fcccffff700 1 -- 172.21.15.57:0/3247000155 learned_addr learned my addr 172.21.15.57:0/3247000155 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.548+0000 7fccd4d9c700 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae5ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fcccffff700 1 -- 172.21.15.57:0/3247000155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 msgr2=0x56491ae5ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fcccffff700 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae5ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fcccffff700 1 -- 172.21.15.57:0/3247000155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 msgr2=0x56491ae583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fcccffff700 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fcccffff700 1 -- 172.21.15.57:0/3247000155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56491abcbe60 con 0x56491ae4d140 2023-10-06T22:16:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fcccffff700 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae4d520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccc0096d90 tx=0x7fccc00097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb7ff80c8f2b5b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fccb67fc700 1 -- 172.21.15.57:0/3247000155 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc009e470 con 0x56491ae4d140 2023-10-06T22:16:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fccb67fc700 1 -- 172.21.15.57:0/3247000155 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc00a26b0 con 0x56491ae4d140 2023-10-06T22:16:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.549+0000 7fccb67fc700 1 -- 172.21.15.57:0/3247000155 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0016330 con 0x56491ae4d140 2023-10-06T22:16:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.550+0000 7fccea09cb80 1 -- 172.21.15.57:0/3247000155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 msgr2=0x56491ae4d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.550+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae4d520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccc0096d90 tx=0x7fccc00097d0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fccb80a5a80 0x7fccb80a7f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccd01988b0 tx=0x7fccc400f5f0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fccea09cb80 1 -- 172.21.15.57:0/3247000155 shutdown_connections 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae5ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae4d520 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3247000155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fccea09cb80 1 -- 172.21.15.57:0/3247000155 >> 172.21.15.57:0/3247000155 conn(0x56491adc22e0 msgr2=0x56491ae494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fccea09cb80 1 -- 172.21.15.57:0/3247000155 shutdown_connections 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.551+0000 7fccea09cb80 1 -- 172.21.15.57:0/3247000155 wait complete. 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.552+0000 7fccea09cb80 1 -- start start 2023-10-06T22:16:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.552+0000 7fccea09cb80 1 Processor -- start 2023-10-06T22:16:22.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4d140 0x56491ae58390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4da60 0x56491ae55610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae55d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccea09cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ada0eb0 con 0x56491ae4d140 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccea09cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ad3d790 con 0x56491ae4da60 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccea09cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ad3d430 con 0x56491ae58910 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccea09cb80 10 client.0 _ll_register_callbacks cb 0x56491ab9a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccd4d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae55d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccd4d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae55d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48518/0 (socket says 172.21.15.57:48518) 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fccd4d9c700 1 -- 172.21.15.57:0/3989881922 learned_addr learned my addr 172.21.15.57:0/3989881922 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fcccffff700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4d140 0x56491ae58390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.553+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4da60 0x56491ae55610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccd4d9c700 1 -- 172.21.15.57:0/3989881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4da60 msgr2=0x56491ae55610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccd4d9c700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4da60 0x56491ae55610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccd4d9c700 1 -- 172.21.15.57:0/3989881922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4d140 msgr2=0x56491ae58390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccd4d9c700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4d140 0x56491ae58390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccd4d9c700 1 -- 172.21.15.57:0/3989881922 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd00fff50 con 0x56491ae58910 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fcccffff700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4d140 0x56491ae58390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccd4d9c700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae55d00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fccc8007f70 tx=0x7fccc8009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e90915db02e34b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccaf7fe700 1 -- 172.21.15.57:0/3989881922 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800e050 con 0x56491ae58910 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56491ae40f90 con 0x56491ae58910 2023-10-06T22:16:22.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.554+0000 7fccaf7fe700 1 -- 172.21.15.57:0/3989881922 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fccc8013ad0 con 0x56491ae58910 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.555+0000 7fccaf7fe700 1 -- 172.21.15.57:0/3989881922 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc8013c90 con 0x56491ae58910 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.555+0000 7fccea09cb80 10 client.26107 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.555+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56491abb4ce0 con 0x56491ae58910 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.555+0000 7fccaf7fe700 1 -- 172.21.15.57:0/3989881922 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56491ae40f90 con 0x56491ae58910 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.555+0000 7fccaf7fe700 1 -- 172.21.15.57:0/3989881922 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc8020450 con 0x56491ae58910 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.555+0000 7fccea09cb80 10 client.26107 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.555+0000 7fccea09cb80 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.556+0000 7fccea09cb80 20 client.26107 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.556+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491ae68470 0x56491ae6a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.556+0000 7fccea09cb80 4 client.26107 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.556+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56491ad99e90 con 0x56491ae68470 2023-10-06T22:16:22.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.556+0000 7fcccffff700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491ae68470 0x56491ae6a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.557+0000 7fcccffff700 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491ae68470 0x56491ae6a890 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a342f9a9f720ddcc server_cookie=71c9ec6b385ad4e4 in_seq=0 out_seq=0 2023-10-06T22:16:22.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.557+0000 7fccaf7fe700 10 client.26107 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.557+0000 7fccaf7fe700 1 -- 172.21.15.57:0/3989881922 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56491ad99e90 con 0x56491ae68470 2023-10-06T22:16:22.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.557+0000 7fccaf7fe700 10 client.26107 handle_command_reply: tid=0 2023-10-06T22:16:22.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.557+0000 7fccea09cb80 1 client.26107 shutdown 2023-10-06T22:16:22.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 msgr2=0x56491ae55d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae55d00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fccc8007f70 tx=0x7fccc8009330 comp rx=0 tx=0).stop 2023-10-06T22:16:22.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 shutdown_connections 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491ae68470 0x56491ae6a890 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae55d00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4d140 0x56491ae58390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3989881922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4da60 0x56491ae55610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 >> 172.21.15.57:0/3989881922 conn(0x56491adc22e0 msgr2=0x56491ae49060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccaf7fe700 0 client.26107 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 shutdown_connections 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.558+0000 7fccea09cb80 1 -- 172.21.15.57:0/3989881922 wait complete. 2023-10-06T22:16:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.559+0000 7fccea09cb80 20 client.26107 trim_cache size 0 max 16384 2023-10-06T22:16:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.585+0000 7fccea09cb80 1 Processor -- start 2023-10-06T22:16:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fccea09cb80 1 -- start start 2023-10-06T22:16:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae54dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae528a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae52de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fccea09cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ac219a0 con 0x56491ae4da60 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fccea09cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ae53ba0 con 0x56491ae4d140 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fccea09cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491ae51480 con 0x56491ae58910 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fcccffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae54dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.587+0000 7fcccffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae54dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34214/0 (socket says 172.21.15.57:34214) 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fcccffff700 1 -- 172.21.15.57:0/3991889925 learned_addr learned my addr 172.21.15.57:0/3991889925 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fcccffff700 1 -- 172.21.15.57:0/3991889925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 msgr2=0x56491ae52de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fcccffff700 1 --2- 172.21.15.57:0/3991889925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae52de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fcccffff700 1 -- 172.21.15.57:0/3991889925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 msgr2=0x56491ae528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fcccffff700 1 --2- 172.21.15.57:0/3991889925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fcccffff700 1 -- 172.21.15.57:0/3991889925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56491abf22d0 con 0x56491ae4d140 2023-10-06T22:16:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fcccffff700 1 --2- 172.21.15.57:0/3991889925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae54dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccc009a4e0 tx=0x7fccc0009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac4a0ba5b184fb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fccb6ffd700 1 -- 172.21.15.57:0/3991889925 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00986c0 con 0x56491ae4d140 2023-10-06T22:16:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fccb6ffd700 1 -- 172.21.15.57:0/3991889925 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc009e170 con 0x56491ae4d140 2023-10-06T22:16:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.588+0000 7fccb6ffd700 1 -- 172.21.15.57:0/3991889925 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0016590 con 0x56491ae4d140 2023-10-06T22:16:22.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 -- 172.21.15.57:0/3991889925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 msgr2=0x56491ae54dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3991889925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae54dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccc009a4e0 tx=0x7fccc0009f80 comp rx=0 tx=0).stop 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 -- 172.21.15.57:0/3991889925 shutdown_connections 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3991889925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae58910 0x56491ae52de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3991889925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae4d140 0x56491ae54dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 --2- 172.21.15.57:0/3991889925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 -- 172.21.15.57:0/3991889925 >> 172.21.15.57:0/3991889925 conn(0x56491adc22e0 msgr2=0x56491ae50970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 -- 172.21.15.57:0/3991889925 shutdown_connections 2023-10-06T22:16:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.589+0000 7fccea09cb80 1 -- 172.21.15.57:0/3991889925 wait complete. 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.590+0000 7fccea09cb80 1 -- start start 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.590+0000 7fccea09cb80 1 Processor -- start 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae4d140 0x56491ae5da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae5dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccea09cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae58910 0x56491ae5e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccea09cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491aec4ac0 con 0x56491ae4da60 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccea09cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491aec4c00 con 0x56491ae58910 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccea09cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491aec4d40 con 0x56491ae4d140 2023-10-06T22:16:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccea09cb80 10 client.0 _ll_register_callbacks cb 0x7fccc8007b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.591+0000 7fccd4d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae58910 0x56491ae5e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fccd4d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae58910 0x56491ae5e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34218/0 (socket says 172.21.15.57:34218) 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fccd4d9c700 1 -- 172.21.15.57:0/1936792846 learned_addr learned my addr 172.21.15.57:0/1936792846 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccffff700 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae4d140 0x56491ae5da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae5dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccffff700 1 -- 172.21.15.57:0/1936792846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae58910 msgr2=0x56491ae5e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccffff700 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae58910 0x56491ae5e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccffff700 1 -- 172.21.15.57:0/1936792846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 msgr2=0x56491ae5dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccffff700 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae5dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccffff700 1 -- 172.21.15.57:0/1936792846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc00b6040 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fcccffff700 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae4d140 0x56491ae5da60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fccc0096820 tx=0x7fccc0005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc5365f59d55ad1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fccaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fccaf7fe700 1 -- 172.21.15.57:0/1936792846 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0096a30 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fccaf7fe700 1 -- 172.21.15.57:0/1936792846 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fccc00986c0 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.592+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc00a26b0 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccaf7fe700 1 -- 172.21.15.57:0/1936792846 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc0016330 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 10 client.26110 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56491abca330 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccaf7fe700 1 -- 172.21.15.57:0/1936792846 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc00a26b0 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccaf7fe700 1 -- 172.21.15.57:0/1936792846 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc00998a0 con 0x56491ae4d140 2023-10-06T22:16:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 10 client.26110 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 10 client.26110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 20 client.26110 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491aec6010 0x56491aec8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 4 client.26110 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T22:16:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56491ad99bb0 con 0x56491aec6010 2023-10-06T22:16:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.593+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491aec6010 0x56491aec8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.594+0000 7fcccf7fe700 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491aec6010 0x56491aec8420 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fce07b43fa2675 server_cookie=c74434760092e8ff in_seq=0 out_seq=0 2023-10-06T22:16:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.594+0000 7fccaf7fe700 10 client.26110 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.595+0000 7fccaf7fe700 1 -- 172.21.15.57:0/1936792846 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56491ad99bb0 con 0x56491aec6010 2023-10-06T22:16:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.595+0000 7fccaf7fe700 10 client.26110 handle_command_reply: tid=0 2023-10-06T22:16:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.595+0000 7fccea09cb80 1 client.26110 shutdown 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.595+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae4d140 msgr2=0x56491ae5da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.595+0000 7fccea09cb80 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae4d140 0x56491ae5da60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fccc0096820 tx=0x7fccc0005910 comp rx=0 tx=0).stop 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 shutdown_connections 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56491aec6010 0x56491aec8420 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56491ae58910 0x56491ae5e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56491ae4d140 0x56491ae5da60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 --2- 172.21.15.57:0/1936792846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56491ae4da60 0x56491ae5dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 >> 172.21.15.57:0/1936792846 conn(0x56491adc22e0 msgr2=0x56491ae4f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccaf7fe700 0 client.26110 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 shutdown_connections 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.598+0000 7fccea09cb80 1 -- 172.21.15.57:0/1936792846 wait complete. 2023-10-06T22:16:22.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.599+0000 7fccea09cb80 20 client.26110 trim_cache size 0 max 16384 2023-10-06T22:16:22.603 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:16:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-06T22:16:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "c61e70d9-5dde-4abe-a4bb-9fadc49ba031", 2023-10-06T22:16:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-06T22:16:22.603 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:16:22.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.602+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fccb80a5a80 msgr2=0x7fccb80a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.602+0000 7fccd67ff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fccb80a5a80 0x7fccb80a7f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccd01988b0 tx=0x7fccc400f5f0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.602+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0101ff0 msgr2=0x7fccd01972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.602+0000 7fccd67ff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0101ff0 0x7fccd01972a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e9c0 tx=0x7fccc000ed80 comp rx=0 tx=0).stop 2023-10-06T22:16:22.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 shutdown_connections 2023-10-06T22:16:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fccb80a5a80 0x7fccb80a7f40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd019bc40 0x7fccd019c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fccd0101ff0 0x7fccd01972a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 --2- 172.21.15.57:0/183596345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccd010b410 0x7fccd01977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 >> 172.21.15.57:0/183596345 conn(0x7fccd00fb830 msgr2=0x7fccd00fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 shutdown_connections 2023-10-06T22:16:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.603+0000 7fccd67ff700 1 -- 172.21.15.57:0/183596345 wait complete. 2023-10-06T22:16:22.630 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:16:22.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.913+0000 7fb5b3fff700 1 -- 172.21.15.57:0/545230943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac075d00 msgr2=0x7fb5ac1117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.913+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/545230943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac075d00 0x7fb5ac1117d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0098f0 tx=0x7fb59c009c00 comp rx=0 tx=0).stop 2023-10-06T22:16:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.914+0000 7fb5b3fff700 1 -- 172.21.15.57:0/545230943 shutdown_connections 2023-10-06T22:16:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.914+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/545230943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac111d10 0x7fb5ac114100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.914+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/545230943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5ac0753e0 0x7fb5ac0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.914+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/545230943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac075d00 0x7fb5ac1117d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.914+0000 7fb5b3fff700 1 -- 172.21.15.57:0/545230943 >> 172.21.15.57:0/545230943 conn(0x7fb5ac0feb60 msgr2=0x7fb5ac100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.914+0000 7fb5b3fff700 1 -- 172.21.15.57:0/545230943 shutdown_connections 2023-10-06T22:16:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.914+0000 7fb5b3fff700 1 -- 172.21.15.57:0/545230943 wait complete. 2023-10-06T22:16:22.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.915+0000 7fb5b3fff700 1 Processor -- start 2023-10-06T22:16:22.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.915+0000 7fb5b3fff700 1 -- start start 2023-10-06T22:16:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac0753e0 0x7fb5ac10cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac075d00 0x7fb5ac10d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5ac111d10 0x7fb5ac1c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b3fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac1c30e0 con 0x7fb5ac0753e0 2023-10-06T22:16:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b3fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac1c3220 con 0x7fb5ac075d00 2023-10-06T22:16:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b3fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac1c33e0 con 0x7fb5ac111d10 2023-10-06T22:16:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac0753e0 0x7fb5ac10cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac0753e0 0x7fb5ac10cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52570/0 (socket says 172.21.15.57:52570) 2023-10-06T22:16:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/95688123 learned_addr learned my addr 172.21.15.57:0/95688123 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.916+0000 7fb5b259c700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5ac111d10 0x7fb5ac1c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.917+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/95688123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5ac111d10 msgr2=0x7fb5ac1c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.917+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5ac111d10 0x7fb5ac1c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.917+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/95688123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac075d00 msgr2=0x7fb5ac10d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.917+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac075d00 0x7fb5ac10d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.917+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/95688123 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c009520 con 0x7fb5ac0753e0 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.917+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac0753e0 0x7fb5ac10cfd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800c920 tx=0x7fb5a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.917+0000 7fb5a2ffd700 1 -- 172.21.15.57:0/95688123 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8007b00 con 0x7fb5ac0753e0 2023-10-06T22:16:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.918+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac1c36c0 con 0x7fb5ac0753e0 2023-10-06T22:16:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.918+0000 7fb5a2ffd700 1 -- 172.21.15.57:0/95688123 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a800ce10 con 0x7fb5ac0753e0 2023-10-06T22:16:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.918+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac1c3c20 con 0x7fb5ac0753e0 2023-10-06T22:16:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.918+0000 7fb5a2ffd700 1 -- 172.21.15.57:0/95688123 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a800e3b0 con 0x7fb5ac0753e0 2023-10-06T22:16:22.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:22 smithi057 ceph-mon[98492]: pgmap v52: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 983 KiB/s wr, 48 op/s 2023-10-06T22:16:22.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:22 smithi057 ceph-mon[98492]: scrub queued for path: / 2023-10-06T22:16:22.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:22 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:22.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:22 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:16:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.921+0000 7fb5a2ffd700 1 -- 172.21.15.57:0/95688123 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb5a800f450 con 0x7fb5ac0753e0 2023-10-06T22:16:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.923+0000 7fb5a2ffd700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5980a1630 0x7fb5980a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.923+0000 7fb5a2ffd700 1 -- 172.21.15.57:0/95688123 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb5a80cc500 con 0x7fb5ac0753e0 2023-10-06T22:16:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.923+0000 7fb5b159a700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5980a1630 0x7fb5980a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.924+0000 7fb5b159a700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5980a1630 0x7fb5980a3af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009810 tx=0x7fb59c0058e0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 Processor -- start 2023-10-06T22:16:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 -- start start 2023-10-06T22:16:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd6f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f9e0 0x558dadd7a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558dadd7cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadcd8460 con 0x558dadd6f0c0 2023-10-06T22:16:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadcc9c20 con 0x558dadd7a890 2023-10-06T22:16:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.925+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadcc7540 con 0x558dadd6f9e0 2023-10-06T22:16:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd6f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd6f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52578/0 (socket says 172.21.15.57:52578) 2023-10-06T22:16:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/491949887 learned_addr learned my addr 172.21.15.57:0/491949887 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b259c700 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558dadd7cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b159a700 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f9e0 0x558dadd7a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/491949887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f9e0 msgr2=0x558dadd7a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f9e0 0x558dadd7a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/491949887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 msgr2=0x558dadd7cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558dadd7cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/491949887 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dadaedde0 con 0x558dadd6f0c0 2023-10-06T22:16:22.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.926+0000 7fb5b259c700 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558dadd7cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:22.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.927+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd6f4a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8095a30 tx=0x7fb5a80991a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd78338fe724fae0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.927+0000 7fb597fff700 1 -- 172.21.15.57:0/491949887 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a809d680 con 0x558dadd6f0c0 2023-10-06T22:16:22.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.927+0000 7fb597fff700 1 -- 172.21.15.57:0/491949887 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a8095040 con 0x558dadd6f0c0 2023-10-06T22:16:22.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.927+0000 7fb597fff700 1 -- 172.21.15.57:0/491949887 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80999f0 con 0x558dadd6f0c0 2023-10-06T22:16:22.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 -- 172.21.15.57:0/491949887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 msgr2=0x558dadd6f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd6f4a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8095a30 tx=0x7fb5a80991a0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 -- 172.21.15.57:0/491949887 shutdown_connections 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558dadd7cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd6f4a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/491949887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f9e0 0x558dadd7a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 -- 172.21.15.57:0/491949887 >> 172.21.15.57:0/491949887 conn(0x558dadce4260 msgr2=0x558dadd6b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.928+0000 7fb5c7949b80 1 -- 172.21.15.57:0/491949887 shutdown_connections 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.929+0000 7fb5c7949b80 1 -- 172.21.15.57:0/491949887 wait complete. 2023-10-06T22:16:22.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.929+0000 7fb5c7949b80 1 -- start start 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.930+0000 7fb5c7949b80 1 Processor -- start 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.930+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd77620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.930+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd77b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.930+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd7a890 0x558dadd780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.930+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadcc2e30 con 0x558dadd6f0c0 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.930+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadc5f710 con 0x558dadd6f9e0 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.930+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadc5f3b0 con 0x558dadd7a890 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5c7949b80 10 client.0 _ll_register_callbacks cb 0x558dadabc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd7a890 0x558dadd780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd7a890 0x558dadd780a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48574/0 (socket says 172.21.15.57:48574) 2023-10-06T22:16:22.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 -- 172.21.15.57:0/198467397 learned_addr learned my addr 172.21.15.57:0/198467397 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b159a700 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd77b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 -- 172.21.15.57:0/198467397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 msgr2=0x558dadd77b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd77b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 -- 172.21.15.57:0/198467397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 msgr2=0x558dadd77620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd77620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 -- 172.21.15.57:0/198467397 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a4007550 con 0x558dadd7a890 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b159a700 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd77b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.931+0000 7fb5b259c700 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd7a890 0x558dadd780a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4007d40 tx=0x7fb5a40198c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892304bc8f8e2c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.932+0000 7fb5957fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.932+0000 7fb5957fa700 1 -- 172.21.15.57:0/198467397 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400c070 con 0x558dadd7a890 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.932+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dadd62f10 con 0x558dadd7a890 2023-10-06T22:16:22.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.932+0000 7fb5957fa700 1 -- 172.21.15.57:0/198467397 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb5a4013440 con 0x558dadd7a890 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.932+0000 7fb5957fa700 1 -- 172.21.15.57:0/198467397 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a4013600 con 0x558dadd7a890 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.933+0000 7fb5c7949b80 10 client.26119 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.933+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb5ac1c3c20 con 0x558dadd7a890 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5957fa700 1 -- 172.21.15.57:0/198467397 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558dadd62f10 con 0x558dadd7a890 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5957fa700 1 -- 172.21.15.57:0/198467397 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb5a4027930 con 0x558dadd7a890 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5c7949b80 10 client.26119 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5c7949b80 10 client.26119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5c7949b80 20 client.26119 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadd8a470 0x558dadd8c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5c7949b80 4 client.26119 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:22.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dadcbbe10 con 0x558dadd8a470 2023-10-06T22:16:22.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.935+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadd8a470 0x558dadd8c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.936+0000 7fb5957fa700 10 client.26119 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.937+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadd8a470 0x558dadd8c890 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e65ea10d077a78 server_cookie=a86f5b42abe622af in_seq=0 out_seq=0 2023-10-06T22:16:22.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.937+0000 7fb5957fa700 1 -- 172.21.15.57:0/198467397 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558dadcbbe10 con 0x558dadd8a470 2023-10-06T22:16:22.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.937+0000 7fb5957fa700 10 client.26119 handle_command_reply: tid=0 2023-10-06T22:16:22.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.938+0000 7fb5c7949b80 1 client.26119 shutdown 2023-10-06T22:16:22.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd7a890 msgr2=0x558dadd780a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd7a890 0x558dadd780a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4007d40 tx=0x7fb5a40198c0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 shutdown_connections 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadd8a470 0x558dadd8c890 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd7a890 0x558dadd780a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5957fa700 0 client.26119 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f0c0 0x558dadd77620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/198467397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd77b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.940+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 >> 172.21.15.57:0/198467397 conn(0x558dadce4260 msgr2=0x558dadd6afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.941+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 shutdown_connections 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.941+0000 7fb5c7949b80 1 -- 172.21.15.57:0/198467397 wait complete. 2023-10-06T22:16:22.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.942+0000 7fb5c7949b80 20 client.26119 trim_cache size 0 max 16384 2023-10-06T22:16:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.961+0000 7fb5c7949b80 1 Processor -- start 2023-10-06T22:16:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.961+0000 7fb5c7949b80 1 -- start start 2023-10-06T22:16:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.961+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558dadd782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd81410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd7a890 0x558dadd74890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadb43920 con 0x558dadd7a890 2023-10-06T22:16:22.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadd87ad0 con 0x558dadd6f9e0 2023-10-06T22:16:22.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dadd86dc0 con 0x558dadd6f0c0 2023-10-06T22:16:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd81410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd81410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34258/0 (socket says 172.21.15.57:34258) 2023-10-06T22:16:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 -- 172.21.15.57:0/4030655152 learned_addr learned my addr 172.21.15.57:0/4030655152 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558dadd782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b259c700 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd7a890 0x558dadd74890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 -- 172.21.15.57:0/4030655152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 msgr2=0x558dadd782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558dadd782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 -- 172.21.15.57:0/4030655152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd7a890 msgr2=0x558dadd74890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd7a890 0x558dadd74890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b159a700 1 -- 172.21.15.57:0/4030655152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dadad6c60 con 0x558dadd6f9e0 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558dadd782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.962+0000 7fb5b259c700 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd7a890 0x558dadd74890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.963+0000 7fb5b159a700 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd81410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0098f0 tx=0x7fb59c004000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0880ed00724db4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.963+0000 7fb5a0ff9700 1 -- 172.21.15.57:0/4030655152 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c029040 con 0x558dadd6f9e0 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.963+0000 7fb5a0ff9700 1 -- 172.21.15.57:0/4030655152 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb59c005040 con 0x558dadd6f9e0 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.963+0000 7fb5a0ff9700 1 -- 172.21.15.57:0/4030655152 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c01c070 con 0x558dadd6f9e0 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 -- 172.21.15.57:0/4030655152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 msgr2=0x558dadd81410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd81410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0098f0 tx=0x7fb59c004000 comp rx=0 tx=0).stop 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 -- 172.21.15.57:0/4030655152 shutdown_connections 2023-10-06T22:16:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd7a890 0x558dadd74890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558dadd782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/4030655152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd6f9e0 0x558dadd81410 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 -- 172.21.15.57:0/4030655152 >> 172.21.15.57:0/4030655152 conn(0x558dadce4260 msgr2=0x558dadd77b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 -- 172.21.15.57:0/4030655152 shutdown_connections 2023-10-06T22:16:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.964+0000 7fb5c7949b80 1 -- 172.21.15.57:0/4030655152 wait complete. 2023-10-06T22:16:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 -- start start 2023-10-06T22:16:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 Processor -- start 2023-10-06T22:16:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558daddd2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f9e0 0x558daddd2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558daddd6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daddd6db0 con 0x558dadd6f9e0 2023-10-06T22:16:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daddd6f20 con 0x558dadd7a890 2023-10-06T22:16:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558daddd70e0 con 0x558dadd6f0c0 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.965+0000 7fb5c7949b80 10 client.0 _ll_register_callbacks cb 0x7fb5a40076e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558daddd6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558daddd2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b159a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f9e0 0x558daddd2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558daddd2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48596/0 (socket says 172.21.15.57:48596) 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/3796517495 learned_addr learned my addr 172.21.15.57:0/3796517495 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558daddd6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34264/0 (socket says 172.21.15.57:34264) 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/3796517495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 msgr2=0x558daddd6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558daddd6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/3796517495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f9e0 msgr2=0x558daddd2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f9e0 0x558daddd2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 -- 172.21.15.57:0/3796517495 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dadaedde0 con 0x558dadd6f0c0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b259c700 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558daddd6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b159a700 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f9e0 0x558daddd2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5b1d9b700 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558daddd2440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a8099300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c9d5f22ad3fb1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.966+0000 7fb5957fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5957fa700 1 -- 172.21.15.57:0/3796517495 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a809d680 con 0x558dadd6f0c0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5a8095040 con 0x558dadd6f0c0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5957fa700 1 -- 172.21.15.57:0/3796517495 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb5a8007c60 con 0x558dadd6f0c0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5957fa700 1 -- 172.21.15.57:0/3796517495 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a8007e20 con 0x558dadd6f0c0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 10 client.26125 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb5ac114bf0 con 0x558dadd6f0c0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5957fa700 1 -- 172.21.15.57:0/3796517495 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a8095040 con 0x558dadd6f0c0 2023-10-06T22:16:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5957fa700 1 -- 172.21.15.57:0/3796517495 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb5a8060030 con 0x558dadd6f0c0 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 10 client.26125 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 20 client.26125 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadddf5b0 0x558dadde19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 4 client.26125 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.967+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558dadcbbb30 con 0x558dadddf5b0 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.968+0000 7fb5b159a700 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadddf5b0 0x558dadde19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.968+0000 7fb5957fa700 10 client.26125 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.968+0000 7fb5b159a700 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadddf5b0 0x558dadde19d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1360a3130d623205 server_cookie=466273ddb9c12552 in_seq=0 out_seq=0 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.968+0000 7fb5957fa700 1 -- 172.21.15.57:0/3796517495 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558dadcbbb30 con 0x558dadddf5b0 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.968+0000 7fb5957fa700 10 client.26125 handle_command_reply: tid=0 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 client.26125 shutdown 2023-10-06T22:16:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 msgr2=0x558daddd2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558daddd2440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a8099300 comp rx=0 tx=0).stop 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 shutdown_connections 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558dadddf5b0 0x558dadde19d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558dadd7a890 0x558daddd6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558dadd6f0c0 0x558daddd2440 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 --2- 172.21.15.57:0/3796517495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558dadd6f9e0 0x558daddd2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5957fa700 0 client.26125 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 >> 172.21.15.57:0/3796517495 conn(0x558dadce4260 msgr2=0x558dadd71120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 shutdown_connections 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 1 -- 172.21.15.57:0/3796517495 wait complete. 2023-10-06T22:16:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.969+0000 7fb5c7949b80 20 client.26125 trim_cache size 0 max 16384 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (0 inodes in the stack)", 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout: "c61e70d9-5dde-4abe-a4bb-9fadc49ba031": { 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "c61e70d9-5dde-4abe-a4bb-9fadc49ba031", 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:16:22.977 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:16:22.978 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:16:22.978 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:16:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5980a1630 msgr2=0x7fb5980a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5980a1630 0x7fb5980a3af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009810 tx=0x7fb59c0058e0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac0753e0 msgr2=0x7fb5ac10cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac0753e0 0x7fb5ac10cfd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800c920 tx=0x7fb5a800cc30 comp rx=0 tx=0).stop 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 shutdown_connections 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb5980a1630 0x7fb5980a3af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5ac111d10 0x7fb5ac1c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5ac0753e0 0x7fb5ac10cfd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 --2- 172.21.15.57:0/95688123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5ac075d00 0x7fb5ac10d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 >> 172.21.15.57:0/95688123 conn(0x7fb5ac0feb60 msgr2=0x7fb5ac100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 shutdown_connections 2023-10-06T22:16:22.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:22.970+0000 7fb5b3fff700 1 -- 172.21.15.57:0/95688123 wait complete. 2023-10-06T22:16:22.991 INFO:tasks.cephfs.filesystem:scrub status for tag:c61e70d9-5dde-4abe-a4bb-9fadc49ba031 - {'path': '/', 'tag': 'c61e70d9-5dde-4abe-a4bb-9fadc49ba031', 'options': 'recursive,force'} 2023-10-06T22:16:23.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:22 smithi156 ceph-mon[134358]: pgmap v52: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 983 KiB/s wr, 48 op/s 2023-10-06T22:16:23.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:22 smithi156 ceph-mon[134358]: scrub queued for path: / 2023-10-06T22:16:23.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:22 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:23.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:22 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:16:23.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:22 smithi190 ceph-mon[134891]: pgmap v52: 97 pgs: 97 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 983 KiB/s wr, 48 op/s 2023-10-06T22:16:23.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:22 smithi190 ceph-mon[134891]: scrub queued for path: / 2023-10-06T22:16:23.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:22 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:23.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:22 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:16:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 -- 172.21.15.57:0/3750105733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e40732e0 msgr2=0x7f13e40736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 --2- 172.21.15.57:0/3750105733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e40732e0 0x7f13e40736c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009990 tx=0x7f13d8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 -- 172.21.15.57:0/3750105733 shutdown_connections 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 --2- 172.21.15.57:0/3750105733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e410d6a0 0x7f13e410fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 --2- 172.21.15.57:0/3750105733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e40732e0 0x7f13e40736c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 --2- 172.21.15.57:0/3750105733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4073c00 0x7f13e410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 -- 172.21.15.57:0/3750105733 >> 172.21.15.57:0/3750105733 conn(0x7f13e40fc950 msgr2=0x7f13e40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 -- 172.21.15.57:0/3750105733 shutdown_connections 2023-10-06T22:16:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.170+0000 7f13eadff700 1 -- 172.21.15.57:0/3750105733 wait complete. 2023-10-06T22:16:24.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:23 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:16:24.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:23 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 Processor -- start 2023-10-06T22:16:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 -- start start 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40732e0 0x7f13e41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4073c00 0x7f13e4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e410d6a0 0x7f13e419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419e3e0 con 0x7f13e4073c00 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419e550 con 0x7f13e410d6a0 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.171+0000 7f13eadff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419e710 con 0x7f13e40732e0 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e939c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e410d6a0 0x7f13e419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e8b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40732e0 0x7f13e41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e939c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e410d6a0 0x7f13e419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34284/0 (socket says 172.21.15.57:34284) 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e939c700 1 -- 172.21.15.57:0/2298734665 learned_addr learned my addr 172.21.15.57:0/2298734665 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e3fff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4073c00 0x7f13e4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e8b9b700 1 -- 172.21.15.57:0/2298734665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e410d6a0 msgr2=0x7f13e419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e410d6a0 0x7f13e419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e8b9b700 1 -- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4073c00 msgr2=0x7f13e4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4073c00 0x7f13e4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e8b9b700 1 -- 172.21.15.57:0/2298734665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009520 con 0x7f13e40732e0 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e3fff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4073c00 0x7f13e4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.172+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40732e0 0x7f13e41991f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009990 tx=0x7f13d800f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.173+0000 7f13e1ffb700 1 -- 172.21.15.57:0/2298734665 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d801d070 con 0x7f13e40732e0 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.173+0000 7f13e1ffb700 1 -- 172.21.15.57:0/2298734665 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13d8005da0 con 0x7f13e40732e0 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.173+0000 7f13e1ffb700 1 -- 172.21.15.57:0/2298734665 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d80047e0 con 0x7f13e40732e0 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.173+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d80095f0 con 0x7f13e40732e0 2023-10-06T22:16:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.173+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e41a6050 con 0x7f13e40732e0 2023-10-06T22:16:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.176+0000 7f13e1ffb700 1 -- 172.21.15.57:0/2298734665 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f13d8005230 con 0x7f13e40732e0 2023-10-06T22:16:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.177+0000 7f13fe7c7b80 1 Processor -- start 2023-10-06T22:16:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.177+0000 7f13fe7c7b80 1 -- start start 2023-10-06T22:16:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284661e400 con 0x5628466b5520 2023-10-06T22:16:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284660fbc0 con 0x5628466c03d0 2023-10-06T22:16:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284660d4e0 con 0x5628466b4c00 2023-10-06T22:16:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13e8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13e939c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13e939c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48622/0 (socket says 172.21.15.57:48622) 2023-10-06T22:16:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13e939c700 1 -- 172.21.15.57:0/3807029008 learned_addr learned my addr 172.21.15.57:0/3807029008 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13e3fff700 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.178+0000 7f13e1ffb700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13d40a1670 0x7f13d40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e1ffb700 1 -- 172.21.15.57:0/2298734665 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f13d80cdd60 con 0x7f13e40732e0 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e3fff700 1 -- 172.21.15.57:0/3807029008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 msgr2=0x5628466b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e3fff700 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e3fff700 1 -- 172.21.15.57:0/3807029008 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 msgr2=0x5628466bfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e3fff700 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466bfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e3fff700 1 -- 172.21.15.57:0/3807029008 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562846433d80 con 0x5628466c03d0 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e3fff700 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c27c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009e20 tx=0x7f13d00116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87a183c4db5c800 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13e939c700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13d40a1670 0x7f13d40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13ceffd700 1 -- 172.21.15.57:0/3807029008 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0011e20 con 0x5628466c03d0 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13ceffd700 1 -- 172.21.15.57:0/3807029008 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13d0012910 con 0x5628466c03d0 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.179+0000 7f13ceffd700 1 -- 172.21.15.57:0/3807029008 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000f940 con 0x5628466c03d0 2023-10-06T22:16:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13e939c700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13d40a1670 0x7f13d40a3b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009fd0 tx=0x7f13dc004ad0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/3807029008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 msgr2=0x5628466c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c27c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009e20 tx=0x7f13d00116c0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/3807029008 shutdown_connections 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c27c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/3807029008 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466bfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/3807029008 >> 172.21.15.57:0/3807029008 conn(0x56284662a200 msgr2=0x5628466b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/3807029008 shutdown_connections 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.180+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/3807029008 wait complete. 2023-10-06T22:16:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 -- start start 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 Processor -- start 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466b5520 0x5628466bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466c03d0 0x5628466bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846608dd0 con 0x5628466c03d0 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628465a56b0 con 0x5628466b5520 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628465a5350 con 0x5628466b4c00 2023-10-06T22:16:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.181+0000 7f13fe7c7b80 10 client.0 _ll_register_callbacks cb 0x5628464027a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466b5520 0x5628466bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466b5520 0x5628466bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34302/0 (socket says 172.21.15.57:34302) 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 -- 172.21.15.57:0/4040296678 learned_addr learned my addr 172.21.15.57:0/4040296678 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e8b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e939c700 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466c03d0 0x5628466bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 -- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 msgr2=0x5628466bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 -- 172.21.15.57:0/4040296678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466c03d0 msgr2=0x5628466bd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466c03d0 0x5628466bd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 -- 172.21.15.57:0/4040296678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d0009a80 con 0x5628466b5520 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13e3fff700 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466b5520 0x5628466bd090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13d000a010 tx=0x7f13d0012cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97ff2b4782ea809f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.182+0000 7f13c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13c3fff700 1 -- 172.21.15.57:0/4040296678 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0012660 con 0x5628466b5520 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13c3fff700 1 -- 172.21.15.57:0/4040296678 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f13d0004020 con 0x5628466b5520 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628466a8d50 con 0x5628466b5520 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13c3fff700 1 -- 172.21.15.57:0/4040296678 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13d00041e0 con 0x5628466b5520 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13fe7c7b80 10 client.26147 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56284641cc00 con 0x5628466b5520 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13c3fff700 1 -- 172.21.15.57:0/4040296678 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628466a8d50 con 0x5628466b5520 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13c3fff700 1 -- 172.21.15.57:0/4040296678 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f13d0022040 con 0x5628466b5520 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13fe7c7b80 10 client.26147 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13fe7c7b80 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.183+0000 7f13fe7c7b80 20 client.26147 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.184+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5628466cfd30 0x5628466d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.184+0000 7f13fe7c7b80 4 client.26147 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.184+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562846601db0 con 0x5628466cfd30 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.184+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5628466cfd30 0x5628466d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.184+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5628466cfd30 0x5628466d2150 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28dc440a86a807c server_cookie=e941b9218ebf6995 in_seq=0 out_seq=0 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.184+0000 7f13c3fff700 10 client.26147 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13c3fff700 1 -- 172.21.15.57:0/4040296678 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562846601db0 con 0x5628466cfd30 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13c3fff700 10 client.26147 handle_command_reply: tid=0 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 client.26147 shutdown 2023-10-06T22:16:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466b5520 msgr2=0x5628466bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466b5520 0x5628466bd090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13d000a010 tx=0x7f13d0012cb0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 shutdown_connections 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5628466cfd30 0x5628466d2150 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466c03d0 0x5628466bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/4040296678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466b5520 0x5628466bd090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13c3fff700 0 client.26147 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 >> 172.21.15.57:0/4040296678 conn(0x56284662a200 msgr2=0x5628466b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.185+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 shutdown_connections 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.186+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/4040296678 wait complete. 2023-10-06T22:16:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.186+0000 7f13fe7c7b80 20 client.26147 trim_cache size 0 max 16384 2023-10-06T22:16:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.203+0000 7f13fe7c7b80 1 Processor -- start 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13fe7c7b80 1 -- start start 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b4c00 0x5628466d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b5520 0x5628466c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466bf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628464898c0 con 0x5628466b4c00 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628466b5e20 con 0x5628466c03d0 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628466ad860 con 0x5628466b5520 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13e939c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b5520 0x5628466c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13e8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b4c00 0x5628466d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13e939c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466bf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34312/0 (socket says 172.21.15.57:34312) 2023-10-06T22:16:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13e939c700 1 -- 172.21.15.57:0/2407954361 learned_addr learned my addr 172.21.15.57:0/2407954361 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.204+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b5520 0x5628466c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48644/0 (socket says 172.21.15.57:48644) 2023-10-06T22:16:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13e8b9b700 1 -- 172.21.15.57:0/2407954361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b5520 msgr2=0x5628466c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2407954361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b5520 0x5628466c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13e8b9b700 1 -- 172.21.15.57:0/2407954361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 msgr2=0x5628466bf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2407954361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466bf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13e8b9b700 1 -- 172.21.15.57:0/2407954361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56284645a1f0 con 0x5628466b4c00 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2407954361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b4c00 0x5628466d2120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f13d80cb490 tx=0x7f13d80a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6973d432c14087f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13cf7fe700 1 -- 172.21.15.57:0/2407954361 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d809a920 con 0x5628466b4c00 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13cf7fe700 1 -- 172.21.15.57:0/2407954361 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13d809a740 con 0x5628466b4c00 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.205+0000 7f13cf7fe700 1 -- 172.21.15.57:0/2407954361 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d809b040 con 0x5628466b4c00 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2407954361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b4c00 msgr2=0x5628466d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2407954361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b4c00 0x5628466d2120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f13d80cb490 tx=0x7f13d80a1040 comp rx=0 tx=0).stop 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2407954361 shutdown_connections 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2407954361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466bf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2407954361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b4c00 0x5628466d2120 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2407954361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b5520 0x5628466c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2407954361 >> 172.21.15.57:0/2407954361 conn(0x56284662a200 msgr2=0x5628466cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2407954361 shutdown_connections 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.206+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2407954361 wait complete. 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.207+0000 7f13fe7c7b80 1 -- start start 2023-10-06T22:16:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.207+0000 7f13fe7c7b80 1 Processor -- start 2023-10-06T22:16:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13fe7c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846722d00 con 0x5628466b5520 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846722e70 con 0x5628466c03d0 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13fe7c7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562846723030 con 0x5628466b4c00 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13fe7c7b80 10 client.0 _ll_register_callbacks cb 0x7f13d80971a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13e8b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52662/0 (socket says 172.21.15.57:52662) 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13e8b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48654/0 (socket says 172.21.15.57:48654) 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13e3fff700 1 -- 172.21.15.57:0/2712550401 learned_addr learned my addr 172.21.15.57:0/2712550401 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.208+0000 7f13e939c700 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13e3fff700 1 -- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 msgr2=0x5628466c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13e3fff700 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13e3fff700 1 -- 172.21.15.57:0/2712550401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 msgr2=0x5628466c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13e3fff700 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13e3fff700 1 -- 172.21.15.57:0/2712550401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e41a6050 con 0x5628466b5520 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13e3fff700 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466c4ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009e80 tx=0x7f13d0012890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922c513eee651ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13c3fff700 1 -- 172.21.15.57:0/2712550401 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0004b50 con 0x5628466b5520 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13c3fff700 1 -- 172.21.15.57:0/2712550401 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f13d0000e40 con 0x5628466b5520 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.209+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13d0012910 con 0x5628466b5520 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13c3fff700 1 -- 172.21.15.57:0/2712550401 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13d0024460 con 0x5628466b5520 2023-10-06T22:16:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 10 client.16134 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562846432250 con 0x5628466b5520 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13c3fff700 1 -- 172.21.15.57:0/2712550401 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13d0012910 con 0x5628466b5520 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13c3fff700 1 -- 172.21.15.57:0/2712550401 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f13d0011730 con 0x5628466b5520 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 10 client.16134 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 20 client.16134 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562846724400 0x5628467267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 4 client.16134 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.210+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562846601ad0 con 0x562846724400 2023-10-06T22:16:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.211+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562846724400 0x5628467267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.237+0000 7f13e8b9b700 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562846724400 0x5628467267f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fe9f6acf314607 server_cookie=4c3d27dd55153354 in_seq=0 out_seq=0 2023-10-06T22:16:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.237+0000 7f13c3fff700 10 client.16134 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:24.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:23 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:16:24.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:23 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.265+0000 7f13c3fff700 1 -- 172.21.15.57:0/2712550401 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562846601ad0 con 0x562846724400 2023-10-06T22:16:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.265+0000 7f13c3fff700 10 client.16134 handle_command_reply: tid=0 2023-10-06T22:16:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.265+0000 7f13fe7c7b80 1 client.16134 shutdown 2023-10-06T22:16:24.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:23 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:16:24.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:23 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.353+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 msgr2=0x5628466c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.353+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466c4ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009e80 tx=0x7f13d0012890 comp rx=0 tx=0).stop 2023-10-06T22:16:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.362+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 shutdown_connections 2023-10-06T22:16:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.362+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562846724400 0x5628467267f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.362+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5628466c03d0 0x5628466c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.362+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628466b4c00 0x5628466c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.362+0000 7f13fe7c7b80 1 --2- 172.21.15.57:0/2712550401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628466b5520 0x5628466c4ab0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.362+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 >> 172.21.15.57:0/2712550401 conn(0x56284662a200 msgr2=0x5628466b83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.363+0000 7f13c3fff700 0 client.16134 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.363+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 shutdown_connections 2023-10-06T22:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.363+0000 7f13fe7c7b80 1 -- 172.21.15.57:0/2712550401 wait complete. 2023-10-06T22:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.364+0000 7f13fe7c7b80 20 client.16134 trim_cache size 0 max 16384 2023-10-06T22:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:16:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.413+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13d40a1670 msgr2=0x7f13d40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.413+0000 7f13eadff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13d40a1670 0x7f13d40a3b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009fd0 tx=0x7f13dc004ad0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.413+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40732e0 msgr2=0x7f13e41991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.413+0000 7f13eadff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40732e0 0x7f13e41991f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009990 tx=0x7f13d800f670 comp rx=0 tx=0).stop 2023-10-06T22:16:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.414+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 shutdown_connections 2023-10-06T22:16:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.414+0000 7f13eadff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13d40a1670 0x7f13d40a3b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.414+0000 7f13eadff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13e410d6a0 0x7f13e419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.414+0000 7f13eadff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40732e0 0x7f13e41991f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.414+0000 7f13eadff700 1 --2- 172.21.15.57:0/2298734665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13e4073c00 0x7f13e4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.414+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 >> 172.21.15.57:0/2298734665 conn(0x7f13e40fc950 msgr2=0x7f13e410f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.420+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 shutdown_connections 2023-10-06T22:16:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:24.421+0000 7f13eadff700 1 -- 172.21.15.57:0/2298734665 wait complete. 2023-10-06T22:16:25.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:24 smithi057 ceph-mon[98492]: pgmap v53: 97 pgs: 97 active+clean; 45 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2023-10-06T22:16:25.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:24 smithi156 ceph-mon[134358]: pgmap v53: 97 pgs: 97 active+clean; 45 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2023-10-06T22:16:25.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:24 smithi190 ceph-mon[134891]: pgmap v53: 97 pgs: 97 active+clean; 45 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2023-10-06T22:16:27.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:26 smithi057 ceph-mon[98492]: pgmap v54: 97 pgs: 97 active+clean; 45 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 173 op/s 2023-10-06T22:16:27.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:26 smithi156 ceph-mon[134358]: pgmap v54: 97 pgs: 97 active+clean; 45 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 173 op/s 2023-10-06T22:16:27.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:26 smithi190 ceph-mon[134891]: pgmap v54: 97 pgs: 97 active+clean; 45 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 173 op/s 2023-10-06T22:16:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:28 smithi057 ceph-mon[98492]: pgmap v55: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 219 op/s 2023-10-06T22:16:29.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:28 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:29.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:28 smithi156 ceph-mon[134358]: pgmap v55: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 219 op/s 2023-10-06T22:16:29.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:28 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:29.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:28 smithi190 ceph-mon[134891]: pgmap v55: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 219 op/s 2023-10-06T22:16:29.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:28 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:31.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:30 smithi057 ceph-mon[98492]: pgmap v56: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 206 op/s 2023-10-06T22:16:31.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:30 smithi156 ceph-mon[134358]: pgmap v56: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 206 op/s 2023-10-06T22:16:31.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:30 smithi190 ceph-mon[134891]: pgmap v56: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 206 op/s 2023-10-06T22:16:32.033 INFO:tasks.workunit.client.0.smithi057.stdout: 26 409327 496 261465 658 149399 1113 2023-10-06T22:16:33.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:32 smithi156 ceph-mon[134358]: pgmap v57: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 206 op/s 2023-10-06T22:16:33.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:32 smithi190 ceph-mon[134891]: pgmap v57: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 206 op/s 2023-10-06T22:16:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:32 smithi057 ceph-mon[98492]: pgmap v57: 97 pgs: 97 active+clean; 55 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 206 op/s 2023-10-06T22:16:34.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:33 smithi156 ceph-mon[134358]: pgmap v58: 97 pgs: 97 active+clean; 63 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2023-10-06T22:16:34.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:33 smithi190 ceph-mon[134891]: pgmap v58: 97 pgs: 97 active+clean; 63 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2023-10-06T22:16:34.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:33 smithi057 ceph-mon[98492]: pgmap v58: 97 pgs: 97 active+clean; 63 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2023-10-06T22:16:35.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:34 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:16:35.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:34 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:16:35.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:34 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:16:36.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:35 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:16:36.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:35 smithi156 ceph-mon[134358]: pgmap v59: 97 pgs: 97 active+clean; 63 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 70 op/s 2023-10-06T22:16:36.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:35 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:16:36.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:35 smithi190 ceph-mon[134891]: pgmap v59: 97 pgs: 97 active+clean; 63 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 70 op/s 2023-10-06T22:16:36.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:35 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:16:36.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:35 smithi057 ceph-mon[98492]: pgmap v59: 97 pgs: 97 active+clean; 63 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 70 op/s 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 -- 172.21.15.57:0/4167590777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4103300 msgr2=0x7f06f41036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 --2- 172.21.15.57:0/4167590777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4103300 0x7f06f41036e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009990 tx=0x7f06e4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 -- 172.21.15.57:0/4167590777 shutdown_connections 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 --2- 172.21.15.57:0/4167590777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4108420 0x7f06f410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 --2- 172.21.15.57:0/4167590777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f4103300 0x7f06f41036e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 --2- 172.21.15.57:0/4167590777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f4103cb0 0x7f06f4107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 -- 172.21.15.57:0/4167590777 >> 172.21.15.57:0/4167590777 conn(0x7f06f40feb20 msgr2=0x7f06f4100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 -- 172.21.15.57:0/4167590777 shutdown_connections 2023-10-06T22:16:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.725+0000 7f06f93ff700 1 -- 172.21.15.57:0/4167590777 wait complete. 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 Processor -- start 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 -- start start 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4103300 0x7f06f4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f4108420 0x7f06f41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f419dd10 0x7f06f419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e790 con 0x7f06f4108420 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e900 con 0x7f06f419dd10 2023-10-06T22:16:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f93ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419eac0 con 0x7f06f4103300 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4103300 0x7f06f4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4103300 0x7f06f4199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48334/0 (socket says 172.21.15.57:48334) 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.727+0000 7f06f2ffd700 1 -- 172.21.15.57:0/1594396493 learned_addr learned my addr 172.21.15.57:0/1594396493 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f37fe700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f419dd10 0x7f06f419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f4108420 0x7f06f41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f37fe700 1 -- 172.21.15.57:0/1594396493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4103300 msgr2=0x7f06f4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f37fe700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4103300 0x7f06f4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f37fe700 1 -- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f4108420 msgr2=0x7f06f41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f37fe700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f4108420 0x7f06f41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f37fe700 1 -- 172.21.15.57:0/1594396493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8009520 con 0x7f06f419dd10 2023-10-06T22:16:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4103300 0x7f06f4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f4108420 0x7f06f41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.728+0000 7f06f37fe700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f419dd10 0x7f06f419e190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00efc0 tx=0x7f06ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.729+0000 7f06dbfff700 1 -- 172.21.15.57:0/1594396493 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec018070 con 0x7f06f419dd10 2023-10-06T22:16:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.729+0000 7f06dbfff700 1 -- 172.21.15.57:0/1594396493 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06ec00f040 con 0x7f06f419dd10 2023-10-06T22:16:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.729+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e40095f0 con 0x7f06f419dd10 2023-10-06T22:16:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.729+0000 7f06dbfff700 1 -- 172.21.15.57:0/1594396493 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00cd40 con 0x7f06f419dd10 2023-10-06T22:16:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.729+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f41a6310 con 0x7f06f419dd10 2023-10-06T22:16:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.733+0000 7f06dbfff700 1 -- 172.21.15.57:0/1594396493 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f06ec003a00 con 0x7f06f419dd10 2023-10-06T22:16:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.734+0000 7f070cceeb80 1 Processor -- start 2023-10-06T22:16:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.734+0000 7f070cceeb80 1 -- start start 2023-10-06T22:16:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x561521518fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x561521523e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x5615215267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f070cceeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561521482400 con 0x5615215243d0 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f070cceeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561521473bc0 con 0x561521518c00 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f070cceeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615214714e0 con 0x561521519520 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x561521523e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x561521523e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48350/0 (socket says 172.21.15.57:48350) 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f06f2ffd700 1 -- 172.21.15.57:0/1482954065 learned_addr learned my addr 172.21.15.57:0/1482954065 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x561521518fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f06dbfff700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06dc0a1670 0x7f06dc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x5615215267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.735+0000 7f06dbfff700 1 -- 172.21.15.57:0/1594396493 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f06ec0ccbb0 con 0x7f06f419dd10 2023-10-06T22:16:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f27fc700 1 -- 172.21.15.57:0/1482954065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 msgr2=0x561521523e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x561521523e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f27fc700 1 -- 172.21.15.57:0/1482954065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 msgr2=0x5615215267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x5615215267c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f27fc700 1 -- 172.21.15.57:0/1482954065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561521297d80 con 0x561521518c00 2023-10-06T22:16:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f37fe700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06dc0a1670 0x7f06dc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x561521518fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06f419a980 tx=0x7f06e8009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b828ef723071a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06d97fa700 1 -- 172.21.15.57:0/1482954065 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e800f980 con 0x561521518c00 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06d97fa700 1 -- 172.21.15.57:0/1482954065 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06e8004030 con 0x561521518c00 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06f37fe700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06dc0a1670 0x7f06dc0a3b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0ca370 tx=0x7f06ec004cc0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.736+0000 7f06d97fa700 1 -- 172.21.15.57:0/1482954065 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004960 con 0x561521518c00 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 -- 172.21.15.57:0/1482954065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 msgr2=0x561521518fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x561521518fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06f419a980 tx=0x7f06e8009800 comp rx=0 tx=0).stop 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 -- 172.21.15.57:0/1482954065 shutdown_connections 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x5615215267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x561521518fe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1482954065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x561521523e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 -- 172.21.15.57:0/1482954065 >> 172.21.15.57:0/1482954065 conn(0x56152148e200 msgr2=0x561521515080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 -- 172.21.15.57:0/1482954065 shutdown_connections 2023-10-06T22:16:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.737+0000 7f070cceeb80 1 -- 172.21.15.57:0/1482954065 wait complete. 2023-10-06T22:16:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.738+0000 7f070cceeb80 1 -- start start 2023-10-06T22:16:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.738+0000 7f070cceeb80 1 Processor -- start 2023-10-06T22:16:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x5615215211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561521519520 0x561521521710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615215243d0 0x561521521c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f070cceeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152146cdd0 con 0x561521519520 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f070cceeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615214096b0 con 0x5615215243d0 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f070cceeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561521409350 con 0x561521518c00 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f070cceeb80 10 client.0 _ll_register_callbacks cb 0x5615212667a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f06f27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x5615215211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f06f27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x5615215211d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48358/0 (socket says 172.21.15.57:48358) 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f06f27fc700 1 -- 172.21.15.57:0/1612071772 learned_addr learned my addr 172.21.15.57:0/1612071772 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561521519520 0x561521521710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.739+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615215243d0 0x561521521c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06f27fc700 1 -- 172.21.15.57:0/1612071772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615215243d0 msgr2=0x561521521c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615215243d0 0x561521521c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06f27fc700 1 -- 172.21.15.57:0/1612071772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561521519520 msgr2=0x561521521710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561521519520 0x561521521710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06f27fc700 1 -- 172.21.15.57:0/1612071772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e8019040 con 0x561521518c00 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561521519520 0x561521521710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06f27fc700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x5615215211d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06e8011b10 tx=0x7f06e8009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac57059a59086c21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06d6ff5700 1 -- 172.21.15.57:0/1612071772 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004440 con 0x561521518c00 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56152150cd50 con 0x561521518c00 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06d6ff5700 1 -- 172.21.15.57:0/1612071772 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f06e800f980 con 0x561521518c00 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f06d6ff5700 1 -- 172.21.15.57:0/1612071772 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06e80045a0 con 0x561521518c00 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.740+0000 7f070cceeb80 10 client.26155 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561521280c00 con 0x561521518c00 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f06d6ff5700 1 -- 172.21.15.57:0/1612071772 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56152150cd50 con 0x561521518c00 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f06d6ff5700 1 -- 172.21.15.57:0/1612071772 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f06e801d070 con 0x561521518c00 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f070cceeb80 10 client.26155 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f070cceeb80 10 client.26155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f070cceeb80 20 client.26155 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x561521536150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f070cceeb80 4 client.26155 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561521465db0 con 0x561521533d30 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.741+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x561521536150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.742+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x561521536150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3106ff4c40aaa734 server_cookie=56f49de14f81677a in_seq=0 out_seq=0 2023-10-06T22:16:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.742+0000 7f06d6ff5700 10 client.26155 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.743+0000 7f06d6ff5700 1 -- 172.21.15.57:0/1612071772 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561521465db0 con 0x561521533d30 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.743+0000 7f06d6ff5700 10 client.26155 handle_command_reply: tid=0 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.743+0000 7f070cceeb80 1 client.26155 shutdown 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 msgr2=0x5615215211d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x5615215211d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06e8011b10 tx=0x7f06e8009400 comp rx=0 tx=0).stop 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 shutdown_connections 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x561521536150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615215243d0 0x561521521c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x5615215211d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f06d6ff5700 0 client.26155 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 --2- 172.21.15.57:0/1612071772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561521519520 0x561521521710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 >> 172.21.15.57:0/1612071772 conn(0x56152148e200 msgr2=0x561521514c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 shutdown_connections 2023-10-06T22:16:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 1 -- 172.21.15.57:0/1612071772 wait complete. 2023-10-06T22:16:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.744+0000 7f070cceeb80 20 client.26155 trim_cache size 0 max 16384 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.761+0000 7f070cceeb80 1 Processor -- start 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.761+0000 7f070cceeb80 1 -- start start 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x56152152af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521519520 0x56152151baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x56152151e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f070cceeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615212ed8c0 con 0x5615215243d0 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f070cceeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561521515df0 con 0x561521519520 2023-10-06T22:16:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f070cceeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152152b490 con 0x561521518c00 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x56152152af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521519520 0x56152151baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521519520 0x56152151baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:32908/0 (socket says 172.21.15.57:32908) 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 -- 172.21.15.57:0/322007038 learned_addr learned my addr 172.21.15.57:0/322007038 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f27fc700 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x56152151e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 -- 172.21.15.57:0/322007038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 msgr2=0x56152152af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x56152152af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 -- 172.21.15.57:0/322007038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 msgr2=0x56152151e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x56152151e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f2ffd700 1 -- 172.21.15.57:0/322007038 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615212be1f0 con 0x561521519520 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.762+0000 7f06f27fc700 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x56152151e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.763+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521519520 0x56152151baa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f06e4005950 tx=0x7f06e400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e86f6428308f057 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.763+0000 7f06d9ffb700 1 -- 172.21.15.57:0/322007038 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e402a070 con 0x561521519520 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.763+0000 7f06d9ffb700 1 -- 172.21.15.57:0/322007038 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06e402e070 con 0x561521519520 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.763+0000 7f06d9ffb700 1 -- 172.21.15.57:0/322007038 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e401d040 con 0x561521519520 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 -- 172.21.15.57:0/322007038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521519520 msgr2=0x56152151baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521519520 0x56152151baa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f06e4005950 tx=0x7f06e400b260 comp rx=0 tx=0).stop 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 -- 172.21.15.57:0/322007038 shutdown_connections 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x56152151e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521518c00 0x56152152af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 --2- 172.21.15.57:0/322007038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521519520 0x56152151baa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 -- 172.21.15.57:0/322007038 >> 172.21.15.57:0/322007038 conn(0x56152148e200 msgr2=0x56152151c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 -- 172.21.15.57:0/322007038 shutdown_connections 2023-10-06T22:16:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 -- 172.21.15.57:0/322007038 wait complete. 2023-10-06T22:16:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 -- start start 2023-10-06T22:16:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.764+0000 7f070cceeb80 1 Processor -- start 2023-10-06T22:16:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x56152157bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x56152157c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f070cceeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x561521586bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f070cceeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152157c950 con 0x5615215243d0 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f070cceeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615215870f0 con 0x561521518c00 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f070cceeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561521587230 con 0x561521519520 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f070cceeb80 10 client.0 _ll_register_callbacks cb 0x7f06e8011c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x56152157c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f06f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x56152157c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48368/0 (socket says 172.21.15.57:48368) 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f06f2ffd700 1 -- 172.21.15.57:0/399104832 learned_addr learned my addr 172.21.15.57:0/399104832 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f06f27fc700 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x56152157bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.765+0000 7f06f27fc700 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x561521586bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06f2ffd700 1 -- 172.21.15.57:0/399104832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 msgr2=0x56152157bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x56152157bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06f2ffd700 1 -- 172.21.15.57:0/399104832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 msgr2=0x561521586bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x561521586bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06f2ffd700 1 -- 172.21.15.57:0/399104832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f4110810 con 0x561521519520 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06f2ffd700 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x56152157c260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06e40058b0 tx=0x7f06e4004e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f89903de29fcb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06d6ff5700 1 -- 172.21.15.57:0/399104832 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e402a070 con 0x561521519520 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f06d6ff5700 1 -- 172.21.15.57:0/399104832 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f06e4005580 con 0x561521519520 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.766+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06e8004030 con 0x561521519520 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f06d6ff5700 1 -- 172.21.15.57:0/399104832 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06e401d040 con 0x561521519520 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 10 client.26161 fetch_fsmap learned FSMap version 30 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561521296250 con 0x561521519520 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f06d6ff5700 1 -- 172.21.15.57:0/399104832 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06e4042080 con 0x561521519520 2023-10-06T22:16:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f06d6ff5700 1 -- 172.21.15.57:0/399104832 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f06e400fb00 con 0x561521519520 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 10 client.26161 fetch_fsmap finished waiting for FSMap version 30 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 20 client.26161 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x56152158a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 4 client.26161 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.767+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561521465ad0 con 0x561521533d30 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.768+0000 7f06f27fc700 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x56152158a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.768+0000 7f06f27fc700 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x56152158a4a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d80280cfb8b90da server_cookie=3ec36d66a57a1874 in_seq=0 out_seq=0 2023-10-06T22:16:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:37.768+0000 7f06d6ff5700 10 client.26161 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.761+0000 7f06d6ff5700 1 -- 172.21.15.57:0/399104832 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561521465ad0 con 0x561521533d30 2023-10-06T22:16:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.761+0000 7f06d6ff5700 10 client.26161 handle_command_reply: tid=0 2023-10-06T22:16:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.761+0000 7f070cceeb80 1 client.26161 shutdown 2023-10-06T22:16:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.762+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 msgr2=0x56152157c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.762+0000 7f070cceeb80 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x56152157c260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06e40058b0 tx=0x7f06e4004e80 comp rx=0 tx=0).stop 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 shutdown_connections 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f070cceeb80 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561521533d30 0x56152158a4a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f070cceeb80 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5615215243d0 0x561521586bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f070cceeb80 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561521518c00 0x56152157bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f070cceeb80 1 --2- 172.21.15.57:0/399104832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561521519520 0x56152157c260 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 >> 172.21.15.57:0/399104832 conn(0x56152148e200 msgr2=0x561521522a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f06d6ff5700 0 client.26161 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.765+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 shutdown_connections 2023-10-06T22:16:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.766+0000 7f070cceeb80 1 -- 172.21.15.57:0/399104832 wait complete. 2023-10-06T22:16:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.776+0000 7f070cceeb80 20 client.26161 trim_cache size 0 max 16384 2023-10-06T22:16:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:16:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:16:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:16:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:16:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.777+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06dc0a1670 msgr2=0x7f06dc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.777+0000 7f06f93ff700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06dc0a1670 0x7f06dc0a3b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0ca370 tx=0x7f06ec004cc0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.777+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f419dd10 msgr2=0x7f06f419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.777+0000 7f06f93ff700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f419dd10 0x7f06f419e190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00efc0 tx=0x7f06ec00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.778+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 shutdown_connections 2023-10-06T22:16:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.778+0000 7f06f93ff700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06dc0a1670 0x7f06dc0a3b30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.778+0000 7f06f93ff700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06f419dd10 0x7f06f419e190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.778+0000 7f06f93ff700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f4103300 0x7f06f4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.778+0000 7f06f93ff700 1 --2- 172.21.15.57:0/1594396493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f4108420 0x7f06f41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.778+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 >> 172.21.15.57:0/1594396493 conn(0x7f06f40feb20 msgr2=0x7f06f4106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.784+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 shutdown_connections 2023-10-06T22:16:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:38.784+0000 7f06f93ff700 1 -- 172.21.15.57:0/1594396493 wait complete. 2023-10-06T22:16:39.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:38 smithi190 ceph-mon[134891]: pgmap v60: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 92 op/s 2023-10-06T22:16:39.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:38 smithi057 ceph-mon[98492]: pgmap v60: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 92 op/s 2023-10-06T22:16:39.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:38 smithi156 ceph-mon[134358]: pgmap v60: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 92 op/s 2023-10-06T22:16:40.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:39 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:40.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:39 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:40.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:39 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:41.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:40 smithi190 ceph-mon[134891]: pgmap v61: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 45 op/s 2023-10-06T22:16:41.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:40 smithi057 ceph-mon[98492]: pgmap v61: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 45 op/s 2023-10-06T22:16:41.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:40 smithi156 ceph-mon[134358]: pgmap v61: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 45 op/s 2023-10-06T22:16:42.033 INFO:tasks.workunit.client.0.smithi057.stdout: 33 322650 364 223801 486 137356 1318 2023-10-06T22:16:43.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:42 smithi190 ceph-mon[134891]: pgmap v62: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 45 op/s 2023-10-06T22:16:43.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:42 smithi057 ceph-mon[98492]: pgmap v62: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 45 op/s 2023-10-06T22:16:43.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:42 smithi156 ceph-mon[134358]: pgmap v62: 97 pgs: 97 active+clean; 72 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 45 op/s 2023-10-06T22:16:45.183 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:44 smithi057 ceph-mon[98492]: pgmap v63: 97 pgs: 97 active+clean; 77 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2023-10-06T22:16:45.183 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:44 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:45.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:44 smithi156 ceph-mon[134358]: pgmap v63: 97 pgs: 97 active+clean; 77 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2023-10-06T22:16:45.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:44 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:45.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:44 smithi190 ceph-mon[134891]: pgmap v63: 97 pgs: 97 active+clean; 77 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2023-10-06T22:16:45.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:44 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:46 smithi057 ceph-mon[98492]: pgmap v64: 97 pgs: 97 active+clean; 77 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2023-10-06T22:16:47.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:46 smithi156 ceph-mon[134358]: pgmap v64: 97 pgs: 97 active+clean; 77 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2023-10-06T22:16:47.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:46 smithi190 ceph-mon[134891]: pgmap v64: 97 pgs: 97 active+clean; 77 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2023-10-06T22:16:49.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:48 smithi057 ceph-mon[98492]: pgmap v65: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-10-06T22:16:49.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:48 smithi156 ceph-mon[134358]: pgmap v65: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-10-06T22:16:49.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:48 smithi190 ceph-mon[134891]: pgmap v65: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-10-06T22:16:50.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:49 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:50.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:49 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:50.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:49 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:16:50.906 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:50 smithi057 ceph-mon[98492]: pgmap v66: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2023-10-06T22:16:51.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:50 smithi156 ceph-mon[134358]: pgmap v66: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2023-10-06T22:16:51.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:50 smithi190 ceph-mon[134891]: pgmap v66: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2023-10-06T22:16:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.967+0000 7fb2adbff700 1 -- 172.21.15.57:0/220975821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073280 msgr2=0x7fb2a8073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.967+0000 7fb2adbff700 1 --2- 172.21.15.57:0/220975821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073280 0x7fb2a8073660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb2980098f0 tx=0x7fb298009c00 comp rx=0 tx=0).stop 2023-10-06T22:16:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.967+0000 7fb2adbff700 1 -- 172.21.15.57:0/220975821 shutdown_connections 2023-10-06T22:16:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.967+0000 7fb2adbff700 1 --2- 172.21.15.57:0/220975821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2a810d6f0 0x7fb2a810fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.967+0000 7fb2adbff700 1 --2- 172.21.15.57:0/220975821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073280 0x7fb2a8073660 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.967+0000 7fb2adbff700 1 --2- 172.21.15.57:0/220975821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2a8073ba0 0x7fb2a810d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.967+0000 7fb2adbff700 1 -- 172.21.15.57:0/220975821 >> 172.21.15.57:0/220975821 conn(0x7fb2a80fc940 msgr2=0x7fb2a80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.968+0000 7fb2adbff700 1 -- 172.21.15.57:0/220975821 shutdown_connections 2023-10-06T22:16:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.968+0000 7fb2adbff700 1 -- 172.21.15.57:0/220975821 wait complete. 2023-10-06T22:16:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.969+0000 7fb2adbff700 1 Processor -- start 2023-10-06T22:16:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.969+0000 7fb2adbff700 1 -- start start 2023-10-06T22:16:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2adbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2a8073280 0x7fb2a8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2adbff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073ba0 0x7fb2a8199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2adbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2a810d6f0 0x7fb2a819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2adbff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a819e440 con 0x7fb2a8073ba0 2023-10-06T22:16:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2adbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a819e5b0 con 0x7fb2a810d6f0 2023-10-06T22:16:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2adbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a819e770 con 0x7fb2a8073280 2023-10-06T22:16:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073ba0 0x7fb2a8199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2a810d6f0 0x7fb2a819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073ba0 0x7fb2a8199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36180/0 (socket says 172.21.15.57:36180) 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1387253776 learned_addr learned my addr 172.21.15.57:0/1387253776 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.970+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2a8073280 0x7fb2a8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1387253776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2a8073280 msgr2=0x7fb2a8199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2a8073280 0x7fb2a8199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1387253776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2a810d6f0 msgr2=0x7fb2a819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2a810d6f0 0x7fb2a819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1387253776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0095f0 con 0x7fb2a8073ba0 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a7fff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2a810d6f0 0x7fb2a819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073ba0 0x7fb2a8199790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00ec10 tx=0x7fb29c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a4ff9700 1 -- 172.21.15.57:0/1387253776 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00cc90 con 0x7fb2a8073ba0 2023-10-06T22:16:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a4ff9700 1 -- 172.21.15.57:0/1387253776 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29c0044e0 con 0x7fb2a8073ba0 2023-10-06T22:16:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb298009520 con 0x7fb2a8073ba0 2023-10-06T22:16:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2a4ff9700 1 -- 172.21.15.57:0/1387253776 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c010430 con 0x7fb2a8073ba0 2023-10-06T22:16:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.971+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a81a60b0 con 0x7fb2a8073ba0 2023-10-06T22:16:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.975+0000 7fb2a4ff9700 1 -- 172.21.15.57:0/1387253776 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb29c003fb0 con 0x7fb2a8073ba0 2023-10-06T22:16:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.977+0000 7fb2c1449b80 1 Processor -- start 2023-10-06T22:16:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.977+0000 7fb2c1449b80 1 -- start start 2023-10-06T22:16:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.977+0000 7fb2a4ff9700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2900a15f0 0x7fb2900a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.977+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57d5efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57d69e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d6c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a4ff9700 1 -- 172.21.15.57:0/1387253776 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb29c014070 con 0x7fb2a8073ba0 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57cc8400 con 0x555b57d6a3d0 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57cb9bc0 con 0x555b57d5ec00 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57cb74e0 con 0x555b57d5f520 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d6c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57d69e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d6c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36182/0 (socket says 172.21.15.57:36182) 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a7fff700 1 -- 172.21.15.57:0/187082313 learned_addr learned my addr 172.21.15.57:0/187082313 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a7fff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2900a15f0 0x7fb2900a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57d5efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/187082313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 msgr2=0x555b57d5efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57d5efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/187082313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 msgr2=0x555b57d6c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d6c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/187082313 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b57addd80 con 0x555b57d5f520 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.978+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57d5efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.979+0000 7fb2a7fff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2900a15f0 0x7fb2900a3ab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009fd0 tx=0x7fb2a000b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.979+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57d69e90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0cada0 tx=0x7fb29c09a520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7564dcea8d0346 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.979+0000 7fb28dffb700 1 -- 172.21.15.57:0/187082313 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0850c0 con 0x555b57d5f520 2023-10-06T22:16:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.979+0000 7fb28dffb700 1 -- 172.21.15.57:0/187082313 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29c016330 con 0x555b57d5f520 2023-10-06T22:16:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.979+0000 7fb28dffb700 1 -- 172.21.15.57:0/187082313 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c098a60 con 0x555b57d5f520 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 -- 172.21.15.57:0/187082313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 msgr2=0x555b57d69e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57d69e90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0cada0 tx=0x7fb29c09a520 comp rx=0 tx=0).stop 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 -- 172.21.15.57:0/187082313 shutdown_connections 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d6c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57d5efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/187082313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57d69e90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 -- 172.21.15.57:0/187082313 >> 172.21.15.57:0/187082313 conn(0x555b57cd4200 msgr2=0x555b57d5b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 -- 172.21.15.57:0/187082313 shutdown_connections 2023-10-06T22:16:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.980+0000 7fb2c1449b80 1 -- 172.21.15.57:0/187082313 wait complete. 2023-10-06T22:16:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.981+0000 7fb2c1449b80 1 -- start start 2023-10-06T22:16:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.981+0000 7fb2c1449b80 1 Processor -- start 2023-10-06T22:16:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5f520 0x555b57d67670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d67bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57cb2dd0 con 0x555b57d6a3d0 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57c4f6b0 con 0x555b57d5f520 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57c4f350 con 0x555b57d5ec00 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2c1449b80 10 client.0 _ll_register_callbacks cb 0x555b57aac7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5f520 0x555b57d67670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d67bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5f520 0x555b57d67670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49092/0 (socket says 172.21.15.57:49092) 2023-10-06T22:16:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.982+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1793801328 learned_addr learned my addr 172.21.15.57:0/1793801328 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 msgr2=0x555b57d67130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d67130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1793801328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 msgr2=0x555b57d67bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d67bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1793801328 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a81a60b0 con 0x555b57d5f520 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5f520 0x555b57d67670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0cae00 tx=0x7fb29c09a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb4ed74b5d14d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb286ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb286ffd700 1 -- 172.21.15.57:0/1793801328 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0a29c0 con 0x555b57d5f520 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb286ffd700 1 -- 172.21.15.57:0/1793801328 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb29c0b5070 con 0x555b57d5f520 2023-10-06T22:16:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b57d52d50 con 0x555b57d5f520 2023-10-06T22:16:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb286ffd700 1 -- 172.21.15.57:0/1793801328 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb29c09ad30 con 0x555b57d5f520 2023-10-06T22:16:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2c1449b80 10 client.26183 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.983+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555b57ac6c00 con 0x555b57d5f520 2023-10-06T22:16:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb286ffd700 1 -- 172.21.15.57:0/1793801328 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b57d52d50 con 0x555b57d5f520 2023-10-06T22:16:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb286ffd700 1 -- 172.21.15.57:0/1793801328 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb29c099b90 con 0x555b57d5f520 2023-10-06T22:16:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb2c1449b80 10 client.26183 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb2c1449b80 10 client.26183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb2c1449b80 20 client.26183 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57d79b60 0x555b57d7bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb2c1449b80 4 client.26183 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b57cabdb0 con 0x555b57d79b60 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.984+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57d79b60 0x555b57d7bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.985+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57d79b60 0x555b57d7bf80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a117fd0fd8b427 server_cookie=a5ee3f58e14b5079 in_seq=0 out_seq=0 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.985+0000 7fb286ffd700 10 client.26183 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb286ffd700 1 -- 172.21.15.57:0/1793801328 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555b57cabdb0 con 0x555b57d79b60 2023-10-06T22:16:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb286ffd700 10 client.26183 handle_command_reply: tid=0 2023-10-06T22:16:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 client.26183 shutdown 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5f520 msgr2=0x555b57d67670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5f520 0x555b57d67670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0cae00 tx=0x7fb29c09a650 comp rx=0 tx=0).stop 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 shutdown_connections 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57d79b60 0x555b57d7bf80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57d67bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb286ffd700 0 client.26183 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1793801328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5f520 0x555b57d67670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 >> 172.21.15.57:0/1793801328 conn(0x555b57cd4200 msgr2=0x555b57d5ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 shutdown_connections 2023-10-06T22:16:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1793801328 wait complete. 2023-10-06T22:16:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:51.986+0000 7fb2c1449b80 20 client.26183 trim_cache size 0 max 16384 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 Processor -- start 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 -- start start 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d5f520 0x555b57d5ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d6a3d0 0x555b57d5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57b338c0 con 0x555b57d5f520 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57d5bdc0 con 0x555b57d6a3d0 2023-10-06T22:16:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.003+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57d71460 con 0x555b57d5ec00 2023-10-06T22:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d5f520 0x555b57d5ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d6a3d0 0x555b57d5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d5f520 0x555b57d5ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36196/0 (socket says 172.21.15.57:36196) 2023-10-06T22:16:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1806603386 learned_addr learned my addr 172.21.15.57:0/1806603386 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1806603386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 msgr2=0x555b57d70f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d70f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1806603386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d6a3d0 msgr2=0x555b57d5b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d6a3d0 0x555b57d5b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 -- 172.21.15.57:0/1806603386 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b57b041f0 con 0x555b57d5f520 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a7fff700 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d6a3d0 0x555b57d5b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.004+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d5f520 0x555b57d5ab50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00c5b0 tx=0x7fb29c0caea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1ca5573ce5ad76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb28e7fc700 1 -- 172.21.15.57:0/1806603386 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c09a6c0 con 0x555b57d5f520 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb28e7fc700 1 -- 172.21.15.57:0/1806603386 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29c09ad30 con 0x555b57d5f520 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb28e7fc700 1 -- 172.21.15.57:0/1806603386 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c09a890 con 0x555b57d5f520 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1806603386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d5f520 msgr2=0x555b57d5ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d5f520 0x555b57d5ab50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00c5b0 tx=0x7fb29c0caea0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1806603386 shutdown_connections 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d6a3d0 0x555b57d5b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5ec00 0x555b57d70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/1806603386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d5f520 0x555b57d5ab50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.005+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1806603386 >> 172.21.15.57:0/1806603386 conn(0x555b57cd4200 msgr2=0x555b57d62430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.006+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1806603386 shutdown_connections 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.006+0000 7fb2c1449b80 1 -- 172.21.15.57:0/1806603386 wait complete. 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.006+0000 7fb2c1449b80 1 -- start start 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.006+0000 7fb2c1449b80 1 Processor -- start 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57dc18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57dc1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2c1449b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57dcc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57dc25e0 con 0x555b57d6a3d0 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57dc2750 con 0x555b57d5ec00 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2c1449b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b57dccea0 con 0x555b57d5f520 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2c1449b80 10 client.0 _ll_register_callbacks cb 0x7fb2a0009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57dc18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57dc18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49106/0 (socket says 172.21.15.57:49106) 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 -- 172.21.15.57:0/50543279 learned_addr learned my addr 172.21.15.57:0/50543279 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a7fff700 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57dcc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57dc1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 -- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 msgr2=0x555b57dc1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57dc1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 -- 172.21.15.57:0/50543279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 msgr2=0x555b57dcc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57dcc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.007+0000 7fb2a77fe700 1 -- 172.21.15.57:0/50543279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b57ac6c00 con 0x555b57d5ec00 2023-10-06T22:16:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb2a77fe700 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57dc18e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2980098c0 tx=0x7fb2980261a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7839d4465884cbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb286ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb286ffd700 1 -- 172.21.15.57:0/50543279 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29801c040 con 0x555b57d5ec00 2023-10-06T22:16:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb29c016330 con 0x555b57d5ec00 2023-10-06T22:16:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb286ffd700 1 -- 172.21.15.57:0/50543279 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fb298026560 con 0x555b57d5ec00 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb286ffd700 1 -- 172.21.15.57:0/50543279 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb298026c70 con 0x555b57d5ec00 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb2c1449b80 10 client.26189 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.008+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555b57adc250 con 0x555b57d5ec00 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb286ffd700 1 -- 172.21.15.57:0/50543279 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29c016330 con 0x555b57d5ec00 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb286ffd700 1 -- 172.21.15.57:0/50543279 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb298015020 con 0x555b57d5ec00 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb2c1449b80 10 client.26189 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb2c1449b80 10 client.26189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb2c1449b80 20 client.26189 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57dce670 0x555b57dd0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb2c1449b80 4 client.26189 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555b57cabad0 con 0x555b57dce670 2023-10-06T22:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.009+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57dce670 0x555b57dd0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.010+0000 7fb286ffd700 10 client.26189 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:52.010+0000 7fb2a6ffd700 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57dce670 0x555b57dd0a90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3aa47b511ee2565 server_cookie=3206259d9b072ea3 in_seq=0 out_seq=0 2023-10-06T22:16:52.033 INFO:tasks.workunit.client.0.smithi057.stdout: 38 148207 206 108199 283 79293 801 2023-10-06T22:16:52.992 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:16:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.093+0000 7fb286ffd700 1 -- 172.21.15.57:0/50543279 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555b57cabad0 con 0x555b57dce670 2023-10-06T22:16:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.093+0000 7fb286ffd700 10 client.26189 handle_command_reply: tid=0 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.093+0000 7fb2c1449b80 1 client.26189 shutdown 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 msgr2=0x555b57dc18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57dc18e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2980098c0 tx=0x7fb2980261a0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 shutdown_connections 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555b57dce670 0x555b57dd0a90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555b57d6a3d0 0x555b57dcc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b57d5ec00 0x555b57dc18e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb286ffd700 0 client.26189 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 --2- 172.21.15.57:0/50543279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b57d5f520 0x555b57dc1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 >> 172.21.15.57:0/50543279 conn(0x555b57cd4200 msgr2=0x555b57d5aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 shutdown_connections 2023-10-06T22:16:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 1 -- 172.21.15.57:0/50543279 wait complete. 2023-10-06T22:16:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.094+0000 7fb2c1449b80 20 client.26189 trim_cache size 0 max 16384 2023-10-06T22:16:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:16:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:16:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:16:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2900a15f0 msgr2=0x7fb2900a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2900a15f0 0x7fb2900a3ab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009fd0 tx=0x7fb2a000b040 comp rx=0 tx=0).stop 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073ba0 msgr2=0x7fb2a8199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073ba0 0x7fb2a8199790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00ec10 tx=0x7fb29c00ef20 comp rx=0 tx=0).stop 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 shutdown_connections 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2900a15f0 0x7fb2900a3ab0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2a810d6f0 0x7fb2a819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2a8073280 0x7fb2a8199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 --2- 172.21.15.57:0/1387253776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2a8073ba0 0x7fb2a8199790 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 >> 172.21.15.57:0/1387253776 conn(0x7fb2a80fc940 msgr2=0x7fb2a810f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 shutdown_connections 2023-10-06T22:16:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:16:53.095+0000 7fb2adbff700 1 -- 172.21.15.57:0/1387253776 wait complete. 2023-10-06T22:16:53.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:52 smithi057 ceph-mon[98492]: pgmap v67: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2023-10-06T22:16:53.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 -- 172.21.15.57:0/355446375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81050b0 msgr2=0x7f12a810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 --2- 172.21.15.57:0/355446375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81050b0 0x7f12a810cc30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f12a000b0a0 tx=0x7f12a000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 -- 172.21.15.57:0/355446375 shutdown_connections 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 --2- 172.21.15.57:0/355446375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81050b0 0x7f12a810cc30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 --2- 172.21.15.57:0/355446375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a80fff90 0x7f12a8100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 --2- 172.21.15.57:0/355446375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a8100940 0x7f12a8104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 -- 172.21.15.57:0/355446375 >> 172.21.15.57:0/355446375 conn(0x7f12a80fb830 msgr2=0x7f12a80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 -- 172.21.15.57:0/355446375 shutdown_connections 2023-10-06T22:16:53.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.221+0000 7f12b01ff700 1 -- 172.21.15.57:0/355446375 wait complete. 2023-10-06T22:16:53.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.222+0000 7f12b01ff700 1 Processor -- start 2023-10-06T22:16:53.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.223+0000 7f12b01ff700 1 -- start start 2023-10-06T22:16:53.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.223+0000 7f12b01ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a80fff90 0x7f12a8101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.223+0000 7f12b01ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a81050b0 0x7f12a8101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.223+0000 7f12b01ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81024a0 0x7f12a81a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.223+0000 7f12b01ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81a56e0 con 0x7f12a80fff90 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.223+0000 7f12b01ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81a5850 con 0x7f12a81024a0 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.223+0000 7f12b01ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81a5a10 con 0x7f12a81050b0 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a80fff90 0x7f12a8101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a80fff90 0x7f12a8101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36228/0 (socket says 172.21.15.57:36228) 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12adf9b700 1 -- 172.21.15.57:0/892696816 learned_addr learned my addr 172.21.15.57:0/892696816 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ae79c700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81024a0 0x7f12a81a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ad79a700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a81050b0 0x7f12a8101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ae79c700 1 -- 172.21.15.57:0/892696816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a81050b0 msgr2=0x7f12a8101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ae79c700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a81050b0 0x7f12a8101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ae79c700 1 -- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a80fff90 msgr2=0x7f12a8101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ae79c700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a80fff90 0x7f12a8101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ae79c700 1 -- 172.21.15.57:0/892696816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298009520 con 0x7f12a81024a0 2023-10-06T22:16:53.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12adf9b700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a80fff90 0x7f12a8101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:53.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.224+0000 7f12ae79c700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81024a0 0x7f12a81a5050 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12a00062a0 tx=0x7f12a00077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.225+0000 7f129effd700 1 -- 172.21.15.57:0/892696816 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a000e040 con 0x7f12a81024a0 2023-10-06T22:16:53.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.225+0000 7f129effd700 1 -- 172.21.15.57:0/892696816 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12a00094c0 con 0x7f12a81024a0 2023-10-06T22:16:53.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.225+0000 7f129effd700 1 -- 172.21.15.57:0/892696816 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0005040 con 0x7f12a81024a0 2023-10-06T22:16:53.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.225+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a0009d00 con 0x7f12a81024a0 2023-10-06T22:16:53.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.225+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a81a5f80 con 0x7f12a81024a0 2023-10-06T22:16:53.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.229+0000 7f129effd700 1 -- 172.21.15.57:0/892696816 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f12a001d470 con 0x7f12a81024a0 2023-10-06T22:16:53.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.230+0000 7f12c3a75b80 1 Processor -- start 2023-10-06T22:16:53.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.230+0000 7f12c3a75b80 1 -- start start 2023-10-06T22:16:53.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c0c0 0x5593f2c6c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c77350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c79c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2bd5460 con 0x5593f2c77890 2023-10-06T22:16:53.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2bc6c20 con 0x5593f2c6c9e0 2023-10-06T22:16:53.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2bc4540 con 0x5593f2c6c0c0 2023-10-06T22:16:53.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12ad79a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c79c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12ad79a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c79c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36232/0 (socket says 172.21.15.57:36232) 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12ad79a700 1 -- 172.21.15.57:0/4176690965 learned_addr learned my addr 172.21.15.57:0/4176690965 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12adf9b700 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c77350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12ad79a700 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c0c0 0x5593f2c6c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12adf9b700 1 -- 172.21.15.57:0/4176690965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c0c0 msgr2=0x5593f2c6c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12adf9b700 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c0c0 0x5593f2c6c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12adf9b700 1 -- 172.21.15.57:0/4176690965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 msgr2=0x5593f2c79c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12adf9b700 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c79c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.231+0000 7f12adf9b700 1 -- 172.21.15.57:0/4176690965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f29eade0 con 0x5593f2c6c9e0 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f12ad79a700 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c79c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f129effd700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f12940a1740 0x7f12940a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f129effd700 1 -- 172.21.15.57:0/892696816 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f12a00ce560 con 0x7f12a81024a0 2023-10-06T22:16:53.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f12adf9b700 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c77350 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12a4011570 tx=0x7f12a4011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=868bc1b8e88a0008 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f12ad79a700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f12940a1740 0x7f12940a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f1293fff700 1 -- 172.21.15.57:0/4176690965 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4004230 con 0x5593f2c6c9e0 2023-10-06T22:16:53.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f1293fff700 1 -- 172.21.15.57:0/4176690965 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12a4004390 con 0x5593f2c6c9e0 2023-10-06T22:16:53.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.232+0000 7f1293fff700 1 -- 172.21.15.57:0/4176690965 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4019920 con 0x5593f2c6c9e0 2023-10-06T22:16:53.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 -- 172.21.15.57:0/4176690965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 msgr2=0x5593f2c77350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c77350 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12a4011570 tx=0x7f12a4011880 comp rx=0 tx=0).stop 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12ad79a700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f12940a1740 0x7f12940a3c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1298009fd0 tx=0x7f1298009400 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 -- 172.21.15.57:0/4176690965 shutdown_connections 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c79c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c0c0 0x5593f2c6c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/4176690965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c77350 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 -- 172.21.15.57:0/4176690965 >> 172.21.15.57:0/4176690965 conn(0x5593f2be1260 msgr2=0x5593f2c68450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.233+0000 7f12c3a75b80 1 -- 172.21.15.57:0/4176690965 shutdown_connections 2023-10-06T22:16:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.234+0000 7f12c3a75b80 1 -- 172.21.15.57:0/4176690965 wait complete. 2023-10-06T22:16:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.234+0000 7f12c3a75b80 1 -- start start 2023-10-06T22:16:53.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.235+0000 7f12c3a75b80 1 Processor -- start 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.235+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c77290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c9e0 0x5593f2c74510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2c74b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2bbfe30 con 0x5593f2c6c9e0 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2b5c710 con 0x5593f2c6c0c0 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2b5c3b0 con 0x5593f2c77890 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12c3a75b80 10 client.0 _ll_register_callbacks cb 0x5593f29b9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:53.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c77290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c77290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49130/0 (socket says 172.21.15.57:49130) 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 -- 172.21.15.57:0/13423297 learned_addr learned my addr 172.21.15.57:0/13423297 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12ad79a700 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c9e0 0x5593f2c74510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12ae79c700 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2c74b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 -- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 msgr2=0x5593f2c74b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2c74b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 -- 172.21.15.57:0/13423297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c9e0 msgr2=0x5593f2c74510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c9e0 0x5593f2c74510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.236+0000 7f12adf9b700 1 -- 172.21.15.57:0/13423297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129801b040 con 0x5593f2c6c0c0 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.237+0000 7f12adf9b700 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c77290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009880 tx=0x7f12a4005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60aa723729f38dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.237+0000 7f12917fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.237+0000 7f12917fa700 1 -- 172.21.15.57:0/13423297 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4005040 con 0x5593f2c6c0c0 2023-10-06T22:16:53.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.237+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593f2c5ff10 con 0x5593f2c6c0c0 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.237+0000 7f12917fa700 1 -- 172.21.15.57:0/13423297 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f12a40051a0 con 0x5593f2c6c0c0 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.237+0000 7f12917fa700 1 -- 172.21.15.57:0/13423297 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a4005360 con 0x5593f2c6c0c0 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.237+0000 7f12c3a75b80 10 client.26201 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f12a4009d60 con 0x5593f2c6c0c0 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12917fa700 1 -- 172.21.15.57:0/13423297 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593f2c5ff10 con 0x5593f2c6c0c0 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12917fa700 1 -- 172.21.15.57:0/13423297 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f12a4021040 con 0x5593f2c6c0c0 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12c3a75b80 10 client.26201 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12c3a75b80 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12c3a75b80 20 client.26201 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2c981f0 0x5593f2c9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12c3a75b80 4 client.26201 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:53.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.238+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593f2bb8e10 con 0x5593f2c981f0 2023-10-06T22:16:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.239+0000 7f12ad79a700 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2c981f0 0x5593f2c9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.239+0000 7f12ad79a700 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2c981f0 0x5593f2c9a610 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e8e15736a140e5 server_cookie=829645e425363166 in_seq=0 out_seq=0 2023-10-06T22:16:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.239+0000 7f12917fa700 10 client.26201 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12917fa700 1 -- 172.21.15.57:0/13423297 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5593f2bb8e10 con 0x5593f2c981f0 2023-10-06T22:16:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12917fa700 10 client.26201 handle_command_reply: tid=0 2023-10-06T22:16:53.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 client.26201 shutdown 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 msgr2=0x5593f2c77290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c77290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009880 tx=0x7f12a4005e40 comp rx=0 tx=0).stop 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 shutdown_connections 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2c981f0 0x5593f2c9a610 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2c74b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c77290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/13423297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c9e0 0x5593f2c74510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 >> 172.21.15.57:0/13423297 conn(0x5593f2be1260 msgr2=0x5593f2c67fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12917fa700 0 client.26201 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:53.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 shutdown_connections 2023-10-06T22:16:53.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.240+0000 7f12c3a75b80 1 -- 172.21.15.57:0/13423297 wait complete. 2023-10-06T22:16:53.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.241+0000 7f12c3a75b80 20 client.26201 trim_cache size 0 max 16384 2023-10-06T22:16:53.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:52 smithi156 ceph-mon[134358]: pgmap v67: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2023-10-06T22:16:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.275+0000 7f12c3a75b80 1 Processor -- start 2023-10-06T22:16:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.275+0000 7f12c3a75b80 1 -- start start 2023-10-06T22:16:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c76e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c9e0 0x5593f2c73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2a40920 con 0x5593f2c77890 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2c6d2e0 con 0x5593f2c6c0c0 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2c75af0 con 0x5593f2c6c9e0 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c76e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c76e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49144/0 (socket says 172.21.15.57:49144) 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12adf9b700 1 -- 172.21.15.57:0/1786138251 learned_addr learned my addr 172.21.15.57:0/1786138251 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12ad79a700 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c9e0 0x5593f2c73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.276+0000 7f12ae79c700 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12adf9b700 1 -- 172.21.15.57:0/1786138251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c9e0 msgr2=0x5593f2c73cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12adf9b700 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c9e0 0x5593f2c73cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12adf9b700 1 -- 172.21.15.57:0/1786138251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 msgr2=0x5593f2c74210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12adf9b700 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c74210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12adf9b700 1 -- 172.21.15.57:0/1786138251 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f2c64140 con 0x5593f2c6c0c0 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12ad79a700 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c9e0 0x5593f2c73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12ae79c700 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c74210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.277+0000 7f12adf9b700 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c76e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005a80 tx=0x7f12a4009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240028497ca9e9f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.278+0000 7f129cff9700 1 -- 172.21.15.57:0/1786138251 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4019500 con 0x5593f2c6c0c0 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.278+0000 7f129cff9700 1 -- 172.21.15.57:0/1786138251 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12a4003730 con 0x5593f2c6c0c0 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.278+0000 7f129cff9700 1 -- 172.21.15.57:0/1786138251 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4004ad0 con 0x5593f2c6c0c0 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 -- 172.21.15.57:0/1786138251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 msgr2=0x5593f2c76e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c76e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005a80 tx=0x7f12a4009670 comp rx=0 tx=0).stop 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 -- 172.21.15.57:0/1786138251 shutdown_connections 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c77890 0x5593f2c74210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c0c0 0x5593f2c76e30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/1786138251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c6c9e0 0x5593f2c73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 -- 172.21.15.57:0/1786138251 >> 172.21.15.57:0/1786138251 conn(0x5593f2be1260 msgr2=0x5593f2c6f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 -- 172.21.15.57:0/1786138251 shutdown_connections 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 -- 172.21.15.57:0/1786138251 wait complete. 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.279+0000 7f12c3a75b80 1 -- start start 2023-10-06T22:16:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.280+0000 7f12c3a75b80 1 Processor -- start 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.280+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c0c0 0x5593f2c8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.280+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c8d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12c3a75b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2ceb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12c3a75b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2c8dcb0 con 0x5593f2c6c0c0 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c0c0 0x5593f2c8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12adf9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c0c0 0x5593f2c8d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36262/0 (socket says 172.21.15.57:36262) 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12adf9b700 1 -- 172.21.15.57:0/2882977213 learned_addr learned my addr 172.21.15.57:0/2882977213 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2ceb560 con 0x5593f2c6c9e0 2023-10-06T22:16:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f2ceb6a0 con 0x5593f2c77890 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12c3a75b80 10 client.0 _ll_register_callbacks cb 0x7f1298009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.281+0000 7f12ae79c700 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2ceb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12adf9b700 1 -- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 msgr2=0x5593f2ceb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12adf9b700 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2ceb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12adf9b700 1 -- 172.21.15.57:0/2882977213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 msgr2=0x5593f2c8d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12adf9b700 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c8d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12adf9b700 1 -- 172.21.15.57:0/2882977213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129802e9f0 con 0x5593f2c6c0c0 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12adf9b700 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c0c0 0x5593f2c8d090 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12a4011570 tx=0x7f12a4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a79e939499f902 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12917fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12917fa700 1 -- 172.21.15.57:0/2882977213 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40197f0 con 0x5593f2c6c0c0 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593f2bb8b30 con 0x5593f2c6c0c0 2023-10-06T22:16:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12917fa700 1 -- 172.21.15.57:0/2882977213 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f12a4000e40 con 0x5593f2c6c0c0 2023-10-06T22:16:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.282+0000 7f12917fa700 1 -- 172.21.15.57:0/2882977213 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a403edb0 con 0x5593f2c6c0c0 2023-10-06T22:16:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.283+0000 7f12c3a75b80 10 client.16188 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.283+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f12a4009d60 con 0x5593f2c6c0c0 2023-10-06T22:16:53.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.283+0000 7f12917fa700 1 -- 172.21.15.57:0/2882977213 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593f2bb8b30 con 0x5593f2c6c0c0 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.284+0000 7f12917fa700 1 -- 172.21.15.57:0/2882977213 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f12a4021070 con 0x5593f2c6c0c0 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.284+0000 7f12c3a75b80 10 client.16188 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.284+0000 7f12c3a75b80 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.284+0000 7f12c3a75b80 20 client.16188 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.284+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2cec980 0x5593f2ceed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.284+0000 7f12c3a75b80 4 client.16188 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.284+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5593f2a0bf70 con 0x5593f2cec980 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.285+0000 7f12ad79a700 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2cec980 0x5593f2ceed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.285+0000 7f12ad79a700 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2cec980 0x5593f2ceed70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92790cae487d57bb server_cookie=612a09ed993d1310 in_seq=0 out_seq=0 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.285+0000 7f12917fa700 10 client.16188 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.285+0000 7f12917fa700 1 -- 172.21.15.57:0/2882977213 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5593f2a0bf70 con 0x5593f2cec980 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.285+0000 7f12917fa700 10 client.16188 handle_command_reply: tid=0 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.285+0000 7f12c3a75b80 1 client.16188 shutdown 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c0c0 msgr2=0x5593f2c8d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c0c0 0x5593f2c8d090 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12a4011570 tx=0x7f12a4003d50 comp rx=0 tx=0).stop 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 shutdown_connections 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593f2cec980 0x5593f2ceed70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593f2c77890 0x5593f2ceb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593f2c6c0c0 0x5593f2c8d090 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 --2- 172.21.15.57:0/2882977213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f2c6c9e0 0x5593f2c8d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 >> 172.21.15.57:0/2882977213 conn(0x5593f2be1260 msgr2=0x5593f2c67660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12917fa700 0 client.16188 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:53.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 shutdown_connections 2023-10-06T22:16:53.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 1 -- 172.21.15.57:0/2882977213 wait complete. 2023-10-06T22:16:53.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.286+0000 7f12c3a75b80 20 client.16188 trim_cache size 0 max 16384 2023-10-06T22:16:53.292 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:16:53.292 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-06T22:16:53.292 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-06T22:16:53.292 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f12940a1740 msgr2=0x7f12940a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f12940a1740 0x7f12940a3c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1298009fd0 tx=0x7f1298009400 comp rx=0 tx=0).stop 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81024a0 msgr2=0x7f12a81a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81024a0 0x7f12a81a5050 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12a00062a0 tx=0x7f12a00077f0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 shutdown_connections 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f12940a1740 0x7f12940a3c00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a81024a0 0x7f12a81a5050 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f12a80fff90 0x7f12a8101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 --2- 172.21.15.57:0/892696816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a81050b0 0x7f12a8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 >> 172.21.15.57:0/892696816 conn(0x7f12a80fb830 msgr2=0x7f12a80680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 shutdown_connections 2023-10-06T22:16:53.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.287+0000 7f12b01ff700 1 -- 172.21.15.57:0/892696816 wait complete. 2023-10-06T22:16:53.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T22:16:53.317 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:16:53.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:52 smithi190 ceph-mon[134891]: pgmap v67: 97 pgs: 97 active+clean; 100 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2023-10-06T22:16:53.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.529+0000 7f48ad5ff700 1 -- 172.21.15.57:0/988893820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a8101a60 msgr2=0x7f48a8105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.529+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/988893820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a8101a60 0x7f48a8105aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f489c009990 tx=0x7f489c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.529+0000 7f48ad5ff700 1 -- 172.21.15.57:0/988893820 shutdown_connections 2023-10-06T22:16:53.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.529+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/988893820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48a81061d0 0x7f48a810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.529+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/988893820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a81010b0 0x7f48a8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.529+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/988893820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a8101a60 0x7f48a8105aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.529+0000 7f48ad5ff700 1 -- 172.21.15.57:0/988893820 >> 172.21.15.57:0/988893820 conn(0x7f48a80fc8d0 msgr2=0x7f48a80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.530+0000 7f48ad5ff700 1 -- 172.21.15.57:0/988893820 shutdown_connections 2023-10-06T22:16:53.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.530+0000 7f48ad5ff700 1 -- 172.21.15.57:0/988893820 wait complete. 2023-10-06T22:16:53.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 Processor -- start 2023-10-06T22:16:53.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 -- start start 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a81010b0 0x7f48a81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a81061d0 0x7f48a8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48a819da70 0x7f48a819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a819e4f0 con 0x7f48a819da70 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a819e660 con 0x7f48a81010b0 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48ad5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a819e820 con 0x7f48a81061d0 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a81010b0 0x7f48a81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a81010b0 0x7f48a81990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49172/0 (socket says 172.21.15.57:49172) 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48a6ffd700 1 -- 172.21.15.57:0/2600519439 learned_addr learned my addr 172.21.15.57:0/2600519439 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:53.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.531+0000 7f48a77fe700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48a819da70 0x7f48a819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a67fc700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a81061d0 0x7f48a8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a6ffd700 1 -- 172.21.15.57:0/2600519439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a81061d0 msgr2=0x7f48a8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a6ffd700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a81061d0 0x7f48a8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a6ffd700 1 -- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48a819da70 msgr2=0x7f48a819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a6ffd700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48a819da70 0x7f48a819def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a6ffd700 1 -- 172.21.15.57:0/2600519439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0009c40 con 0x7f48a81010b0 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a77fe700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48a819da70 0x7f48a819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48a6ffd700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a81010b0 0x7f48a81990d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9a0 tx=0x7f489800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f488ffff700 1 -- 172.21.15.57:0/2600519439 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898009790 con 0x7f48a81010b0 2023-10-06T22:16:53.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f488ffff700 1 -- 172.21.15.57:0/2600519439 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489800ce80 con 0x7f48a81010b0 2023-10-06T22:16:53.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f488ffff700 1 -- 172.21.15.57:0/2600519439 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898018680 con 0x7f48a81010b0 2023-10-06T22:16:53.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.532+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c0095f0 con 0x7f48a81010b0 2023-10-06T22:16:53.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.533+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a81a60d0 con 0x7f48a81010b0 2023-10-06T22:16:53.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.533+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a8067cd0 con 0x7f48a81010b0 2023-10-06T22:16:53.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.536+0000 7f488ffff700 1 -- 172.21.15.57:0/2600519439 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4898010430 con 0x7f48a81010b0 2023-10-06T22:16:53.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.537+0000 7f488ffff700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f48900a16b0 0x7f48900a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:53.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.537+0000 7f488ffff700 1 -- 172.21.15.57:0/2600519439 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4898014070 con 0x7f48a81010b0 2023-10-06T22:16:53.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.537+0000 7f48a67fc700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f48900a16b0 0x7f48900a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:53.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.538+0000 7f48a67fc700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f48900a16b0 0x7f48900a3b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b2e0 tx=0x7f489c005960 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:53.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.539+0000 7f488ffff700 1 -- 172.21.15.57:0/2600519439 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4898096710 con 0x7f48a81010b0 2023-10-06T22:16:53.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.810+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a8101c60 con 0x7f48a81010b0 2023-10-06T22:16:53.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.811+0000 7f488ffff700 1 -- 172.21.15.57:0/2600519439 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11377 (secure 0 0 0) 0x7f4898099d30 con 0x7f48a81010b0 2023-10-06T22:16:53.814 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:16:53.814 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:16:48.385521+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25784":{"gid":25784,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/1662104434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1662104434},{"type":"v1","addr":"172.21.15.156:6835","nonce":1662104434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:16:53.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f48900a16b0 msgr2=0x7f48900a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f48900a16b0 0x7f48900a3b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b2e0 tx=0x7f489c005960 comp rx=0 tx=0).stop 2023-10-06T22:16:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a81010b0 msgr2=0x7f48a81990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a81010b0 0x7f48a81990d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9a0 tx=0x7f489800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 shutdown_connections 2023-10-06T22:16:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f48900a16b0 0x7f48900a3b70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48a819da70 0x7f48a819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48a81010b0 0x7f48a81990d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 --2- 172.21.15.57:0/2600519439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48a81061d0 0x7f48a8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:53.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 >> 172.21.15.57:0/2600519439 conn(0x7f48a80fc8d0 msgr2=0x7f48a80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:53.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 shutdown_connections 2023-10-06T22:16:53.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:53.814+0000 7f48ad5ff700 1 -- 172.21.15.57:0/2600519439 wait complete. 2023-10-06T22:16:53.818 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2023-10-06T22:16:53.836 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T22:16:54.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.062+0000 7fdba51ff700 1 -- 172.21.15.57:0/2824201576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102030 msgr2=0x7fdba0102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.062+0000 7fdba51ff700 1 --2- 172.21.15.57:0/2824201576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102030 0x7fdba0102410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009990 tx=0x7fdb90009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.062+0000 7fdba51ff700 1 -- 172.21.15.57:0/2824201576 shutdown_connections 2023-10-06T22:16:54.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.062+0000 7fdba51ff700 1 --2- 172.21.15.57:0/2824201576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba010b450 0x7fdba010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.062+0000 7fdba51ff700 1 --2- 172.21.15.57:0/2824201576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102030 0x7fdba0102410 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.062+0000 7fdba51ff700 1 --2- 172.21.15.57:0/2824201576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102950 0x7fdba010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.062+0000 7fdba51ff700 1 -- 172.21.15.57:0/2824201576 >> 172.21.15.57:0/2824201576 conn(0x7fdba00fb830 msgr2=0x7fdba00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.063+0000 7fdba51ff700 1 -- 172.21.15.57:0/2824201576 shutdown_connections 2023-10-06T22:16:54.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.063+0000 7fdba51ff700 1 -- 172.21.15.57:0/2824201576 wait complete. 2023-10-06T22:16:54.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 Processor -- start 2023-10-06T22:16:54.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 -- start start 2023-10-06T22:16:54.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102030 0x7fdba0196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102950 0x7fdba0197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba010b450 0x7fdba019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba019bfa0 con 0x7fdba010b450 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba019c110 con 0x7fdba0102950 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdba51ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba019c2d0 con 0x7fdba0102030 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102030 0x7fdba0196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102030 0x7fdba0196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:34718/0 (socket says 172.21.15.57:34718) 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102950 0x7fdba0197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.064+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/1047893350 learned_addr learned my addr 172.21.15.57:0/1047893350 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba010b450 0x7fdba019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/1047893350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102950 msgr2=0x7fdba0197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102950 0x7fdba0197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba010b450 msgr2=0x7fdba019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba010b450 0x7fdba019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/1047893350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900095f0 con 0x7fdba0102030 2023-10-06T22:16:54.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102030 0x7fdba0196fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005910 tx=0x7fdb90005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb87fff700 1 -- 172.21.15.57:0/1047893350 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9001d070 con 0x7fdba0102030 2023-10-06T22:16:54.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdb87fff700 1 -- 172.21.15.57:0/1047893350 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb90022470 con 0x7fdba0102030 2023-10-06T22:16:54.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.065+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba019c550 con 0x7fdba0102030 2023-10-06T22:16:54.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.066+0000 7fdb87fff700 1 -- 172.21.15.57:0/1047893350 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90017650 con 0x7fdba0102030 2023-10-06T22:16:54.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.066+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba01a3df0 con 0x7fdba0102030 2023-10-06T22:16:54.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.070+0000 7fdb87fff700 1 -- 172.21.15.57:0/1047893350 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdb900225e0 con 0x7fdba0102030 2023-10-06T22:16:54.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.071+0000 7fdbb8aedb80 1 Processor -- start 2023-10-06T22:16:54.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.071+0000 7fdbb8aedb80 1 -- start start 2023-10-06T22:16:54.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a750460 con 0x55ed0a7f2890 2023-10-06T22:16:54.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a741c20 con 0x55ed0a7e70c0 2023-10-06T22:16:54.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a73f540 con 0x55ed0a7e79e0 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdb9f59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdb9f59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7e74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49190/0 (socket says 172.21.15.57:49190) 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdb9f59e700 1 -- 172.21.15.57:0/2523842960 learned_addr learned my addr 172.21.15.57:0/2523842960 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdb87fff700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdb880a15f0 0x7fdb880a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.072+0000 7fdb87fff700 1 -- 172.21.15.57:0/1047893350 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fdb900ce660 con 0x7fdba0102030 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/2523842960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 msgr2=0x55ed0a7e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/2523842960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7f2890 msgr2=0x55ed0a7f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/2523842960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0a565de0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdb880a15f0 0x7fdb880a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f2350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb900cbdc0 tx=0x7fdb90005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f9ed2d4a9e44cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb857fa700 1 -- 172.21.15.57:0/2523842960 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9009b5d0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb857fa700 1 -- 172.21.15.57:0/2523842960 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb900d3070 con 0x55ed0a7e79e0 2023-10-06T22:16:54.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.073+0000 7fdb857fa700 1 -- 172.21.15.57:0/2523842960 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9009f950 con 0x55ed0a7e79e0 2023-10-06T22:16:54.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2523842960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7e79e0 msgr2=0x55ed0a7f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdb880a15f0 0x7fdb880a3ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb980094d0 tx=0x7fdb98004ad0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f2350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb900cbdc0 tx=0x7fdb90005f10 comp rx=0 tx=0).stop 2023-10-06T22:16:54.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2523842960 shutdown_connections 2023-10-06T22:16:54.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2523842960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f2350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2523842960 >> 172.21.15.57:0/2523842960 conn(0x55ed0a75c260 msgr2=0x55ed0a7e3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2523842960 shutdown_connections 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.074+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2523842960 wait complete. 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.075+0000 7fdbb8aedb80 1 -- start start 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.075+0000 7fdbb8aedb80 1 Processor -- start 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.075+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a73ae30 con 0x55ed0a7e70c0 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a6d7710 con 0x55ed0a7e79e0 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a6d73b0 con 0x55ed0a7f2890 2023-10-06T22:16:54.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdbb8aedb80 10 client.0 _ll_register_callbacks cb 0x55ed0a534800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49202/0 (socket says 172.21.15.57:49202) 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 -- 172.21.15.57:0/4000869827 learned_addr learned my addr 172.21.15.57:0/4000869827 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 -- 172.21.15.57:0/4000869827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 msgr2=0x55ed0a7f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 -- 172.21.15.57:0/4000869827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 msgr2=0x55ed0a7f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 -- 172.21.15.57:0/4000869827 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9800f360 con 0x55ed0a7e79e0 2023-10-06T22:16:54.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f75c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb940127f0 tx=0x7fdb94012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d84b146a7129d4a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.076+0000 7fdb82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdb82ff5700 1 -- 172.21.15.57:0/4000869827 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94017070 con 0x55ed0a7e79e0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed0a7daf10 con 0x55ed0a7e79e0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdb82ff5700 1 -- 172.21.15.57:0/4000869827 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fdb9400f4e0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdb82ff5700 1 -- 172.21.15.57:0/4000869827 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb9400f6a0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdbb8aedb80 10 client.26222 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fdba01a3df0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdb82ff5700 1 -- 172.21.15.57:0/4000869827 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed0a7daf10 con 0x55ed0a7e79e0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdb82ff5700 1 -- 172.21.15.57:0/4000869827 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fdb940107e0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.077+0000 7fdbb8aedb80 10 client.26222 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdbb8aedb80 10 client.26222 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:16:54.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdbb8aedb80 20 client.26222 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a806300 0x55ed0a808720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdbb8aedb80 4 client.26222 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:54.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed0a733e10 con 0x55ed0a806300 2023-10-06T22:16:54.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a806300 0x55ed0a808720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a806300 0x55ed0a808720 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c63cc690e730f4bf server_cookie=42cb3fdc506f54b4 in_seq=0 out_seq=0 2023-10-06T22:16:54.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.078+0000 7fdb82ff5700 10 client.26222 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:54.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.079+0000 7fdb82ff5700 1 -- 172.21.15.57:0/4000869827 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed0a733e10 con 0x55ed0a806300 2023-10-06T22:16:54.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.079+0000 7fdb82ff5700 10 client.26222 handle_command_reply: tid=0 2023-10-06T22:16:54.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.079+0000 7fdbb8aedb80 1 client.26222 shutdown 2023-10-06T22:16:54.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 msgr2=0x55ed0a7f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f75c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb940127f0 tx=0x7fdb94012b00 comp rx=0 tx=0).stop 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 shutdown_connections 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a806300 0x55ed0a808720 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a7f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdb82ff5700 0 client.26222 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/4000869827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7f75c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 >> 172.21.15.57:0/4000869827 conn(0x55ed0a75c260 msgr2=0x55ed0a7e2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 shutdown_connections 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/4000869827 wait complete. 2023-10-06T22:16:54.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.080+0000 7fdbb8aedb80 20 client.26222 trim_cache size 0 max 16384 2023-10-06T22:16:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 Processor -- start 2023-10-06T22:16:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 -- start start 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a8025c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a5bb920 con 0x55ed0a7e70c0 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a7fb5f0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.094+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a7ed730 con 0x55ed0a7f2890 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7ec070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49204/0 (socket says 172.21.15.57:49204) 2023-10-06T22:16:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 -- 172.21.15.57:0/3057814611 learned_addr learned my addr 172.21.15.57:0/3057814611 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a8025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 -- 172.21.15.57:0/3057814611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 msgr2=0x55ed0a8025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a8025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 -- 172.21.15.57:0/3057814611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 msgr2=0x55ed0a7fb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7fb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 -- 172.21.15.57:0/3057814611 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0a54ec60 con 0x55ed0a7e79e0 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a8025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.095+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7ec070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb94010b90 tx=0x7fdb94009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc92b0b6d30b766 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdb85ffb700 1 -- 172.21.15.57:0/3057814611 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94017040 con 0x55ed0a7e79e0 2023-10-06T22:16:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdb85ffb700 1 -- 172.21.15.57:0/3057814611 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb940108f0 con 0x55ed0a7e79e0 2023-10-06T22:16:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdb85ffb700 1 -- 172.21.15.57:0/3057814611 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94003900 con 0x55ed0a7e79e0 2023-10-06T22:16:54.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/3057814611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 msgr2=0x55ed0a7ec070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7ec070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb94010b90 tx=0x7fdb94009750 comp rx=0 tx=0).stop 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/3057814611 shutdown_connections 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a8025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e70c0 0x55ed0a7fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/3057814611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e79e0 0x55ed0a7ec070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/3057814611 >> 172.21.15.57:0/3057814611 conn(0x55ed0a75c260 msgr2=0x55ed0a7e1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/3057814611 shutdown_connections 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.096+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/3057814611 wait complete. 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.097+0000 7fdbb8aedb80 1 -- start start 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.097+0000 7fdbb8aedb80 1 Processor -- start 2023-10-06T22:16:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a853370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e79e0 0x55ed0a8538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdbb8aedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a852100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a852640 con 0x55ed0a7e79e0 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a8527b0 con 0x55ed0a7e70c0 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdbb8aedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0a852970 con 0x55ed0a7f2890 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdbb8aedb80 10 client.0 _ll_register_callbacks cb 0x7fdb94009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a853370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e79e0 0x55ed0a8538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a853370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49216/0 (socket says 172.21.15.57:49216) 2023-10-06T22:16:54.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/2722257744 learned_addr learned my addr 172.21.15.57:0/2722257744 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9f59e700 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a852100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/2722257744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 msgr2=0x55ed0a852100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a852100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/2722257744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e79e0 msgr2=0x55ed0a8538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e79e0 0x55ed0a8538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.098+0000 7fdb9ed9d700 1 -- 172.21.15.57:0/2722257744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed0a565de0 con 0x55ed0a7e70c0 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdb9ed9d700 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a853370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb900ce8e0 tx=0x7fdb90009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b568f93da7e54b8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdb82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdb82ff5700 1 -- 172.21.15.57:0/2722257744 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9009b160 con 0x55ed0a7e70c0 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb900d3070 con 0x55ed0a7e70c0 2023-10-06T22:16:54.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdb82ff5700 1 -- 172.21.15.57:0/2722257744 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fdb9009bf00 con 0x55ed0a7e70c0 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdb82ff5700 1 -- 172.21.15.57:0/2722257744 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb90097b40 con 0x55ed0a7e70c0 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdbb8aedb80 10 client.26228 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fdba01a3df0 con 0x55ed0a7e70c0 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdb82ff5700 1 -- 172.21.15.57:0/2722257744 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb90065080 con 0x55ed0a7e70c0 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.099+0000 7fdb82ff5700 1 -- 172.21.15.57:0/2722257744 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fdb900975f0 con 0x55ed0a7e70c0 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdbb8aedb80 10 client.26228 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdbb8aedb80 10 client.26228 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdbb8aedb80 20 client.26228 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a85aeb0 0x55ed0a85d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdbb8aedb80 4 client.26228 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:54.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed0a733b30 con 0x55ed0a85aeb0 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a85aeb0 0x55ed0a85d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdb82ff5700 10 client.26228 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.100+0000 7fdb9e59c700 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a85aeb0 0x55ed0a85d2d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9147b875b221d307 server_cookie=4af20c388639c54e in_seq=0 out_seq=0 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdb82ff5700 1 -- 172.21.15.57:0/2722257744 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed0a733b30 con 0x55ed0a85aeb0 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdb82ff5700 10 client.26228 handle_command_reply: tid=0 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 client.26228 shutdown 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 msgr2=0x55ed0a853370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a853370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb900ce8e0 tx=0x7fdb90009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 shutdown_connections 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ed0a85aeb0 0x55ed0a85d2d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed0a7f2890 0x55ed0a852100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0a7e70c0 0x55ed0a853370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 --2- 172.21.15.57:0/2722257744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed0a7e79e0 0x55ed0a8538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdb82ff5700 0 client.26228 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 >> 172.21.15.57:0/2722257744 conn(0x55ed0a75c260 msgr2=0x55ed0a7ecc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 shutdown_connections 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 1 -- 172.21.15.57:0/2722257744 wait complete. 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.101+0000 7fdbb8aedb80 20 client.26228 trim_cache size 0 max 16384 2023-10-06T22:16:54.109 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:54.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdb880a15f0 msgr2=0x7fdb880a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdb880a15f0 0x7fdb880a3ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb980094d0 tx=0x7fdb98004ad0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102030 msgr2=0x7fdba0196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102030 0x7fdba0196fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005910 tx=0x7fdb90005720 comp rx=0 tx=0).stop 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 shutdown_connections 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdb880a15f0 0x7fdb880a3ab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba010b450 0x7fdba019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102030 0x7fdba0196fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 --2- 172.21.15.57:0/1047893350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdba0102950 0x7fdba0197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 >> 172.21.15.57:0/1047893350 conn(0x7fdba00fb830 msgr2=0x7fdba00fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 shutdown_connections 2023-10-06T22:16:54.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.102+0000 7fdba51ff700 1 -- 172.21.15.57:0/1047893350 wait complete. 2023-10-06T22:16:54.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:53 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2600519439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:16:54.122 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T22:16:54.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:53 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2600519439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:16:54.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:53 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2600519439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:16:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 -- 172.21.15.57:0/2570643543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 msgr2=0x7f36bc0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2570643543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc0736c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009990 tx=0x7f36b8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 -- 172.21.15.57:0/2570643543 shutdown_connections 2023-10-06T22:16:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2570643543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36bc10d680 0x7f36bc10faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2570643543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc0736c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2570643543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f36bc073c00 0x7f36bc10d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 -- 172.21.15.57:0/2570643543 >> 172.21.15.57:0/2570643543 conn(0x7f36bc0fc950 msgr2=0x7f36bc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 -- 172.21.15.57:0/2570643543 shutdown_connections 2023-10-06T22:16:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.341+0000 7f36c43ff700 1 -- 172.21.15.57:0/2570643543 wait complete. 2023-10-06T22:16:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 Processor -- start 2023-10-06T22:16:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 -- start start 2023-10-06T22:16:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f36bc073c00 0x7f36bc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36bc10d680 0x7f36bc19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19e1a0 con 0x7f36bc073c00 2023-10-06T22:16:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19e310 con 0x7f36bc0732e0 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.342+0000 7f36c43ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19e4d0 con 0x7f36bc10d680 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49230/0 (socket says 172.21.15.57:49230) 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 -- 172.21.15.57:0/2770856890 learned_addr learned my addr 172.21.15.57:0/2770856890 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c199a700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f36bc073c00 0x7f36bc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c299c700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36bc10d680 0x7f36bc19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 -- 172.21.15.57:0/2770856890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36bc10d680 msgr2=0x7f36bc19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36bc10d680 0x7f36bc19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 -- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f36bc073c00 msgr2=0x7f36bc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f36bc073c00 0x7f36bc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 -- 172.21.15.57:0/2770856890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac009520 con 0x7f36bc0732e0 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c299c700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36bc10d680 0x7f36bc19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.343+0000 7f36c219b700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc1991c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005910 tx=0x7f36b8004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.344+0000 7f36b37fe700 1 -- 172.21.15.57:0/2770856890 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b801d070 con 0x7f36bc0732e0 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.344+0000 7f36b37fe700 1 -- 172.21.15.57:0/2770856890 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b8022470 con 0x7f36bc0732e0 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.344+0000 7f36b37fe700 1 -- 172.21.15.57:0/2770856890 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8017650 con 0x7f36bc0732e0 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.344+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b80095f0 con 0x7f36bc0732e0 2023-10-06T22:16:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.344+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc1a5e10 con 0x7f36bc0732e0 2023-10-06T22:16:54.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.347+0000 7f36b37fe700 1 -- 172.21.15.57:0/2770856890 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f36b80225e0 con 0x7f36bc0732e0 2023-10-06T22:16:54.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.348+0000 7f36d7dd0b80 1 Processor -- start 2023-10-06T22:16:54.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.348+0000 7f36d7dd0b80 1 -- start start 2023-10-06T22:16:54.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36b37fe700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f36a80a1670 0x7f36a80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36b37fe700 1 -- 172.21.15.57:0/2770856890 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f36b80ce350 con 0x7f36bc0732e0 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e20c0 0x55b9961e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e29e0 0x55b9961ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99614b460 con 0x55b9961e29e0 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99613cc20 con 0x55b9961e20c0 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99613a540 con 0x55b9961ed890 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36c299c700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f36a80a1670 0x7f36a80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36c219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e20c0 0x55b9961e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36c219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e20c0 0x55b9961e24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49232/0 (socket says 172.21.15.57:49232) 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36c219b700 1 -- 172.21.15.57:0/1526332060 learned_addr learned my addr 172.21.15.57:0/1526332060 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36c299c700 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.349+0000 7f36c199a700 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e29e0 0x55b9961ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36c219b700 1 -- 172.21.15.57:0/1526332060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 msgr2=0x55b9961efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36c219b700 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36c219b700 1 -- 172.21.15.57:0/1526332060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e29e0 msgr2=0x55b9961ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36c219b700 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e29e0 0x55b9961ed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36c219b700 1 -- 172.21.15.57:0/1526332060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b995f60de0 con 0x55b9961e20c0 2023-10-06T22:16:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36c299c700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f36a80a1670 0x7f36a80a3b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f36b40094f0 tx=0x7f36b4009240 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36c219b700 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e20c0 0x55b9961e24a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f36b8024000 tx=0x7f36b8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f815ec64f22633 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36b0bfe700 1 -- 172.21.15.57:0/1526332060 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b809bad0 con 0x55b9961e20c0 2023-10-06T22:16:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36b0bfe700 1 -- 172.21.15.57:0/1526332060 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b809b3d0 con 0x55b9961e20c0 2023-10-06T22:16:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.350+0000 7f36b0bfe700 1 -- 172.21.15.57:0/1526332060 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b809f340 con 0x55b9961e20c0 2023-10-06T22:16:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/1526332060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e20c0 msgr2=0x55b9961e24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e20c0 0x55b9961e24a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f36b8024000 tx=0x7f36b8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/1526332060 shutdown_connections 2023-10-06T22:16:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e20c0 0x55b9961e24a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/1526332060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e29e0 0x55b9961ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/1526332060 >> 172.21.15.57:0/1526332060 conn(0x55b996157260 msgr2=0x55b9961de450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/1526332060 shutdown_connections 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.351+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/1526332060 wait complete. 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 -- start start 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 Processor -- start 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961ee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b996135e30 con 0x55b9961e20c0 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9960d2710 con 0x55b9961ed890 2023-10-06T22:16:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9960d23b0 con 0x55b9961e29e0 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.352+0000 7f36d7dd0b80 10 client.0 _ll_register_callbacks cb 0x55b995f2f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c219b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c199a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961ee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c199a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961ee1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:34790/0 (socket says 172.21.15.57:34790) 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c199a700 1 -- 172.21.15.57:0/3042674159 learned_addr learned my addr 172.21.15.57:0/3042674159 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49242/0 (socket says 172.21.15.57:49242) 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 -- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 msgr2=0x55b9961ee1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961ee1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 -- 172.21.15.57:0/3042674159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 msgr2=0x55b9961edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 -- 172.21.15.57:0/3042674159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc1a5e10 con 0x55b9961ed890 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c199a700 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36c299c700 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961ee710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f36b40090f0 tx=0x7f36b4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88725e0b77316954 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.353+0000 7f36a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36a5ffb700 1 -- 172.21.15.57:0/3042674159 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b400e050 con 0x55b9961ed890 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9961d5f10 con 0x55b9961ed890 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36a5ffb700 1 -- 172.21.15.57:0/3042674159 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f36b4021070 con 0x55b9961ed890 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36a5ffb700 1 -- 172.21.15.57:0/3042674159 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b401bdd0 con 0x55b9961ed890 2023-10-06T22:16:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36d7dd0b80 10 client.26240 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b995f49c60 con 0x55b9961ed890 2023-10-06T22:16:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36a5ffb700 1 -- 172.21.15.57:0/3042674159 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b4033020 con 0x55b9961ed890 2023-10-06T22:16:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.354+0000 7f36a5ffb700 1 -- 172.21.15.57:0/3042674159 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f36b401a3e0 con 0x55b9961ed890 2023-10-06T22:16:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36d7dd0b80 10 client.26240 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36d7dd0b80 10 client.26240 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:16:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36d7dd0b80 20 client.26240 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b9961fcf80 0x55b9961ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36d7dd0b80 4 client.26240 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b99612ee10 con 0x55b9961fcf80 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36c219b700 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b9961fcf80 0x55b9961ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36c219b700 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b9961fcf80 0x55b9961ff3a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dea8ef8fcf6f6482 server_cookie=1973a9758b277ee in_seq=0 out_seq=0 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.355+0000 7f36a5ffb700 10 client.26240 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36a5ffb700 1 -- 172.21.15.57:0/3042674159 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b99612ee10 con 0x55b9961fcf80 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36a5ffb700 10 client.26240 handle_command_reply: tid=0 2023-10-06T22:16:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 client.26240 shutdown 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 msgr2=0x55b9961ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961ee710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f36b40090f0 tx=0x7f36b4004b40 comp rx=0 tx=0).stop 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 shutdown_connections 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b9961fcf80 0x55b9961ff3a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961ee710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36a5ffb700 0 client.26240 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3042674159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.356+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 >> 172.21.15.57:0/3042674159 conn(0x55b996157260 msgr2=0x55b9961ddfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.357+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 shutdown_connections 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.357+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3042674159 wait complete. 2023-10-06T22:16:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.357+0000 7f36d7dd0b80 20 client.26240 trim_cache size 0 max 16384 2023-10-06T22:16:54.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 Processor -- start 2023-10-06T22:16:54.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 -- start start 2023-10-06T22:16:54.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961ee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961e7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b995fb6920 con 0x55b9961e20c0 2023-10-06T22:16:54.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9961e3020 con 0x55b9961ed890 2023-10-06T22:16:54.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.371+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9961f98f0 con 0x55b9961e29e0 2023-10-06T22:16:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c219b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961e7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49258/0 (socket says 172.21.15.57:49258) 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 -- 172.21.15.57:0/3724780582 learned_addr learned my addr 172.21.15.57:0/3724780582 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c199a700 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 -- 172.21.15.57:0/3724780582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 msgr2=0x55b9961f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 -- 172.21.15.57:0/3724780582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 msgr2=0x55b9961ee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961ee380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 -- 172.21.15.57:0/3724780582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9961ec570 con 0x55b9961ed890 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36c299c700 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961e7020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36b4004b10 tx=0x7f36b40098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d3d85684f181e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.372+0000 7f36b13ff700 1 -- 172.21.15.57:0/3724780582 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b400e040 con 0x55b9961ed890 2023-10-06T22:16:54.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36b13ff700 1 -- 172.21.15.57:0/3724780582 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b4033040 con 0x55b9961ed890 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36b13ff700 1 -- 172.21.15.57:0/3724780582 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b402f920 con 0x55b9961ed890 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3724780582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 msgr2=0x55b9961e7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961e7020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36b4004b10 tx=0x7f36b40098a0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3724780582 shutdown_connections 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961ed890 0x55b9961e7020 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/3724780582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961e29e0 0x55b9961f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3724780582 >> 172.21.15.57:0/3724780582 conn(0x55b996157260 msgr2=0x55b9961edc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3724780582 shutdown_connections 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.373+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/3724780582 wait complete. 2023-10-06T22:16:54.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.374+0000 7f36d7dd0b80 1 -- start start 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.374+0000 7f36d7dd0b80 1 Processor -- start 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e29e0 0x55b9961f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36d7dd0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9961f10b0 con 0x55b9961e20c0 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9961f1220 con 0x55b9961e29e0 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36d7dd0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9961f13e0 con 0x55b9961ed890 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36d7dd0b80 10 client.0 _ll_register_callbacks cb 0x7f36b4019c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e29e0 0x55b9961f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c219b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e29e0 0x55b9961f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49264/0 (socket says 172.21.15.57:49264) 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 -- 172.21.15.57:0/245577746 learned_addr learned my addr 172.21.15.57:0/245577746 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c299c700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 -- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 msgr2=0x55b9961f0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961f0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 -- 172.21.15.57:0/245577746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 msgr2=0x55b9961f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c199a700 1 -- 172.21.15.57:0/245577746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b995f49c60 con 0x55b9961e29e0 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c299c700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.375+0000 7f36c219b700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36c199a700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e29e0 0x55b9961f2ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36ac005e10 tx=0x7f36ac010f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1c3215d7ca290f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36a5ffb700 1 -- 172.21.15.57:0/245577746 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac017040 con 0x55b9961e29e0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36b809b3d0 con 0x55b9961e29e0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36a5ffb700 1 -- 172.21.15.57:0/245577746 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f36ac004500 con 0x55b9961e29e0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36a5ffb700 1 -- 172.21.15.57:0/245577746 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36ac00f800 con 0x55b9961e29e0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36d7dd0b80 10 client.26246 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b995f87250 con 0x55b9961e29e0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36a5ffb700 1 -- 172.21.15.57:0/245577746 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ac02b020 con 0x55b9961e29e0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36a5ffb700 1 -- 172.21.15.57:0/245577746 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f36ac003cc0 con 0x55b9961e29e0 2023-10-06T22:16:54.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36d7dd0b80 10 client.26246 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36d7dd0b80 10 client.26246 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.376+0000 7f36d7dd0b80 20 client.26246 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.377+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b996252020 0x55b996254440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.377+0000 7f36d7dd0b80 4 client.26246 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.377+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b99612eb30 con 0x55b996252020 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.377+0000 7f36c219b700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b996252020 0x55b996254440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.377+0000 7f36c219b700 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b996252020 0x55b996254440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3c0b2a2912096a3 server_cookie=9f8682b6f1c7dd13 in_seq=0 out_seq=0 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.377+0000 7f36a5ffb700 10 client.26246 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36a5ffb700 1 -- 172.21.15.57:0/245577746 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b99612eb30 con 0x55b996252020 2023-10-06T22:16:54.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36a5ffb700 10 client.26246 handle_command_reply: tid=0 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 client.26246 shutdown 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e29e0 msgr2=0x55b9961f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e29e0 0x55b9961f2ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36ac005e10 tx=0x7f36ac010f40 comp rx=0 tx=0).stop 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 shutdown_connections 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x55b996252020 0x55b996254440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9961ed890 0x55b9961f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9961e20c0 0x55b9961f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 --2- 172.21.15.57:0/245577746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b9961e29e0 0x55b9961f2ab0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36a5ffb700 0 client.26246 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 >> 172.21.15.57:0/245577746 conn(0x55b996157260 msgr2=0x55b9961eba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 shutdown_connections 2023-10-06T22:16:54.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 1 -- 172.21.15.57:0/245577746 wait complete. 2023-10-06T22:16:54.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.378+0000 7f36d7dd0b80 20 client.26246 trim_cache size 0 max 16384 2023-10-06T22:16:54.388 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:54.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f36a80a1670 msgr2=0x7f36a80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f36a80a1670 0x7f36a80a3b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f36b40094f0 tx=0x7f36b4009240 comp rx=0 tx=0).stop 2023-10-06T22:16:54.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 msgr2=0x7f36bc1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc1991c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005910 tx=0x7f36b8004710 comp rx=0 tx=0).stop 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 shutdown_connections 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f36a80a1670 0x7f36a80a3b30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36bc10d680 0x7f36bc19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36bc0732e0 0x7f36bc1991c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 --2- 172.21.15.57:0/2770856890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f36bc073c00 0x7f36bc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 >> 172.21.15.57:0/2770856890 conn(0x7f36bc0fc950 msgr2=0x7f36bc10f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 shutdown_connections 2023-10-06T22:16:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.379+0000 7f36c43ff700 1 -- 172.21.15.57:0/2770856890 wait complete. 2023-10-06T22:16:54.400 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T22:16:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.627+0000 7f5a4659d700 1 -- 172.21.15.57:0/1700568382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a40105c70 msgr2=0x7f5a40106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.627+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1700568382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a40105c70 0x7f5a40106050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009990 tx=0x7f5a3c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.628+0000 7f5a4659d700 1 -- 172.21.15.57:0/1700568382 shutdown_connections 2023-10-06T22:16:54.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.628+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1700568382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a401007f0 0x7f5a4010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.628+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1700568382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a40105c70 0x7f5a40106050 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.628+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1700568382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a400ffe30 0x7f5a401002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.628+0000 7f5a4659d700 1 -- 172.21.15.57:0/1700568382 >> 172.21.15.57:0/1700568382 conn(0x7f5a400769d0 msgr2=0x7f5a40076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.628+0000 7f5a4659d700 1 -- 172.21.15.57:0/1700568382 shutdown_connections 2023-10-06T22:16:54.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.628+0000 7f5a4659d700 1 -- 172.21.15.57:0/1700568382 wait complete. 2023-10-06T22:16:54.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.629+0000 7f5a4659d700 1 Processor -- start 2023-10-06T22:16:54.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.629+0000 7f5a4659d700 1 -- start start 2023-10-06T22:16:54.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.629+0000 7f5a4659d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a400ffe30 0x7f5a40101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.629+0000 7f5a4659d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a401007f0 0x7f5a40101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.629+0000 7f5a4659d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a40105c70 0x7f5a401a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4659d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a401025a0 con 0x7f5a400ffe30 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4659d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a401a5420 con 0x7f5a401007f0 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4659d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a401a5560 con 0x7f5a40105c70 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a400ffe30 0x7f5a40101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a44d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a401007f0 0x7f5a40101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a400ffe30 0x7f5a40101830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36382/0 (socket says 172.21.15.57:36382) 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 -- 172.21.15.57:0/1898671477 learned_addr learned my addr 172.21.15.57:0/1898671477 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a40105c70 0x7f5a401a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 -- 172.21.15.57:0/1898671477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a40105c70 msgr2=0x7f5a401a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a40105c70 0x7f5a401a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 -- 172.21.15.57:0/1898671477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a401007f0 msgr2=0x7f5a40101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a401007f0 0x7f5a40101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a4559b700 1 -- 172.21.15.57:0/1898671477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30009520 con 0x7f5a400ffe30 2023-10-06T22:16:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a401007f0 0x7f5a40101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.630+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a40105c70 0x7f5a401a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.631+0000 7f5a4559b700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a400ffe30 0x7f5a40101830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005910 tx=0x7f5a3c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.631+0000 7f5a367fc700 1 -- 172.21.15.57:0/1898671477 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c01d070 con 0x7f5a400ffe30 2023-10-06T22:16:54.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.631+0000 7f5a367fc700 1 -- 172.21.15.57:0/1898671477 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a3c022470 con 0x7f5a400ffe30 2023-10-06T22:16:54.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.631+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c0095f0 con 0x7f5a400ffe30 2023-10-06T22:16:54.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.631+0000 7f5a367fc700 1 -- 172.21.15.57:0/1898671477 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c017650 con 0x7f5a400ffe30 2023-10-06T22:16:54.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.631+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a401a59f0 con 0x7f5a400ffe30 2023-10-06T22:16:54.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.635+0000 7f5a367fc700 1 -- 172.21.15.57:0/1898671477 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5a3c0225e0 con 0x7f5a400ffe30 2023-10-06T22:16:54.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.636+0000 7f5a367fc700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5a2c0a1690 0x7f5a2c0a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.636+0000 7f5a5b0f5b80 1 Processor -- start 2023-10-06T22:16:54.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.636+0000 7f5a367fc700 1 -- 172.21.15.57:0/1898671477 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f5a3c0ce660 con 0x7f5a400ffe30 2023-10-06T22:16:54.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a5b0f5b80 1 -- start start 2023-10-06T22:16:54.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5a2c0a1690 0x7f5a2c0a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37dd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa373b460 con 0x558aa37d22a0 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa372cc20 con 0x558aa37dda70 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa372a540 con 0x558aa37d2bc0 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a45d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a45d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37dfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49308/0 (socket says 172.21.15.57:49308) 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.637+0000 7f5a45d9c700 1 -- 172.21.15.57:0/931879931 learned_addr learned my addr 172.21.15.57:0/931879931 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5a2c0a1690 0x7f5a2c0a3b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009fd0 tx=0x7f5a30009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a45d9c700 1 -- 172.21.15.57:0/931879931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 msgr2=0x558aa37dd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37dd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a45d9c700 1 -- 172.21.15.57:0/931879931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 msgr2=0x558aa37d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a45d9c700 1 -- 172.21.15.57:0/931879931 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558aa3550de0 con 0x558aa37dda70 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37dfe60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a38003e40 tx=0x7f5a38009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e46547b9d91fbae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a2b7fe700 1 -- 172.21.15.57:0/931879931 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380092d0 con 0x558aa37dda70 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a2b7fe700 1 -- 172.21.15.57:0/931879931 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a3800f040 con 0x558aa37dda70 2023-10-06T22:16:54.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.638+0000 7f5a2b7fe700 1 -- 172.21.15.57:0/931879931 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800e750 con 0x558aa37dda70 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/931879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 msgr2=0x558aa37dfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37dfe60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a38003e40 tx=0x7f5a38009920 comp rx=0 tx=0).stop 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/931879931 shutdown_connections 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37dfe60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/931879931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/931879931 >> 172.21.15.57:0/931879931 conn(0x558aa3747260 msgr2=0x558aa37ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/931879931 shutdown_connections 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.639+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/931879931 wait complete. 2023-10-06T22:16:54.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.640+0000 7f5a5b0f5b80 1 -- start start 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.640+0000 7f5a5b0f5b80 1 Processor -- start 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d22a0 0x558aa37da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37d2bc0 0x558aa37dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37dda70 0x558aa37db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa3725e30 con 0x558aa37dda70 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa36c2710 con 0x558aa37d2bc0 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa36c23b0 con 0x558aa37d22a0 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a5b0f5b80 10 client.0 _ll_register_callbacks cb 0x558aa351f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a44d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37d2bc0 0x558aa37dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d22a0 0x558aa37da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.641+0000 7f5a44d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37d2bc0 0x558aa37dad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49322/0 (socket says 172.21.15.57:49322) 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a44d9a700 1 -- 172.21.15.57:0/4148650275 learned_addr learned my addr 172.21.15.57:0/4148650275 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37dda70 0x558aa37db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a4559b700 1 -- 172.21.15.57:0/4148650275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37d2bc0 msgr2=0x558aa37dad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a4559b700 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37d2bc0 0x558aa37dad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a4559b700 1 -- 172.21.15.57:0/4148650275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37dda70 msgr2=0x558aa37db290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a4559b700 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37dda70 0x558aa37db290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a4559b700 1 -- 172.21.15.57:0/4148650275 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a38003b20 con 0x558aa37d22a0 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a4559b700 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d22a0 0x558aa37da810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0cbdc0 tx=0x7f5a3c09b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=226a9f5e8260136b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a28ff9700 1 -- 172.21.15.57:0/4148650275 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c09aef0 con 0x558aa37d22a0 2023-10-06T22:16:54.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a28ff9700 1 -- 172.21.15.57:0/4148650275 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f5a3c09b800 con 0x558aa37d22a0 2023-10-06T22:16:54.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.642+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558aa37c6160 con 0x558aa37d22a0 2023-10-06T22:16:54.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a28ff9700 1 -- 172.21.15.57:0/4148650275 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a3c09f950 con 0x558aa37d22a0 2023-10-06T22:16:54.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 10 client.26248 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558aa3539c60 con 0x558aa37d22a0 2023-10-06T22:16:54.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a28ff9700 1 -- 172.21.15.57:0/4148650275 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558aa37c6160 con 0x558aa37d22a0 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a28ff9700 1 -- 172.21.15.57:0/4148650275 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5a3c09cb30 con 0x558aa37d22a0 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 10 client.26248 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 10 client.26248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 20 client.26248 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa37ed290 0x558aa37ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 4 client.26248 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.643+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558aa371ee10 con 0x558aa37ed290 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.644+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa37ed290 0x558aa37ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.644+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa37ed290 0x558aa37ef6b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb9a03ec8a89f57c server_cookie=b571ea2a3dc43718 in_seq=0 out_seq=0 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.644+0000 7f5a28ff9700 10 client.26248 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a28ff9700 1 -- 172.21.15.57:0/4148650275 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558aa371ee10 con 0x558aa37ed290 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a28ff9700 10 client.26248 handle_command_reply: tid=0 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 client.26248 shutdown 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d22a0 msgr2=0x558aa37da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d22a0 0x558aa37da810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0cbdc0 tx=0x7f5a3c09b160 comp rx=0 tx=0).stop 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 shutdown_connections 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa37ed290 0x558aa37ef6b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37dda70 0x558aa37db290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d22a0 0x558aa37da810 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a28ff9700 0 client.26248 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:54.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/4148650275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37d2bc0 0x558aa37dad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 >> 172.21.15.57:0/4148650275 conn(0x558aa3747260 msgr2=0x558aa37ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 shutdown_connections 2023-10-06T22:16:54.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.645+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/4148650275 wait complete. 2023-10-06T22:16:54.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.646+0000 7f5a5b0f5b80 20 client.26248 trim_cache size 0 max 16384 2023-10-06T22:16:54.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.661+0000 7f5a5b0f5b80 1 Processor -- start 2023-10-06T22:16:54.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.661+0000 7f5a5b0f5b80 1 -- start start 2023-10-06T22:16:54.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37d7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa35a6920 con 0x558aa37d22a0 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa37ed0b0 con 0x558aa37dda70 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa37c5ca0 con 0x558aa37d2bc0 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a45d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37d7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36402/0 (socket says 172.21.15.57:36402) 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a4559b700 1 -- 172.21.15.57:0/819927242 learned_addr learned my addr 172.21.15.57:0/819927242 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.662+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a45d9c700 1 -- 172.21.15.57:0/819927242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 msgr2=0x558aa37d9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37d9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a45d9c700 1 -- 172.21.15.57:0/819927242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 msgr2=0x558aa37d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a45d9c700 1 -- 172.21.15.57:0/819927242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558aa3577250 con 0x558aa37dda70 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37d7b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009e50 tx=0x7f5a38006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d117a1398adc5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a2bfff700 1 -- 172.21.15.57:0/819927242 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380092d0 con 0x558aa37dda70 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a2bfff700 1 -- 172.21.15.57:0/819927242 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a38007500 con 0x558aa37dda70 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.663+0000 7f5a2bfff700 1 -- 172.21.15.57:0/819927242 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800e750 con 0x558aa37dda70 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/819927242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 msgr2=0x558aa37d7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37d7b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009e50 tx=0x7f5a38006000 comp rx=0 tx=0).stop 2023-10-06T22:16:54.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/819927242 shutdown_connections 2023-10-06T22:16:54.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37d7b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/819927242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37d9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/819927242 >> 172.21.15.57:0/819927242 conn(0x558aa3747260 msgr2=0x558aa37d9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/819927242 shutdown_connections 2023-10-06T22:16:54.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.664+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/819927242 wait complete. 2023-10-06T22:16:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 -- start start 2023-10-06T22:16:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 Processor -- start 2023-10-06T22:16:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37e2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa37e4bb0 con 0x558aa37d22a0 2023-10-06T22:16:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa37e30b0 con 0x558aa37dda70 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.665+0000 7f5a5b0f5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558aa37e3270 con 0x558aa37d2bc0 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a5b0f5b80 10 client.0 _ll_register_callbacks cb 0x7f5a3c09b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a45d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36412/0 (socket says 172.21.15.57:36412) 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 -- 172.21.15.57:0/2769009374 learned_addr learned my addr 172.21.15.57:0/2769009374 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a44d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 -- 172.21.15.57:0/2769009374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 msgr2=0x558aa37e2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37e2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 -- 172.21.15.57:0/2769009374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 msgr2=0x558aa37e2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37e2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 -- 172.21.15.57:0/2769009374 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558aa3550de0 con 0x558aa37d22a0 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a45d9c700 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a4559b700 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37e42e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c024000 tx=0x7f5a3c09cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8931c4ae200a3c91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.666+0000 7f5a28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a28ff9700 1 -- 172.21.15.57:0/2769009374 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c09b800 con 0x558aa37d22a0 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a28ff9700 1 -- 172.21.15.57:0/2769009374 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f5a3c09f950 con 0x558aa37d22a0 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a3800f040 con 0x558aa37d22a0 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a28ff9700 1 -- 172.21.15.57:0/2769009374 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a3c0a3b00 con 0x558aa37d22a0 2023-10-06T22:16:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a5b0f5b80 10 client.16248 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558aa354f2b0 con 0x558aa37d22a0 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a28ff9700 1 -- 172.21.15.57:0/2769009374 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a3800f040 con 0x558aa37d22a0 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a28ff9700 1 -- 172.21.15.57:0/2769009374 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f5a3c09a760 con 0x558aa37d22a0 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a5b0f5b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a5b0f5b80 10 client.16248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a5b0f5b80 20 client.16248 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.667+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa384a750 0x558aa384cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.668+0000 7f5a5b0f5b80 4 client.16248 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.668+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558aa371eb30 con 0x558aa384a750 2023-10-06T22:16:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.668+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa384a750 0x558aa384cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.668+0000 7f5a44d9a700 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa384a750 0x558aa384cb40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7c195bb0d96b252 server_cookie=b73e0cea388e8470 in_seq=0 out_seq=0 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.668+0000 7f5a28ff9700 10 client.16248 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a28ff9700 1 -- 172.21.15.57:0/2769009374 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558aa371eb30 con 0x558aa384a750 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a28ff9700 10 client.16248 handle_command_reply: tid=0 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 client.16248 shutdown 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 msgr2=0x558aa37e42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37e42e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c024000 tx=0x7f5a3c09cd40 comp rx=0 tx=0).stop 2023-10-06T22:16:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 shutdown_connections 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x558aa384a750 0x558aa384cb40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558aa37dda70 0x558aa37e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a28ff9700 0 client.16248 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aa37d22a0 0x558aa37e42e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 --2- 172.21.15.57:0/2769009374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558aa37d2bc0 0x558aa37e2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 >> 172.21.15.57:0/2769009374 conn(0x558aa3747260 msgr2=0x558aa37db040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 shutdown_connections 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 1 -- 172.21.15.57:0/2769009374 wait complete. 2023-10-06T22:16:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.669+0000 7f5a5b0f5b80 20 client.16248 trim_cache size 0 max 16384 2023-10-06T22:16:54.678 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5a2c0a1690 msgr2=0x7f5a2c0a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5a2c0a1690 0x7f5a2c0a3b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009fd0 tx=0x7f5a30009260 comp rx=0 tx=0).stop 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a400ffe30 msgr2=0x7f5a40101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a400ffe30 0x7f5a40101830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005910 tx=0x7f5a3c005720 comp rx=0 tx=0).stop 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 shutdown_connections 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5a2c0a1690 0x7f5a2c0a3b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a40105c70 0x7f5a401a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a400ffe30 0x7f5a40101830 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 --2- 172.21.15.57:0/1898671477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a401007f0 0x7f5a40101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 >> 172.21.15.57:0/1898671477 conn(0x7f5a400769d0 msgr2=0x7f5a400fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 shutdown_connections 2023-10-06T22:16:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.670+0000 7f5a4659d700 1 -- 172.21.15.57:0/1898671477 wait complete. 2023-10-06T22:16:54.691 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 -- 172.21.15.57:0/348024109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14040732e0 msgr2=0x7f14040736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 --2- 172.21.15.57:0/348024109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14040732e0 0x7f14040736c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009990 tx=0x7f13f4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 -- 172.21.15.57:0/348024109 shutdown_connections 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 --2- 172.21.15.57:0/348024109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f140410d680 0x7f140410faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 --2- 172.21.15.57:0/348024109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14040732e0 0x7f14040736c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 --2- 172.21.15.57:0/348024109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 0x7f140410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 -- 172.21.15.57:0/348024109 >> 172.21.15.57:0/348024109 conn(0x7f14040fc930 msgr2=0x7f14040fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 -- 172.21.15.57:0/348024109 shutdown_connections 2023-10-06T22:16:54.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.924+0000 7f1409bff700 1 -- 172.21.15.57:0/348024109 wait complete. 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.925+0000 7f1409bff700 1 Processor -- start 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1409bff700 1 -- start start 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1409bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14040732e0 0x7f14041991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1409bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 0x7f1404199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1409bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f140410d680 0x7f140419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1409bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140419e1a0 con 0x7f14040732e0 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1409bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140419e310 con 0x7f140410d680 2023-10-06T22:16:54.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1409bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140419e4d0 con 0x7f1404073c00 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f14037fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14040732e0 0x7f14041991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1402ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 0x7f1404199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1402ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 0x7f1404199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:34896/0 (socket says 172.21.15.57:34896) 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f14037fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14040732e0 0x7f14041991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36434/0 (socket says 172.21.15.57:36434) 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1402ffd700 1 -- 172.21.15.57:0/2651192591 learned_addr learned my addr 172.21.15.57:0/2651192591 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.926+0000 7f1403fff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f140410d680 0x7f140419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f14037fe700 1 -- 172.21.15.57:0/2651192591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 msgr2=0x7f1404199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f14037fe700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 0x7f1404199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f14037fe700 1 -- 172.21.15.57:0/2651192591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f140410d680 msgr2=0x7f140419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f14037fe700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f140410d680 0x7f140419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f14037fe700 1 -- 172.21.15.57:0/2651192591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40095f0 con 0x7f14040732e0 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f1402ffd700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 0x7f1404199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f14037fe700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14040732e0 0x7f14041991c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009990 tx=0x7f13f4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.927+0000 7f1400ff9700 1 -- 172.21.15.57:0/2651192591 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f401d070 con 0x7f14040732e0 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.928+0000 7f1400ff9700 1 -- 172.21.15.57:0/2651192591 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f4021410 con 0x7f14040732e0 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.928+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140419e750 con 0x7f14040732e0 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.928+0000 7f1400ff9700 1 -- 172.21.15.57:0/2651192591 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4017650 con 0x7f14040732e0 2023-10-06T22:16:54.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.928+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14041a5ff0 con 0x7f14040732e0 2023-10-06T22:16:54.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.932+0000 7f1400ff9700 1 -- 172.21.15.57:0/2651192591 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f13f4021810 con 0x7f14040732e0 2023-10-06T22:16:54.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.933+0000 7f141d573b80 1 Processor -- start 2023-10-06T22:16:54.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f141d573b80 1 -- start start 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f141d573b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01820c0 0x5653d01824a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f141d573b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f141d573b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d018d890 0x5653d018fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f141d573b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d00eb460 con 0x5653d01820c0 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f141d573b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d00dcc20 con 0x5653d01829e0 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f141d573b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d00da540 con 0x5653d018d890 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f14037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f1402ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d018d890 0x5653d018fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f14037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49352/0 (socket says 172.21.15.57:49352) 2023-10-06T22:16:54.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f1403fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01820c0 0x5653d01824a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.934+0000 7f14037fe700 1 -- 172.21.15.57:0/64479011 learned_addr learned my addr 172.21.15.57:0/64479011 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f1400ff9700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13ec0a1670 0x7f13ec0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f14037fe700 1 -- 172.21.15.57:0/64479011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d018d890 msgr2=0x5653d018fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f14037fe700 1 --2- 172.21.15.57:0/64479011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d018d890 0x5653d018fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f14037fe700 1 -- 172.21.15.57:0/64479011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01820c0 msgr2=0x5653d01824a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f14037fe700 1 --2- 172.21.15.57:0/64479011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01820c0 0x5653d01824a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f14037fe700 1 -- 172.21.15.57:0/64479011 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653cff00de0 con 0x5653d01829e0 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f1400ff9700 1 -- 172.21.15.57:0/2651192591 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f13f40ce330 con 0x7f14040732e0 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f1403fff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13ec0a1670 0x7f13ec0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f14037fe700 1 --2- 172.21.15.57:0/64479011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018d350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13f4097e20 tx=0x7f13f402d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4c13e3a62f41dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.935+0000 7f13e9ffb700 1 -- 172.21.15.57:0/64479011 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f409aef0 con 0x5653d01829e0 2023-10-06T22:16:54.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.936+0000 7f13e9ffb700 1 -- 172.21.15.57:0/64479011 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f409ad10 con 0x5653d01829e0 2023-10-06T22:16:54.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.936+0000 7f13e9ffb700 1 -- 172.21.15.57:0/64479011 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f409bab0 con 0x5653d01829e0 2023-10-06T22:16:54.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.936+0000 7f141d573b80 1 -- 172.21.15.57:0/64479011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 msgr2=0x5653d018d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.936+0000 7f1403fff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13ec0a1670 0x7f13ec0a3b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009ec0 tx=0x7f13fc009040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.936+0000 7f141d573b80 1 --2- 172.21.15.57:0/64479011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018d350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13f4097e20 tx=0x7f13f402d000 comp rx=0 tx=0).stop 2023-10-06T22:16:54.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 -- 172.21.15.57:0/64479011 shutdown_connections 2023-10-06T22:16:54.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 --2- 172.21.15.57:0/64479011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d018d890 0x5653d018fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 --2- 172.21.15.57:0/64479011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01820c0 0x5653d01824a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 --2- 172.21.15.57:0/64479011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018d350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 -- 172.21.15.57:0/64479011 >> 172.21.15.57:0/64479011 conn(0x5653d00f7260 msgr2=0x5653d017e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 -- 172.21.15.57:0/64479011 shutdown_connections 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 -- 172.21.15.57:0/64479011 wait complete. 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 -- start start 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.937+0000 7f141d573b80 1 Processor -- start 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f141d573b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d018dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f141d573b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f141d573b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d018d890 0x5653d018e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f141d573b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d00d5e30 con 0x5653d018d890 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f141d573b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d0072710 con 0x5653d01829e0 2023-10-06T22:16:54.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f141d573b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d00723b0 con 0x5653d01820c0 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f141d573b80 10 client.0 _ll_register_callbacks cb 0x5653cfecf800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f1403fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d018d890 0x5653d018e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f1402ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f1403fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d018d890 0x5653d018e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36454/0 (socket says 172.21.15.57:36454) 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f1403fff700 1 -- 172.21.15.57:0/4075285487 learned_addr learned my addr 172.21.15.57:0/4075285487 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.938+0000 7f14037fe700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d018dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f1403fff700 1 -- 172.21.15.57:0/4075285487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 msgr2=0x5653d018dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f1403fff700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d018dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f1403fff700 1 -- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 msgr2=0x5653d018e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f1403fff700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f1403fff700 1 -- 172.21.15.57:0/4075285487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14041a5ff0 con 0x5653d018d890 2023-10-06T22:16:54.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f1402ffd700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f14037fe700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d018dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:54.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f13e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:16:54.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.939+0000 7f1403fff700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d018d890 0x5653d018e6f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f13fc018cc0 tx=0x7f13fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e39f81fc6f18bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.940+0000 7f13e2ffd700 1 -- 172.21.15.57:0/4075285487 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc01c040 con 0x5653d018d890 2023-10-06T22:16:54.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.940+0000 7f13e2ffd700 1 -- 172.21.15.57:0/4075285487 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f13fc016860 con 0x5653d018d890 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.940+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653d0175f10 con 0x5653d018d890 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.941+0000 7f13e2ffd700 1 -- 172.21.15.57:0/4075285487 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13fc00e040 con 0x5653d018d890 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.941+0000 7f141d573b80 10 client.16260 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.941+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5653cfee9c60 con 0x5653d018d890 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f13e2ffd700 1 -- 172.21.15.57:0/4075285487 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653d0175f10 con 0x5653d018d890 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f13e2ffd700 1 -- 172.21.15.57:0/4075285487 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f13fc02a070 con 0x5653d018d890 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f141d573b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f141d573b80 10 client.16260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f141d573b80 20 client.16260 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f141d573b80 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d019cfc0 0x5653d019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f141d573b80 4 client.16260 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:54.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d00cee10 con 0x5653d019cfc0 2023-10-06T22:16:54.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.942+0000 7f14037fe700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d019cfc0 0x5653d019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.943+0000 7f14037fe700 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d019cfc0 0x5653d019f3e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5539a44921e5c13 server_cookie=796af814d5e039dc in_seq=0 out_seq=0 2023-10-06T22:16:54.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.943+0000 7f13e2ffd700 10 client.16260 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:54.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f13e2ffd700 1 -- 172.21.15.57:0/4075285487 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5653d00cee10 con 0x5653d019cfc0 2023-10-06T22:16:54.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f13e2ffd700 10 client.16260 handle_command_reply: tid=0 2023-10-06T22:16:54.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 client.16260 shutdown 2023-10-06T22:16:54.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d018d890 msgr2=0x5653d018e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d018d890 0x5653d018e6f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f13fc018cc0 tx=0x7f13fc004ad0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 shutdown_connections 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d019cfc0 0x5653d019f3e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d018d890 0x5653d018e6f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d018dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f13e2ffd700 0 client.16260 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 --2- 172.21.15.57:0/4075285487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d01829e0 0x5653d018e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 >> 172.21.15.57:0/4075285487 conn(0x5653d00f7260 msgr2=0x5653d017dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.947+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 shutdown_connections 2023-10-06T22:16:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.948+0000 7f141d573b80 1 -- 172.21.15.57:0/4075285487 wait complete. 2023-10-06T22:16:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.948+0000 7f141d573b80 20 client.16260 trim_cache size 0 max 16384 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.975+0000 7f141d573b80 1 Processor -- start 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.975+0000 7f141d573b80 1 -- start start 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.975+0000 7f141d573b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d0191ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.975+0000 7f141d573b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d018c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.975+0000 7f141d573b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d017dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.975+0000 7f141d573b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653cff56920 con 0x5653d01829e0 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f141d573b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d018ade0 con 0x5653d018d890 2023-10-06T22:16:54.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f141d573b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d017d240 con 0x5653d01820c0 2023-10-06T22:16:54.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d0191ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d0191ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:34934/0 (socket says 172.21.15.57:34934) 2023-10-06T22:16:54.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 -- 172.21.15.57:0/2474203183 learned_addr learned my addr 172.21.15.57:0/2474203183 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f1403fff700 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d017dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f1402ffd700 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d018c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 -- 172.21.15.57:0/2474203183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 msgr2=0x5653d017dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d017dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 -- 172.21.15.57:0/2474203183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 msgr2=0x5653d018c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d018c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 -- 172.21.15.57:0/2474203183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d017a140 con 0x5653d01820c0 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f1403fff700 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d017dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.976+0000 7f14037fe700 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d0191ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13f409b340 tx=0x7f13f40d1e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a9869df8e3e52d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.977+0000 7f13ea7fc700 1 -- 172.21.15.57:0/2474203183 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4056360 con 0x5653d01820c0 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.977+0000 7f13ea7fc700 1 -- 172.21.15.57:0/2474203183 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f409bab0 con 0x5653d01820c0 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.977+0000 7f13ea7fc700 1 -- 172.21.15.57:0/2474203183 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f409f500 con 0x5653d01820c0 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 -- 172.21.15.57:0/2474203183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 msgr2=0x5653d0191ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d0191ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13f409b340 tx=0x7f13f40d1e90 comp rx=0 tx=0).stop 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 -- 172.21.15.57:0/2474203183 shutdown_connections 2023-10-06T22:16:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d017dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d0191ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 --2- 172.21.15.57:0/2474203183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d018c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 -- 172.21.15.57:0/2474203183 >> 172.21.15.57:0/2474203183 conn(0x5653d00f7260 msgr2=0x5653d017d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 -- 172.21.15.57:0/2474203183 shutdown_connections 2023-10-06T22:16:54.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.978+0000 7f141d573b80 1 -- 172.21.15.57:0/2474203183 wait complete. 2023-10-06T22:16:54.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.979+0000 7f141d573b80 1 -- start start 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.979+0000 7f141d573b80 1 Processor -- start 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.979+0000 7f141d573b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d01e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.979+0000 7f141d573b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d01e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.979+0000 7f141d573b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d01e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.979+0000 7f141d573b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d01eff10 con 0x5653d01829e0 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f141d573b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d01f0050 con 0x5653d018d890 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f141d573b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d01f0210 con 0x5653d01820c0 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f141d573b80 10 client.0 _ll_register_callbacks cb 0x7f13fc0188a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d01e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d01e4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:34944/0 (socket says 172.21.15.57:34944) 2023-10-06T22:16:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 -- 172.21.15.57:0/1622383347 learned_addr learned my addr 172.21.15.57:0/1622383347 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f1403fff700 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d01e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f1402ffd700 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d01e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 -- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 msgr2=0x5653d01e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d01e57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 -- 172.21.15.57:0/1622383347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 msgr2=0x5653d01e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d01e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 -- 172.21.15.57:0/1622383347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653cff00de0 con 0x5653d01820c0 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f14037fe700 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d01e4d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f13f4028000 tx=0x7f13f4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee8ca834eefac1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.980+0000 7f13e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f13e2ffd700 1 -- 172.21.15.57:0/1622383347 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4056360 con 0x5653d01820c0 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f13e2ffd700 1 -- 172.21.15.57:0/1622383347 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f13f409f500 con 0x5653d01820c0 2023-10-06T22:16:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13f409ad10 con 0x5653d01820c0 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f13e2ffd700 1 -- 172.21.15.57:0/1622383347 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13f409b3b0 con 0x5653d01820c0 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 10 client.26272 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5653d0199910 con 0x5653d01820c0 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f13e2ffd700 1 -- 172.21.15.57:0/1622383347 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f409ad10 con 0x5653d01820c0 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f13e2ffd700 1 -- 172.21.15.57:0/1622383347 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f13f409a8a0 con 0x5653d01820c0 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 10 client.26272 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 10 client.26272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 20 client.26272 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d01f1560 0x5653d01f3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 4 client.26272 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:54.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.981+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5653d00ceb30 con 0x5653d01f1560 2023-10-06T22:16:54.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.982+0000 7f1402ffd700 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d01f1560 0x5653d01f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:54.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.982+0000 7f1402ffd700 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d01f1560 0x5653d01f3980 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2815a06aa780d42 server_cookie=9ada453c7488635f in_seq=0 out_seq=0 2023-10-06T22:16:54.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.982+0000 7f13e2ffd700 10 client.26272 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f13e2ffd700 1 -- 172.21.15.57:0/1622383347 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5653d00ceb30 con 0x5653d01f1560 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f13e2ffd700 10 client.26272 handle_command_reply: tid=0 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f141d573b80 1 client.26272 shutdown 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 msgr2=0x5653d01e4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f141d573b80 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d01e4d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f13f4028000 tx=0x7f13f4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 shutdown_connections 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f141d573b80 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x5653d01f1560 0x5653d01f3980 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f141d573b80 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653d018d890 0x5653d01e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.985+0000 7f141d573b80 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d01820c0 0x5653d01e4d60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.986+0000 7f141d573b80 1 --2- 172.21.15.57:0/1622383347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5653d01829e0 0x5653d01e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.986+0000 7f13e2ffd700 0 client.26272 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:16:54.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.986+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 >> 172.21.15.57:0/1622383347 conn(0x5653d00f7260 msgr2=0x5653d017def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.986+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 shutdown_connections 2023-10-06T22:16:54.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.986+0000 7f141d573b80 1 -- 172.21.15.57:0/1622383347 wait complete. 2023-10-06T22:16:54.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.986+0000 7f141d573b80 20 client.26272 trim_cache size 0 max 16384 2023-10-06T22:16:54.992 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:54.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13ec0a1670 msgr2=0x7f13ec0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13ec0a1670 0x7f13ec0a3b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009ec0 tx=0x7f13fc009040 comp rx=0 tx=0).stop 2023-10-06T22:16:54.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14040732e0 msgr2=0x7f14041991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14040732e0 0x7f14041991c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009990 tx=0x7f13f4009580 comp rx=0 tx=0).stop 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 shutdown_connections 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13ec0a1670 0x7f13ec0a3b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f140410d680 0x7f140419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14040732e0 0x7f14041991c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 --2- 172.21.15.57:0/2651192591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1404073c00 0x7f1404199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 >> 172.21.15.57:0/2651192591 conn(0x7f14040fc930 msgr2=0x7f140410f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 shutdown_connections 2023-10-06T22:16:54.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:54.987+0000 7f1409bff700 1 -- 172.21.15.57:0/2651192591 wait complete. 2023-10-06T22:16:55.006 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:54 smithi057 ceph-mon[98492]: pgmap v68: 97 pgs: 97 active+clean; 171 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-10-06T22:16:55.008 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 -- 172.21.15.57:0/1141023968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 msgr2=0x7fe49010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 --2- 172.21.15.57:0/1141023968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe49010dd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe488009990 tx=0x7fe488009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 -- 172.21.15.57:0/1141023968 shutdown_connections 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 --2- 172.21.15.57:0/1141023968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe49010dd50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 --2- 172.21.15.57:0/1141023968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4901010b0 0x7fe490101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 --2- 172.21.15.57:0/1141023968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe490101a60 0x7fe490105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 -- 172.21.15.57:0/1141023968 >> 172.21.15.57:0/1141023968 conn(0x7fe4900fc950 msgr2=0x7fe4900fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 -- 172.21.15.57:0/1141023968 shutdown_connections 2023-10-06T22:16:55.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.242+0000 7fe4983ff700 1 -- 172.21.15.57:0/1141023968 wait complete. 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.243+0000 7fe4983ff700 1 Processor -- start 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe4983ff700 1 -- start start 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe4983ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4901010b0 0x7fe490199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe4983ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe490199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe4983ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe49019dac0 0x7fe49019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe4983ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49019e540 con 0x7fe4901010b0 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe4983ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49019e6b0 con 0x7fe4901061d0 2023-10-06T22:16:55.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe4983ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49019e870 con 0x7fe49019dac0 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe49619b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4901010b0 0x7fe490199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe49599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe490199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe49599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe490199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49408/0 (socket says 172.21.15.57:49408) 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe49619b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4901010b0 0x7fe490199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36484/0 (socket says 172.21.15.57:36484) 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.244+0000 7fe49599a700 1 -- 172.21.15.57:0/401358053 learned_addr learned my addr 172.21.15.57:0/401358053 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49619b700 1 -- 172.21.15.57:0/401358053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe49019dac0 msgr2=0x7fe49019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49699c700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe49019dac0 0x7fe49019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49619b700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe49019dac0 0x7fe49019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49619b700 1 -- 172.21.15.57:0/401358053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 msgr2=0x7fe490199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49619b700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe490199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49619b700 1 -- 172.21.15.57:0/401358053 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480009520 con 0x7fe4901010b0 2023-10-06T22:16:55.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49599a700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe490199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:55.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49699c700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe49019dac0 0x7fe49019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:16:55.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.245+0000 7fe49619b700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4901010b0 0x7fe490199120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00ef10 tx=0x7fe48c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:55.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.246+0000 7fe4877fe700 1 -- 172.21.15.57:0/401358053 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c00ca70 con 0x7fe4901010b0 2023-10-06T22:16:55.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.246+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4880095f0 con 0x7fe4901010b0 2023-10-06T22:16:55.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.246+0000 7fe4877fe700 1 -- 172.21.15.57:0/401358053 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe48c00f040 con 0x7fe4901010b0 2023-10-06T22:16:55.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.246+0000 7fe4877fe700 1 -- 172.21.15.57:0/401358053 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0146e0 con 0x7fe4901010b0 2023-10-06T22:16:55.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.246+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4901a6110 con 0x7fe4901010b0 2023-10-06T22:16:55.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.250+0000 7fe4877fe700 1 -- 172.21.15.57:0/401358053 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe48c007500 con 0x7fe4901010b0 2023-10-06T22:16:55.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.251+0000 7fe4abd25b80 1 Processor -- start 2023-10-06T22:16:55.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:54 smithi156 ceph-mon[134358]: pgmap v68: 97 pgs: 97 active+clean; 171 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-10-06T22:16:55.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.251+0000 7fe4abd25b80 1 -- start start 2023-10-06T22:16:55.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e42c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878d9e460 con 0x560878e40890 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878d8fc20 con 0x560878e350c0 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878d8d540 con 0x560878e359e0 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe49699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe49699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49412/0 (socket says 172.21.15.57:49412) 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe49699c700 1 -- 172.21.15.57:0/1528018097 learned_addr learned my addr 172.21.15.57:0/1528018097 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.252+0000 7fe49619b700 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49599a700 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e42c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe4877fe700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe47c0a1670 0x7fe47c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49699c700 1 -- 172.21.15.57:0/1528018097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 msgr2=0x560878e40350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49699c700 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e40350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49699c700 1 -- 172.21.15.57:0/1528018097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 msgr2=0x560878e42c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49699c700 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e42c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49699c700 1 -- 172.21.15.57:0/1528018097 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560878bb3de0 con 0x560878e350c0 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49599a700 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e42c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe4877fe700 1 -- 172.21.15.57:0/401358053 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fe48c0cd430 con 0x7fe4901010b0 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.253+0000 7fe49699c700 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e354a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe488005910 tx=0x7fe488005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ac2e1b355dcdb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:55.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.254+0000 7fe49699c700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe47c0a1670 0x7fe47c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.254+0000 7fe484bfe700 1 -- 172.21.15.57:0/1528018097 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48801e070 con 0x560878e350c0 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.254+0000 7fe484bfe700 1 -- 172.21.15.57:0/1528018097 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe48800bdf0 con 0x560878e350c0 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.254+0000 7fe484bfe700 1 -- 172.21.15.57:0/1528018097 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe488022450 con 0x560878e350c0 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1528018097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 msgr2=0x560878e354a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe49699c700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe47c0a1670 0x7fe47c0a3b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe48800b6f0 tx=0x7fe488022ec0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e354a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe488005910 tx=0x7fe488005c70 comp rx=0 tx=0).stop 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1528018097 shutdown_connections 2023-10-06T22:16:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e42c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e354a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1528018097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e40350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1528018097 >> 172.21.15.57:0/1528018097 conn(0x560878daa260 msgr2=0x560878e31450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1528018097 shutdown_connections 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1528018097 wait complete. 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.255+0000 7fe4abd25b80 1 -- start start 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 1 Processor -- start 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e40250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e359e0 0x560878e3d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e40890 0x560878e3db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878d88e30 con 0x560878e350c0 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878d25710 con 0x560878e359e0 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878d253b0 con 0x560878e40890 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.256+0000 7fe4abd25b80 10 client.0 _ll_register_callbacks cb 0x560878b82800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e359e0 0x560878e3d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49619b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e40250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e359e0 0x560878e3d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49418/0 (socket says 172.21.15.57:49418) 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49599a700 1 -- 172.21.15.57:0/2559517025 learned_addr learned my addr 172.21.15.57:0/2559517025 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49699c700 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e40890 0x560878e3db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49619b700 1 -- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e40890 msgr2=0x560878e3db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49619b700 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e40890 0x560878e3db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49619b700 1 -- 172.21.15.57:0/2559517025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e359e0 msgr2=0x560878e3d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49619b700 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e359e0 0x560878e3d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49619b700 1 -- 172.21.15.57:0/2559517025 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49010e5c0 con 0x560878e350c0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49599a700 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e359e0 0x560878e3d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe49619b700 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e40250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0cab90 tx=0x7fe48c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378a0f5a33c47bee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe479ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.257+0000 7fe479ffb700 1 -- 172.21.15.57:0/2559517025 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0cb010 con 0x560878e350c0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560878e28f10 con 0x560878e350c0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe479ffb700 1 -- 172.21.15.57:0/2559517025 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fe48c007e40 con 0x560878e350c0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe479ffb700 1 -- 172.21.15.57:0/2559517025 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe48c09a1f0 con 0x560878e350c0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 10 client.16278 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560878b9cc60 con 0x560878e350c0 2023-10-06T22:16:55.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe479ffb700 1 -- 172.21.15.57:0/2559517025 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560878e28f10 con 0x560878e350c0 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe479ffb700 1 -- 172.21.15.57:0/2559517025 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe48c09e100 con 0x560878e350c0 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 20 client.16278 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878e50350 0x560878e52770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 4 client.16278 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.258+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560878d81e10 con 0x560878e50350 2023-10-06T22:16:55.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.259+0000 7fe49599a700 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878e50350 0x560878e52770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.259+0000 7fe49599a700 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878e50350 0x560878e52770 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730adf33d15ddf48 server_cookie=bf4aa12e7c103459 in_seq=0 out_seq=0 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.259+0000 7fe479ffb700 10 client.16278 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.260+0000 7fe479ffb700 1 -- 172.21.15.57:0/2559517025 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560878d81e10 con 0x560878e50350 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.260+0000 7fe479ffb700 10 client.16278 handle_command_reply: tid=0 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.260+0000 7fe4abd25b80 1 client.16278 shutdown 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.260+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 msgr2=0x560878e40250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.260+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e40250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0cab90 tx=0x7fe48c00cbd0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 shutdown_connections 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878e50350 0x560878e52770 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e40890 0x560878e3db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e40250 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/2559517025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e359e0 0x560878e3d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 >> 172.21.15.57:0/2559517025 conn(0x560878daa260 msgr2=0x560878e30fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:55.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe479ffb700 0 client.16278 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:55.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 shutdown_connections 2023-10-06T22:16:55.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 1 -- 172.21.15.57:0/2559517025 wait complete. 2023-10-06T22:16:55.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.261+0000 7fe4abd25b80 20 client.16278 trim_cache size 0 max 16384 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.277+0000 7fe4abd25b80 1 Processor -- start 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.277+0000 7fe4abd25b80 1 -- start start 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e3cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e3d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878c09920 con 0x560878e40890 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878e3bab0 con 0x560878e350c0 2023-10-06T22:16:55.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878e393c0 con 0x560878e359e0 2023-10-06T22:16:55.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe49599a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e3cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe49599a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e3cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:34986/0 (socket says 172.21.15.57:34986) 2023-10-06T22:16:55.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe49599a700 1 -- 172.21.15.57:0/3876564431 learned_addr learned my addr 172.21.15.57:0/3876564431 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:55.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.278+0000 7fe49699c700 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e3d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe49599a700 1 -- 172.21.15.57:0/3876564431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 msgr2=0x560878e3fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe49599a700 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe49599a700 1 -- 172.21.15.57:0/3876564431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 msgr2=0x560878e3d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe49599a700 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e3d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe49599a700 1 -- 172.21.15.57:0/3876564431 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560878e2d140 con 0x560878e359e0 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe49699c700 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e3d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe49599a700 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e3cd10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x560878e3ea80 tx=0x7fe480009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0a55874ec3ce5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe4853ff700 1 -- 172.21.15.57:0/3876564431 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480016040 con 0x560878e359e0 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe4853ff700 1 -- 172.21.15.57:0/3876564431 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe480017040 con 0x560878e359e0 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.279+0000 7fe4853ff700 1 -- 172.21.15.57:0/3876564431 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480003bb0 con 0x560878e359e0 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 -- 172.21.15.57:0/3876564431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 msgr2=0x560878e3cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e3cd10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x560878e3ea80 tx=0x7fe480009260 comp rx=0 tx=0).stop 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 -- 172.21.15.57:0/3876564431 shutdown_connections 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e40890 0x560878e3d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e350c0 0x560878e3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/3876564431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e3cd10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 -- 172.21.15.57:0/3876564431 >> 172.21.15.57:0/3876564431 conn(0x560878daa260 msgr2=0x560878e388b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 -- 172.21.15.57:0/3876564431 shutdown_connections 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.280+0000 7fe4abd25b80 1 -- 172.21.15.57:0/3876564431 wait complete. 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 -- start start 2023-10-06T22:16:55.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 Processor -- start 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e45360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e40890 0x560878e45de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878ea3180 con 0x560878e350c0 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878ea32c0 con 0x560878e40890 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.281+0000 7fe4abd25b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560878ea3480 con 0x560878e359e0 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe4abd25b80 10 client.0 _ll_register_callbacks cb 0x7fe48c00ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e40890 0x560878e45de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e40890 0x560878e45de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49432/0 (socket says 172.21.15.57:49432) 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 -- 172.21.15.57:0/1875052517 learned_addr learned my addr 172.21.15.57:0/1875052517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49619b700 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e45360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49599a700 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 -- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 msgr2=0x560878e458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 -- 172.21.15.57:0/1875052517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 msgr2=0x560878e45360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e45360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 -- 172.21.15.57:0/1875052517 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4800118d0 con 0x560878e40890 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe49699c700 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e40890 0x560878e45de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe488006010 tx=0x7fe488022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0490b165607b6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.282+0000 7fe479ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe479ffb700 1 -- 172.21.15.57:0/1875052517 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48801e040 con 0x560878e40890 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe48800bdf0 con 0x560878e40890 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe479ffb700 1 -- 172.21.15.57:0/1875052517 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fe48800b2c0 con 0x560878e40890 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe479ffb700 1 -- 172.21.15.57:0/1875052517 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe48800b8a0 con 0x560878e40890 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 10 client.26297 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560878bda250 con 0x560878e40890 2023-10-06T22:16:55.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe479ffb700 1 -- 172.21.15.57:0/1875052517 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe48800bdf0 con 0x560878e40890 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe479ffb700 1 -- 172.21.15.57:0/1875052517 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe4880179c0 con 0x560878e40890 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 10 client.26297 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 20 client.26297 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878ea4860 0x560878ea6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 4 client.26297 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.283+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560878d81b30 con 0x560878ea4860 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.284+0000 7fe49619b700 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878ea4860 0x560878ea6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:55.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.284+0000 7fe479ffb700 10 client.26297 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:55.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.284+0000 7fe49619b700 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878ea4860 0x560878ea6c80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3e173dcdd7d39f server_cookie=a7b567944e7365ff in_seq=0 out_seq=0 2023-10-06T22:16:55.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.284+0000 7fe479ffb700 1 -- 172.21.15.57:0/1875052517 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560878d81b30 con 0x560878ea4860 2023-10-06T22:16:55.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.284+0000 7fe479ffb700 10 client.26297 handle_command_reply: tid=0 2023-10-06T22:16:55.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.284+0000 7fe4abd25b80 1 client.26297 shutdown 2023-10-06T22:16:55.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e40890 msgr2=0x560878e45de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e40890 0x560878e45de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe488006010 tx=0x7fe488022000 comp rx=0 tx=0).stop 2023-10-06T22:16:55.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 shutdown_connections 2023-10-06T22:16:55.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x560878ea4860 0x560878ea6c80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560878e40890 0x560878e45de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560878e350c0 0x560878e45360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 --2- 172.21.15.57:0/1875052517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560878e359e0 0x560878e458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe479ffb700 0 client.26297 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 >> 172.21.15.57:0/1875052517 conn(0x560878daa260 msgr2=0x560878e30660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 shutdown_connections 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 1 -- 172.21.15.57:0/1875052517 wait complete. 2023-10-06T22:16:55.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.285+0000 7fe4abd25b80 20 client.26297 trim_cache size 0 max 16384 2023-10-06T22:16:55.296 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:16:55.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe47c0a1670 msgr2=0x7fe47c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe47c0a1670 0x7fe47c0a3b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe48800b6f0 tx=0x7fe488022ec0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4901010b0 msgr2=0x7fe490199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4901010b0 0x7fe490199120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00ef10 tx=0x7fe48c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 shutdown_connections 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe47c0a1670 0x7fe47c0a3b30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe49019dac0 0x7fe49019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4901010b0 0x7fe490199120 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 --2- 172.21.15.57:0/401358053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4901061d0 0x7fe490199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 >> 172.21.15.57:0/401358053 conn(0x7fe4900fc950 msgr2=0x7fe4900fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 shutdown_connections 2023-10-06T22:16:55.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:55.286+0000 7fe4983ff700 1 -- 172.21.15.57:0/401358053 wait complete. 2023-10-06T22:16:55.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:54 smithi190 ceph-mon[134891]: pgmap v68: 97 pgs: 97 active+clean; 171 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-10-06T22:16:56.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:55 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:16:56.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:55 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:16:56.309 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T22:16:56.310 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T22:16:56.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:55 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:16:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 -- 172.21.15.57:0/2630925095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab4102910 msgr2=0x7fcab410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/2630925095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab4102910 0x7fcab410adf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009990 tx=0x7fcaa8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 -- 172.21.15.57:0/2630925095 shutdown_connections 2023-10-06T22:16:56.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/2630925095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcab410b410 0x7fcab410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/2630925095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 0x7fcab41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/2630925095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab4102910 0x7fcab410adf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 -- 172.21.15.57:0/2630925095 >> 172.21.15.57:0/2630925095 conn(0x7fcab40fb830 msgr2=0x7fcab40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 -- 172.21.15.57:0/2630925095 shutdown_connections 2023-10-06T22:16:56.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.550+0000 7fcaba7ff700 1 -- 172.21.15.57:0/2630925095 wait complete. 2023-10-06T22:16:56.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 Processor -- start 2023-10-06T22:16:56.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 -- start start 2023-10-06T22:16:56.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 0x7fcab40ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcab4102910 0x7fcab4100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab410b410 0x7fcab41a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab41009b0 con 0x7fcab4101ff0 2023-10-06T22:16:56.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab41a56a0 con 0x7fcab410b410 2023-10-06T22:16:56.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcaba7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab41a57e0 con 0x7fcab4102910 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.552+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 0x7fcab40ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 0x7fcab40ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36534/0 (socket says 172.21.15.57:36534) 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 -- 172.21.15.57:0/1524692572 learned_addr learned my addr 172.21.15.57:0/1524692572 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab410b410 0x7fcab41a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab37fe700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcab4102910 0x7fcab4100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 -- 172.21.15.57:0/1524692572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcab4102910 msgr2=0x7fcab4100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcab4102910 0x7fcab4100180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 -- 172.21.15.57:0/1524692572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab410b410 msgr2=0x7fcab41a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab410b410 0x7fcab41a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 -- 172.21.15.57:0/1524692572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80095f0 con 0x7fcab4101ff0 2023-10-06T22:16:56.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab3fff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 0x7fcab40ffc40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400e9a0 tx=0x7fcaa400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab17fa700 1 -- 172.21.15.57:0/1524692572 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4009790 con 0x7fcab4101ff0 2023-10-06T22:16:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab17fa700 1 -- 172.21.15.57:0/1524692572 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa4010430 con 0x7fcab4101ff0 2023-10-06T22:16:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab41a5a40 con 0x7fcab4101ff0 2023-10-06T22:16:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.553+0000 7fcab17fa700 1 -- 172.21.15.57:0/1524692572 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4018680 con 0x7fcab4101ff0 2023-10-06T22:16:56.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.554+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab41a5f20 con 0x7fcab4101ff0 2023-10-06T22:16:56.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.557+0000 7fcab17fa700 1 -- 172.21.15.57:0/1524692572 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcaa40105a0 con 0x7fcab4101ff0 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.558+0000 7fcace1cdb80 1 Processor -- start 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcab17fa700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fca9c0a15f0 0x7fca9c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcace1cdb80 1 -- start start 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcab17fa700 1 -- 172.21.15.57:0/1524692572 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fcaa4014070 con 0x7fcab4101ff0 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fca9c0a15f0 0x7fca9c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabcf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabda3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabdcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fab384e0 con 0x5647fabcf140 2023-10-06T22:16:56.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fab29ca0 con 0x5647fabcfa60 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fab275c0 con 0x5647fabda910 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabda3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabcf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabda3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49456/0 (socket says 172.21.15.57:49456) 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.559+0000 7fcab37fe700 1 -- 172.21.15.57:0/2856055160 learned_addr learned my addr 172.21.15.57:0/2856055160 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabdcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab37fe700 1 -- 172.21.15.57:0/2856055160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 msgr2=0x5647fabdcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab37fe700 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabdcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab37fe700 1 -- 172.21.15.57:0/2856055160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 msgr2=0x5647fabcf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab37fe700 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabcf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab37fe700 1 -- 172.21.15.57:0/2856055160 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647fa94de60 con 0x5647fabcfa60 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabdcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fca9c0a15f0 0x7fca9c0a3ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab4101d50 tx=0x7fcaac00b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fcab37fe700 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabda3d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8015000 tx=0x7fcaa8005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b382fb796a8ea2f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.560+0000 7fca9a7fc700 1 -- 172.21.15.57:0/2856055160 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa801d070 con 0x5647fabcfa60 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fca9a7fc700 1 -- 172.21.15.57:0/2856055160 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa801e040 con 0x5647fabcfa60 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fca9a7fc700 1 -- 172.21.15.57:0/2856055160 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8004880 con 0x5647fabcfa60 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2856055160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 msgr2=0x5647fabda3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabda3d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8015000 tx=0x7fcaa8005ea0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2856055160 shutdown_connections 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabdcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabcf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2856055160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabda3d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.561+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2856055160 >> 172.21.15.57:0/2856055160 conn(0x5647fab442e0 msgr2=0x5647fabcb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.562+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2856055160 shutdown_connections 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.562+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2856055160 wait complete. 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.562+0000 7fcace1cdb80 1 -- start start 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.562+0000 7fcace1cdb80 1 Processor -- start 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabda300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcfa60 0x5647fabd7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fab22eb0 con 0x5647fabcfa60 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647faabf790 con 0x5647fabcf140 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647faabf430 con 0x5647fabda910 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcace1cdb80 10 client.0 _ll_register_callbacks cb 0x5647fa91c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabda300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabda300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49462/0 (socket says 172.21.15.57:49462) 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcab3fff700 1 -- 172.21.15.57:0/3780860333 learned_addr learned my addr 172.21.15.57:0/3780860333 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcab37fe700 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcfa60 0x5647fabd7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.563+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcab3fff700 1 -- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 msgr2=0x5647fabd7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcab3fff700 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcab3fff700 1 -- 172.21.15.57:0/3780860333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcfa60 msgr2=0x5647fabd7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcab3fff700 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcfa60 0x5647fabd7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcab3fff700 1 -- 172.21.15.57:0/3780860333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab410e590 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcab3fff700 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabda300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4096790 tx=0x7fcaa409aa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3225f2a16a217dec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fca937fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fca937fe700 1 -- 172.21.15.57:0/3780860333 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4094ea0 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fca937fe700 1 -- 172.21.15.57:0/3780860333 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fcaa409e3e0 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647fabc2f90 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fca937fe700 1 -- 172.21.15.57:0/3780860333 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa40a2620 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcace1cdb80 10 client.26309 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5647fa936ce0 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.564+0000 7fca937fe700 1 -- 172.21.15.57:0/3780860333 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647fabc2f90 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fca937fe700 1 -- 172.21.15.57:0/3780860333 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fcaa40a2190 con 0x5647fabcf140 2023-10-06T22:16:56.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fcace1cdb80 10 client.26309 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:56.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fcace1cdb80 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:56.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fcace1cdb80 20 client.26309 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:56.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fabea2b0 0x5647fabec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fcace1cdb80 4 client.26309 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:56.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647fab1be90 con 0x5647fabea2b0 2023-10-06T22:16:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.565+0000 7fcab37fe700 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fabea2b0 0x5647fabec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.566+0000 7fcab37fe700 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fabea2b0 0x5647fabec6d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebc9d0246e0082e server_cookie=d6bfa55625047b46 in_seq=0 out_seq=0 2023-10-06T22:16:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.566+0000 7fca937fe700 10 client.26309 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.566+0000 7fca937fe700 1 -- 172.21.15.57:0/3780860333 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647fab1be90 con 0x5647fabea2b0 2023-10-06T22:16:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.566+0000 7fca937fe700 10 client.26309 handle_command_reply: tid=0 2023-10-06T22:16:56.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.566+0000 7fcace1cdb80 1 client.26309 shutdown 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 msgr2=0x5647fabda300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabda300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4096790 tx=0x7fcaa409aa80 comp rx=0 tx=0).stop 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 shutdown_connections 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fabea2b0 0x5647fabec6d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabda300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fca937fe700 0 client.26309 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/3780860333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcfa60 0x5647fabd7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 >> 172.21.15.57:0/3780860333 conn(0x5647fab442e0 msgr2=0x5647fabcb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 shutdown_connections 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 1 -- 172.21.15.57:0/3780860333 wait complete. 2023-10-06T22:16:56.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.567+0000 7fcace1cdb80 20 client.26309 trim_cache size 0 max 16384 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.585+0000 7fcace1cdb80 1 Processor -- start 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.585+0000 7fcace1cdb80 1 -- start start 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabe13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabd9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fa9a39a0 con 0x5647fabcf140 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fabd0040 con 0x5647fabcfa60 2023-10-06T22:16:56.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fabd8120 con 0x5647fabda910 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabe13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabd9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabd9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49474/0 (socket says 172.21.15.57:49474) 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab37fe700 1 -- 172.21.15.57:0/1721526761 learned_addr learned my addr 172.21.15.57:0/1721526761 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab37fe700 1 -- 172.21.15.57:0/1721526761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 msgr2=0x5647fabd27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab37fe700 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.586+0000 7fcab37fe700 1 -- 172.21.15.57:0/1721526761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 msgr2=0x5647fabe13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.587+0000 7fcab37fe700 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabe13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.587+0000 7fcab37fe700 1 -- 172.21.15.57:0/1721526761 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647fabf1300 con 0x5647fabcfa60 2023-10-06T22:16:56.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.587+0000 7fcab3fff700 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabe13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:56.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.587+0000 7fcab37fe700 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabd9e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80057b0 tx=0x7fcaa8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1676afd83a4af6c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.587+0000 7fca9affd700 1 -- 172.21.15.57:0/1721526761 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8037070 con 0x5647fabcfa60 2023-10-06T22:16:56.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.587+0000 7fca9affd700 1 -- 172.21.15.57:0/1721526761 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa8033370 con 0x5647fabcfa60 2023-10-06T22:16:56.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.587+0000 7fca9affd700 1 -- 172.21.15.57:0/1721526761 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa801d040 con 0x5647fabcfa60 2023-10-06T22:16:56.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 -- 172.21.15.57:0/1721526761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 msgr2=0x5647fabd9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabd9e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80057b0 tx=0x7fcaa8005fd0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 -- 172.21.15.57:0/1721526761 shutdown_connections 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabda910 0x5647fabd27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabcf140 0x5647fabe13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/1721526761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcfa60 0x5647fabd9e30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 -- 172.21.15.57:0/1721526761 >> 172.21.15.57:0/1721526761 conn(0x5647fab442e0 msgr2=0x5647fabc9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 -- 172.21.15.57:0/1721526761 shutdown_connections 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 -- 172.21.15.57:0/1721526761 wait complete. 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.588+0000 7fcace1cdb80 1 -- start start 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 1 Processor -- start 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabdde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabcfa60 0x5647fabde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabda910 0x5647fac3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fabdebf0 con 0x5647fabda910 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fac3e620 con 0x5647fabcf140 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fac3e760 con 0x5647fabcfa60 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcace1cdb80 10 client.0 _ll_register_callbacks cb 0x7fcaa4099bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabcfa60 0x5647fabde350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabcfa60 0x5647fabde350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:35046/0 (socket says 172.21.15.57:35046) 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcab37fe700 1 -- 172.21.15.57:0/2698789100 learned_addr learned my addr 172.21.15.57:0/2698789100 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.589+0000 7fcab8d9c700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabda910 0x5647fac3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab3fff700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabdde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab37fe700 1 -- 172.21.15.57:0/2698789100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 msgr2=0x5647fabdde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab37fe700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabdde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab37fe700 1 -- 172.21.15.57:0/2698789100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabda910 msgr2=0x5647fac3e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab37fe700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabda910 0x5647fac3e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab37fe700 1 -- 172.21.15.57:0/2698789100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647fa936ce0 con 0x5647fabcfa60 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab3fff700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabdde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcab37fe700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabcfa60 0x5647fabde350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009ce0 tx=0x7fcaa80050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7955f1260f3649 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fca937fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fca937fe700 1 -- 172.21.15.57:0/2698789100 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8037070 con 0x5647fabcfa60 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaa801e040 con 0x5647fabcfa60 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.590+0000 7fca937fe700 1 -- 172.21.15.57:0/2698789100 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fcaa8031070 con 0x5647fabcfa60 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fca937fe700 1 -- 172.21.15.57:0/2698789100 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa801d050 con 0x5647fabcfa60 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 10 client.26302 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5647fa94de60 con 0x5647fabcfa60 2023-10-06T22:16:56.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fca937fe700 1 -- 172.21.15.57:0/2698789100 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa801e040 con 0x5647fabcfa60 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fca937fe700 1 -- 172.21.15.57:0/2698789100 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fcaa8004500 con 0x5647fabcfa60 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 10 client.26302 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 10 client.26302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 20 client.26302 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fac3fac0 0x5647fac41ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 4 client.26302 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5647fab1bbb0 con 0x5647fac3fac0 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.591+0000 7fcab3fff700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fac3fac0 0x5647fac41ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.592+0000 7fcab3fff700 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fac3fac0 0x5647fac41ee0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0dec825a0eefcb server_cookie=b239088c33dee078 in_seq=0 out_seq=0 2023-10-06T22:16:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.592+0000 7fca937fe700 10 client.26302 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fca937fe700 1 -- 172.21.15.57:0/2698789100 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5647fab1bbb0 con 0x5647fac3fac0 2023-10-06T22:16:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fca937fe700 10 client.26302 handle_command_reply: tid=0 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 client.26302 shutdown 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabcfa60 msgr2=0x5647fabde350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabcfa60 0x5647fabde350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009ce0 tx=0x7fcaa80050d0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 shutdown_connections 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5647fac3fac0 0x5647fac41ee0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5647fabda910 0x5647fac3e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5647fabcf140 0x5647fabdde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 --2- 172.21.15.57:0/2698789100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647fabcfa60 0x5647fabde350 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fca937fe700 0 client.26302 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.593+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 >> 172.21.15.57:0/2698789100 conn(0x5647fab442e0 msgr2=0x5647fabd27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.594+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 shutdown_connections 2023-10-06T22:16:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.594+0000 7fcace1cdb80 1 -- 172.21.15.57:0/2698789100 wait complete. 2023-10-06T22:16:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.594+0000 7fcace1cdb80 20 client.26302 trim_cache size 0 max 16384 2023-10-06T22:16:56.601 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:16:56.601 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-06T22:16:56.601 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:16:56.601 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-06T22:16:56.601 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:16:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fca9c0a15f0 msgr2=0x7fca9c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fca9c0a15f0 0x7fca9c0a3ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab4101d50 tx=0x7fcaac00b040 comp rx=0 tx=0).stop 2023-10-06T22:16:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 msgr2=0x7fcab40ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 0x7fcab40ffc40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400e9a0 tx=0x7fcaa400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 shutdown_connections 2023-10-06T22:16:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fca9c0a15f0 0x7fca9c0a3ab0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcab410b410 0x7fcab41a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcab4101ff0 0x7fcab40ffc40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 --2- 172.21.15.57:0/1524692572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcab4102910 0x7fcab4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 >> 172.21.15.57:0/1524692572 conn(0x7fcab40fb830 msgr2=0x7fcab410d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 shutdown_connections 2023-10-06T22:16:56.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.595+0000 7fcaba7ff700 1 -- 172.21.15.57:0/1524692572 wait complete. 2023-10-06T22:16:56.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 -- 172.21.15.57:0/3370230855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 msgr2=0x7ffad010d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 --2- 172.21.15.57:0/3370230855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 0x7ffad010d1c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009990 tx=0x7ffac0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 -- 172.21.15.57:0/3370230855 shutdown_connections 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 --2- 172.21.15.57:0/3370230855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffad010d700 0x7ffad010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 --2- 172.21.15.57:0/3370230855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 0x7ffad0073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 --2- 172.21.15.57:0/3370230855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 0x7ffad010d1c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 -- 172.21.15.57:0/3370230855 >> 172.21.15.57:0/3370230855 conn(0x7ffad00fc930 msgr2=0x7ffad00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 -- 172.21.15.57:0/3370230855 shutdown_connections 2023-10-06T22:16:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.836+0000 7ffad85ff700 1 -- 172.21.15.57:0/3370230855 wait complete. 2023-10-06T22:16:56.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.837+0000 7ffad85ff700 1 Processor -- start 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.837+0000 7ffad85ff700 1 -- start start 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad85ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 0x7ffad0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad85ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 0x7ffad01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad85ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffad010d700 0x7ffad019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad85ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad019e150 con 0x7ffad0073260 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad85ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad019e2c0 con 0x7ffad010d700 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad85ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad019e480 con 0x7ffad0073b80 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad639b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 0x7ffad0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad639b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 0x7ffad0199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36574/0 (socket says 172.21.15.57:36574) 2023-10-06T22:16:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad639b700 1 -- 172.21.15.57:0/1663612759 learned_addr learned my addr 172.21.15.57:0/1663612759 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffad010d700 0x7ffad019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.838+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 0x7ffad01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad6b9c700 1 -- 172.21.15.57:0/1663612759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 msgr2=0x7ffad01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 0x7ffad01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad6b9c700 1 -- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 msgr2=0x7ffad0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 0x7ffad0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad6b9c700 1 -- 172.21.15.57:0/1663612759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00095f0 con 0x7ffad010d700 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad639b700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 0x7ffad0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 0x7ffad01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffad010d700 0x7ffad019db10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffac800efc0 tx=0x7ffac800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffac77fe700 1 -- 172.21.15.57:0/1663612759 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8018070 con 0x7ffad010d700 2023-10-06T22:16:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad019e700 con 0x7ffad010d700 2023-10-06T22:16:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffac77fe700 1 -- 172.21.15.57:0/1663612759 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac800f040 con 0x7ffad010d700 2023-10-06T22:16:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffac77fe700 1 -- 172.21.15.57:0/1663612759 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800cd40 con 0x7ffad010d700 2023-10-06T22:16:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.839+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad01a5fa0 con 0x7ffad010d700 2023-10-06T22:16:56.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.843+0000 7ffac77fe700 1 -- 172.21.15.57:0/1663612759 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ffac8003a00 con 0x7ffad010d700 2023-10-06T22:16:56.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 Processor -- start 2023-10-06T22:16:56.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 -- start start 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f0c0 0x55f6d937f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d938a890 0x55f6d938cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d92e8460 con 0x55f6d937f0c0 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d92d9c20 con 0x55f6d937f9e0 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.844+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d92d7540 con 0x55f6d938a890 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad5b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f0c0 0x55f6d937f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49514/0 (socket says 172.21.15.57:49514) 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad5b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f0c0 0x55f6d937f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36584/0 (socket says 172.21.15.57:36584) 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 -- 172.21.15.57:0/4136630604 learned_addr learned my addr 172.21.15.57:0/4136630604 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d938a890 0x55f6d938cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 -- 172.21.15.57:0/4136630604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d938a890 msgr2=0x55f6d938cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d938a890 0x55f6d938cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 -- 172.21.15.57:0/4136630604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f0c0 msgr2=0x55f6d937f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f0c0 0x55f6d937f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 -- 172.21.15.57:0/4136630604 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d90fdde0 con 0x55f6d937f9e0 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d938a890 0x55f6d938cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffad639b700 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938a350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0117e0 tx=0x7ffacc011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb285f83228f6837 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffac77fe700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffabc0a15f0 0x7ffabc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.845+0000 7ffac4bfe700 1 -- 172.21.15.57:0/4136630604 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00f480 con 0x55f6d937f9e0 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffac4bfe700 1 -- 172.21.15.57:0/4136630604 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffacc00faa0 con 0x55f6d937f9e0 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffac4bfe700 1 -- 172.21.15.57:0/4136630604 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc019a00 con 0x55f6d937f9e0 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffac77fe700 1 -- 172.21.15.57:0/1663612759 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7ffac80ccc20 con 0x7ffad010d700 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffabc0a15f0 0x7ffabc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffabc0a15f0 0x7ffabc0a3ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffac00095c0 tx=0x7ffac0009510 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 -- 172.21.15.57:0/4136630604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 msgr2=0x55f6d938a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938a350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0117e0 tx=0x7ffacc011af0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 -- 172.21.15.57:0/4136630604 shutdown_connections 2023-10-06T22:16:56.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d938a890 0x55f6d938cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f0c0 0x55f6d937f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/4136630604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938a350 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 -- 172.21.15.57:0/4136630604 >> 172.21.15.57:0/4136630604 conn(0x55f6d92f4260 msgr2=0x55f6d937b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.846+0000 7ffaebefcb80 1 -- 172.21.15.57:0/4136630604 shutdown_connections 2023-10-06T22:16:56.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.847+0000 7ffaebefcb80 1 -- 172.21.15.57:0/4136630604 wait complete. 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.847+0000 7ffaebefcb80 1 -- start start 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.847+0000 7ffaebefcb80 1 Processor -- start 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d9387610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f9e0 0x55f6d9387b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d938a890 0x55f6d9388090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d92d2e30 con 0x55f6d937f9e0 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d926f710 con 0x55f6d938a890 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d926f3b0 con 0x55f6d937f0c0 2023-10-06T22:16:56.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffaebefcb80 10 client.0 _ll_register_callbacks cb 0x55f6d90cc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad5b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f9e0 0x55f6d9387b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d938a890 0x55f6d9388090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad639b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d9387610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d938a890 0x55f6d9388090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49526/0 (socket says 172.21.15.57:49526) 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 -- 172.21.15.57:0/2917017882 learned_addr learned my addr 172.21.15.57:0/2917017882 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 -- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 msgr2=0x55f6d9387610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d9387610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 -- 172.21.15.57:0/2917017882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f9e0 msgr2=0x55f6d9387b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f9e0 0x55f6d9387b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad6b9c700 1 -- 172.21.15.57:0/2917017882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc011440 con 0x55f6d938a890 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.848+0000 7ffad639b700 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d9387610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d938a890 0x55f6d9388090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffac80a1a70 tx=0x7ffac80997e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4738e02d312923a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffab9ffb700 1 -- 172.21.15.57:0/2917017882 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8004ad0 con 0x55f6d938a890 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffab9ffb700 1 -- 172.21.15.57:0/2917017882 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7ffac809a3a0 con 0x55f6d938a890 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6d9372f10 con 0x55f6d938a890 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffab9ffb700 1 -- 172.21.15.57:0/2917017882 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac8099ca0 con 0x55f6d938a890 2023-10-06T22:16:56.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffaebefcb80 10 client.26327 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f6d90e6c60 con 0x55f6d938a890 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.849+0000 7ffab9ffb700 1 -- 172.21.15.57:0/2917017882 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6d9372f10 con 0x55f6d938a890 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffab9ffb700 1 -- 172.21.15.57:0/2917017882 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ffac80a1b10 con 0x55f6d938a890 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffaebefcb80 10 client.26327 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffaebefcb80 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffaebefcb80 20 client.26327 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d939a2e0 0x55f6d939c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffaebefcb80 4 client.26327 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6d92cbe10 con 0x55f6d939a2e0 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.850+0000 7ffad639b700 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d939a2e0 0x55f6d939c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffad639b700 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d939a2e0 0x55f6d939c700 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77954c5b773ca7db server_cookie=c898cefeae1b6f51 in_seq=0 out_seq=0 2023-10-06T22:16:56.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffab9ffb700 10 client.26327 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffab9ffb700 1 -- 172.21.15.57:0/2917017882 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6d92cbe10 con 0x55f6d939a2e0 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffab9ffb700 10 client.26327 handle_command_reply: tid=0 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffaebefcb80 1 client.26327 shutdown 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d938a890 msgr2=0x55f6d9388090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d938a890 0x55f6d9388090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffac80a1a70 tx=0x7ffac80997e0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 shutdown_connections 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.851+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d939a2e0 0x55f6d939c700 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d938a890 0x55f6d9388090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffab9ffb700 0 client.26327 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d9387610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2917017882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d937f9e0 0x55f6d9387b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 >> 172.21.15.57:0/2917017882 conn(0x55f6d92f4260 msgr2=0x55f6d937b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 shutdown_connections 2023-10-06T22:16:56.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2917017882 wait complete. 2023-10-06T22:16:56.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.852+0000 7ffaebefcb80 20 client.26327 trim_cache size 0 max 16384 2023-10-06T22:16:56.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.875+0000 7ffaebefcb80 1 Processor -- start 2023-10-06T22:16:56.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.875+0000 7ffaebefcb80 1 -- start start 2023-10-06T22:16:56.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f0c0 0x55f6d93913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f9e0 0x55f6d9387ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d9384880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d9153920 con 0x55f6d938a890 2023-10-06T22:16:56.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d9377a20 con 0x55f6d937f0c0 2023-10-06T22:16:56.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d9391910 con 0x55f6d937f9e0 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffad639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f0c0 0x55f6d93913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffad5b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f9e0 0x55f6d9387ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffad639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f0c0 0x55f6d93913d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49532/0 (socket says 172.21.15.57:49532) 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffad639b700 1 -- 172.21.15.57:0/2193075154 learned_addr learned my addr 172.21.15.57:0/2193075154 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffad5b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f9e0 0x55f6d9387ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:35102/0 (socket says 172.21.15.57:35102) 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.876+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d9384880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.877+0000 7ffad639b700 1 -- 172.21.15.57:0/2193075154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f9e0 msgr2=0x55f6d9387ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.877+0000 7ffad639b700 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f9e0 0x55f6d9387ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.877+0000 7ffad639b700 1 -- 172.21.15.57:0/2193075154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 msgr2=0x55f6d9384880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.877+0000 7ffad639b700 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d9384880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.877+0000 7ffad639b700 1 -- 172.21.15.57:0/2193075154 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d9124250 con 0x55f6d937f0c0 2023-10-06T22:16:56.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.877+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f9e0 0x55f6d9387ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.877+0000 7ffad639b700 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f0c0 0x55f6d93913d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000c00 tx=0x7ffacc009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c0ec587eb5a7237 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.878+0000 7ffac53ff700 1 -- 172.21.15.57:0/2193075154 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc011e40 con 0x55f6d937f0c0 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.878+0000 7ffac53ff700 1 -- 172.21.15.57:0/2193075154 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffacc00f720 con 0x55f6d937f0c0 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.878+0000 7ffac53ff700 1 -- 172.21.15.57:0/2193075154 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc003b40 con 0x55f6d937f0c0 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2193075154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f0c0 msgr2=0x55f6d93913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f0c0 0x55f6d93913d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000c00 tx=0x7ffacc009830 comp rx=0 tx=0).stop 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2193075154 shutdown_connections 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d9384880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f0c0 0x55f6d93913d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/2193075154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f9e0 0x55f6d9387ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2193075154 >> 172.21.15.57:0/2193075154 conn(0x55f6d92f4260 msgr2=0x55f6d93828f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2193075154 shutdown_connections 2023-10-06T22:16:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 -- 172.21.15.57:0/2193075154 wait complete. 2023-10-06T22:16:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.879+0000 7ffaebefcb80 1 -- start start 2023-10-06T22:16:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 1 Processor -- start 2023-10-06T22:16:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d938eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d93f5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d938f8a0 con 0x55f6d938a890 2023-10-06T22:16:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d938fa10 con 0x55f6d937f9e0 2023-10-06T22:16:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d93f5ee0 con 0x55f6d937f0c0 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.880+0000 7ffaebefcb80 10 client.0 _ll_register_callbacks cb 0x7ffac8095fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad5b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad5b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49548/0 (socket says 172.21.15.57:49548) 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad639b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d938eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad5b9a700 1 -- 172.21.15.57:0/3774455074 learned_addr learned my addr 172.21.15.57:0/3774455074 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad6b9c700 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d93f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad639b700 1 -- 172.21.15.57:0/3774455074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 msgr2=0x55f6d938f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad639b700 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad639b700 1 -- 172.21.15.57:0/3774455074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 msgr2=0x55f6d93f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad639b700 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d93f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad639b700 1 -- 172.21.15.57:0/3774455074 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac001d070 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffad639b700 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d938eb60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc011730 tx=0x7ffacc009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1d6f913c9c2e99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.881+0000 7ffab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.882+0000 7ffab9ffb700 1 -- 172.21.15.57:0/3774455074 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0052c0 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.882+0000 7ffab9ffb700 1 -- 172.21.15.57:0/3774455074 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7ffacc005420 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.882+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffacc00faa0 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.882+0000 7ffab9ffb700 1 -- 172.21.15.57:0/3774455074 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffacc0055e0 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.882+0000 7ffaebefcb80 10 client.26320 fetch_fsmap learned FSMap version 33 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.882+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f6d90fdde0 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.882+0000 7ffab9ffb700 1 -- 172.21.15.57:0/3774455074 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffacc00faa0 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffab9ffb700 1 -- 172.21.15.57:0/3774455074 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ffacc02e860 con 0x55f6d937f0c0 2023-10-06T22:16:56.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffaebefcb80 10 client.26320 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:16:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffaebefcb80 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:16:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffaebefcb80 20 client.26320 populate_metadata read hostname 'smithi057' 2023-10-06T22:16:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d93f7340 0x55f6d93f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:16:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffaebefcb80 4 client.26320 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:16:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f6d92cbb30 con 0x55f6d93f7340 2023-10-06T22:16:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d93f7340 0x55f6d93f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:16:56.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffad5b9a700 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d93f7340 0x55f6d93f9760 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffe42015b463b67 server_cookie=57b0c891cfbe10c0 in_seq=0 out_seq=0 2023-10-06T22:16:56.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.883+0000 7ffab9ffb700 10 client.26320 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffab9ffb700 1 -- 172.21.15.57:0/3774455074 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f6d92cbb30 con 0x55f6d93f7340 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffab9ffb700 10 client.26320 handle_command_reply: tid=0 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 client.26320 shutdown 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 msgr2=0x55f6d938eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d938eb60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc011730 tx=0x7ffacc009fd0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 shutdown_connections 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f6d93f7340 0x55f6d93f9760 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f6d938a890 0x55f6d93f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffab9ffb700 0 client.26320 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f6d937f0c0 0x55f6d938eb60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 --2- 172.21.15.57:0/3774455074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6d937f9e0 0x55f6d938f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 >> 172.21.15.57:0/3774455074 conn(0x55f6d92f4260 msgr2=0x55f6d9380f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 shutdown_connections 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 1 -- 172.21.15.57:0/3774455074 wait complete. 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.884+0000 7ffaebefcb80 20 client.26320 trim_cache size 0 max 16384 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:16:56.891 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:16:56.892 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:16:56.892 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:16:56.892 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:16:56.892 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:16:56.892 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:16:56.892 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffabc0a15f0 msgr2=0x7ffabc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffabc0a15f0 0x7ffabc0a3ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffac00095c0 tx=0x7ffac0009510 comp rx=0 tx=0).stop 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffad010d700 msgr2=0x7ffad019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffad010d700 0x7ffad019db10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffac800efc0 tx=0x7ffac800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 shutdown_connections 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffabc0a15f0 0x7ffabc0a3ab0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffad010d700 0x7ffad019db10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffad0073260 0x7ffad0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 --2- 172.21.15.57:0/1663612759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0073b80 0x7ffad01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:16:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 >> 172.21.15.57:0/1663612759 conn(0x7ffad00fc930 msgr2=0x7ffad010f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:16:56.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 shutdown_connections 2023-10-06T22:16:56.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:16:56.885+0000 7ffad85ff700 1 -- 172.21.15.57:0/1663612759 wait complete. 2023-10-06T22:16:56.906 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:16:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: pgmap v69: 97 pgs: 97 active+clean; 171 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.2 MiB/s wr, 666 op/s 2023-10-06T22:16:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: scrub queued for path: / 2023-10-06T22:16:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:16:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:16:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:16:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:56 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:57.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: pgmap v69: 97 pgs: 97 active+clean; 171 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.2 MiB/s wr, 666 op/s 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: scrub queued for path: / 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:56 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:57.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: pgmap v69: 97 pgs: 97 active+clean; 171 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.2 MiB/s wr, 666 op/s 2023-10-06T22:16:57.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: scrub queued for path: / 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:16:57.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:56 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:58.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:57 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:16:58.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:57 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:16:58.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:57 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:16:58.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:57 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:16:58.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:57 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:16:58.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:57 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:16:59.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:58 smithi156 ceph-mon[134358]: pgmap v70: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 768 op/s 2023-10-06T22:16:59.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:58 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:16:59.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:58 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:59.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:58 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:59.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:58 smithi190 ceph-mon[134891]: pgmap v70: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 768 op/s 2023-10-06T22:16:59.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:58 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:16:59.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:58 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:59.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:58 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:59.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:58 smithi057 ceph-mon[98492]: pgmap v70: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 768 op/s 2023-10-06T22:16:59.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:58 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:16:59.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:58 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:16:59.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:58 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:17:00.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:16:59 smithi156 ceph-mon[134358]: pgmap v71: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 597 op/s 2023-10-06T22:17:00.322 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:16:59 smithi190 ceph-mon[134891]: pgmap v71: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 597 op/s 2023-10-06T22:17:00.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:16:59 smithi057 ceph-mon[98492]: pgmap v71: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 597 op/s 2023-10-06T22:17:02.033 INFO:tasks.workunit.client.0.smithi057.stdout: 45 160036 467 119587 336 78447 961 2023-10-06T22:17:03.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:02 smithi156 ceph-mon[134358]: pgmap v72: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 597 op/s 2023-10-06T22:17:03.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:02 smithi190 ceph-mon[134891]: pgmap v72: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 597 op/s 2023-10-06T22:17:03.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:02 smithi057 ceph-mon[98492]: pgmap v72: 97 pgs: 97 active+clean; 178 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 597 op/s 2023-10-06T22:17:05.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:04 smithi156 ceph-mon[134358]: pgmap v73: 97 pgs: 97 active+clean; 184 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 652 op/s 2023-10-06T22:17:05.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:04 smithi190 ceph-mon[134891]: pgmap v73: 97 pgs: 97 active+clean; 184 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 652 op/s 2023-10-06T22:17:05.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:04 smithi057 ceph-mon[98492]: pgmap v73: 97 pgs: 97 active+clean; 184 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 652 op/s 2023-10-06T22:17:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.340+0000 7f26703ff700 1 -- 172.21.15.57:0/317741963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f266811a690 msgr2=0x7f266811ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.340+0000 7f26703ff700 1 --2- 172.21.15.57:0/317741963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f266811a690 0x7f266811ca80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f266000b0a0 tx=0x7f266000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.394+0000 7f26703ff700 1 -- 172.21.15.57:0/317741963 shutdown_connections 2023-10-06T22:17:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.394+0000 7f26703ff700 1 --2- 172.21.15.57:0/317741963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f266811a690 0x7f266811ca80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.394+0000 7f26703ff700 1 --2- 172.21.15.57:0/317741963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266810f620 0x7f2668107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.394+0000 7f26703ff700 1 --2- 172.21.15.57:0/317741963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 0x7f26681083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.394+0000 7f26703ff700 1 -- 172.21.15.57:0/317741963 >> 172.21.15.57:0/317741963 conn(0x7f266806cfa0 msgr2=0x7f266806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.417+0000 7f26703ff700 1 -- 172.21.15.57:0/317741963 shutdown_connections 2023-10-06T22:17:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.417+0000 7f26703ff700 1 -- 172.21.15.57:0/317741963 wait complete. 2023-10-06T22:17:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.420+0000 7f26703ff700 1 Processor -- start 2023-10-06T22:17:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.420+0000 7f26703ff700 1 -- start start 2023-10-06T22:17:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.421+0000 7f26703ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266810f620 0x7f2668118b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.421+0000 7f26703ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 0x7f2668119080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.421+0000 7f26703ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f266811a690 0x7f2668111a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.421+0000 7f26703ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2668112240 con 0x7f266811a690 2023-10-06T22:17:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.421+0000 7f26703ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26681123b0 con 0x7f266810fa90 2023-10-06T22:17:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.421+0000 7f26703ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2668112570 con 0x7f266810f620 2023-10-06T22:17:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.421+0000 7f266d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 0x7f2668119080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.422+0000 7f266d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 0x7f2668119080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:51204/0 (socket says 172.21.15.57:51204) 2023-10-06T22:17:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.422+0000 7f266d99a700 1 -- 172.21.15.57:0/3327463034 learned_addr learned my addr 172.21.15.57:0/3327463034 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.422+0000 7f266e19b700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266810f620 0x7f2668118b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.426+0000 7f266d99a700 1 -- 172.21.15.57:0/3327463034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266810f620 msgr2=0x7f2668118b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.426+0000 7f266d99a700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266810f620 0x7f2668118b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.426+0000 7f266d99a700 1 -- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f266811a690 msgr2=0x7f2668111a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.426+0000 7f266d99a700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f266811a690 0x7f2668111a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.426+0000 7f266d99a700 1 -- 172.21.15.57:0/3327463034 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660009d00 con 0x7f266810fa90 2023-10-06T22:17:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.429+0000 7f266d99a700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 0x7f2668119080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f265800b550 tx=0x7f265800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.512+0000 7f265f7fe700 1 -- 172.21.15.57:0/3327463034 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658011830 con 0x7f266810fa90 2023-10-06T22:17:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.512+0000 7f265f7fe700 1 -- 172.21.15.57:0/3327463034 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2658011e50 con 0x7f266810fa90 2023-10-06T22:17:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.512+0000 7f265f7fe700 1 -- 172.21.15.57:0/3327463034 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658010e70 con 0x7f266810fa90 2023-10-06T22:17:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.518+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2668116690 con 0x7f266810fa90 2023-10-06T22:17:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.518+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2668116b20 con 0x7f266810fa90 2023-10-06T22:17:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.527+0000 7f2683e0fb80 1 Processor -- start 2023-10-06T22:17:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.530+0000 7f265f7fe700 1 -- 172.21.15.57:0/3327463034 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2658010510 con 0x7f266810fa90 2023-10-06T22:17:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.533+0000 7f265f7fe700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f26540a1800 0x7f26540a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.533+0000 7f265f7fe700 1 -- 172.21.15.57:0/3327463034 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f26580cca40 con 0x7f266810fa90 2023-10-06T22:17:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.537+0000 7f266e99c700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f26540a1800 0x7f26540a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.537+0000 7f2683e0fb80 1 -- start start 2023-10-06T22:17:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.538+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 0x564ddd35afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.538+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd365e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.538+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.538+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd2c4400 con 0x564ddd35ac00 2023-10-06T22:17:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.538+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd2b5bc0 con 0x564ddd35b520 2023-10-06T22:17:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.538+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd2b34e0 con 0x564ddd3663d0 2023-10-06T22:17:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.539+0000 7f266e99c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.539+0000 7f266e99c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3687c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60756/0 (socket says 172.21.15.57:60756) 2023-10-06T22:17:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.539+0000 7f266e99c700 1 -- 172.21.15.57:0/3832286910 learned_addr learned my addr 172.21.15.57:0/3832286910 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.539+0000 7f266d99a700 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd365e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.539+0000 7f266e99c700 1 -- 172.21.15.57:0/3832286910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 msgr2=0x564ddd365e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.539+0000 7f266e99c700 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd365e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.540+0000 7f266e99c700 1 -- 172.21.15.57:0/3832286910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 msgr2=0x564ddd35afe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:17:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.540+0000 7f266e99c700 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 0x564ddd35afe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.540+0000 7f266e99c700 1 -- 172.21.15.57:0/3832286910 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ddd0d9d80 con 0x564ddd3663d0 2023-10-06T22:17:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.540+0000 7f266e99c700 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3687c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f266000b0a0 tx=0x7f2660007620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a00fe53f66ea3295 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.548+0000 7f266e99c700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f26540a1800 0x7f26540a3cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2660027000 tx=0x7f2660020c60 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.604+0000 7f265cbfe700 1 -- 172.21.15.57:0/3832286910 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000e070 con 0x564ddd3663d0 2023-10-06T22:17:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.604+0000 7f265cbfe700 1 -- 172.21.15.57:0/3832286910 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2660024070 con 0x564ddd3663d0 2023-10-06T22:17:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.604+0000 7f265cbfe700 1 -- 172.21.15.57:0/3832286910 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660020870 con 0x564ddd3663d0 2023-10-06T22:17:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.606+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3832286910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 msgr2=0x564ddd3687c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.606+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3687c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f266000b0a0 tx=0x7f2660007620 comp rx=0 tx=0).stop 2023-10-06T22:17:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.613+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3832286910 shutdown_connections 2023-10-06T22:17:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.613+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3687c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.613+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 0x564ddd35afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.613+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3832286910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd365e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.613+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3832286910 >> 172.21.15.57:0/3832286910 conn(0x564ddd2d0200 msgr2=0x564ddd357080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.620+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3832286910 shutdown_connections 2023-10-06T22:17:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.620+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3832286910 wait complete. 2023-10-06T22:17:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.621+0000 7f2683e0fb80 1 -- start start 2023-10-06T22:17:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.622+0000 7f2683e0fb80 1 Processor -- start 2023-10-06T22:17:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.623+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd3631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.624+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd363720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.624+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd363c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.624+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd2aedd0 con 0x564ddd3663d0 2023-10-06T22:17:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.624+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd24b6b0 con 0x564ddd35b520 2023-10-06T22:17:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.624+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd24b350 con 0x564ddd35ac00 2023-10-06T22:17:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.624+0000 7f2683e0fb80 10 client.0 _ll_register_callbacks cb 0x564ddd0a87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.625+0000 7f266e19b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd3631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.625+0000 7f266d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd363720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.625+0000 7f266e19b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd3631e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60758/0 (socket says 172.21.15.57:60758) 2023-10-06T22:17:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.625+0000 7f266e19b700 1 -- 172.21.15.57:0/3904780439 learned_addr learned my addr 172.21.15.57:0/3904780439 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.627+0000 7f266e19b700 1 -- 172.21.15.57:0/3904780439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 msgr2=0x564ddd363720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.627+0000 7f266e19b700 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd363720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.627+0000 7f266e19b700 1 -- 172.21.15.57:0/3904780439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 msgr2=0x564ddd363c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.627+0000 7f266e19b700 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd363c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.627+0000 7f266e19b700 1 -- 172.21.15.57:0/3904780439 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266811d3b0 con 0x564ddd35ac00 2023-10-06T22:17:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.630+0000 7f266e19b700 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd3631e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f266400c180 tx=0x7f266400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9bc0223858f989 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.637+0000 7f2651ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:17:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.637+0000 7f2651ffb700 1 -- 172.21.15.57:0/3904780439 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640113f0 con 0x564ddd35ac00 2023-10-06T22:17:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.637+0000 7f2651ffb700 1 -- 172.21.15.57:0/3904780439 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2664011550 con 0x564ddd35ac00 2023-10-06T22:17:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.639+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ddd34ed50 con 0x564ddd35ac00 2023-10-06T22:17:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.640+0000 7f2651ffb700 1 -- 172.21.15.57:0/3904780439 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2664011710 con 0x564ddd35ac00 2023-10-06T22:17:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.640+0000 7f2683e0fb80 10 client.26326 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.641+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564ddd0c2c00 con 0x564ddd35ac00 2023-10-06T22:17:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.641+0000 7f2651ffb700 1 -- 172.21.15.57:0/3904780439 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ddd34ed50 con 0x564ddd35ac00 2023-10-06T22:17:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f2651ffb700 1 -- 172.21.15.57:0/3904780439 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2664011870 con 0x564ddd35ac00 2023-10-06T22:17:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f2683e0fb80 10 client.26326 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f2683e0fb80 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f2683e0fb80 20 client.26326 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd375ff0 0x564ddd378410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f2683e0fb80 4 client.26326 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:17:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ddd2a7db0 con 0x564ddd375ff0 2023-10-06T22:17:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.643+0000 7f266d99a700 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd375ff0 0x564ddd378410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.647+0000 7f266d99a700 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd375ff0 0x564ddd378410 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdfa2a2dede88b5b server_cookie=7fabe108dbd872ce in_seq=0 out_seq=0 2023-10-06T22:17:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.647+0000 7f2651ffb700 10 client.26326 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.648+0000 7f2651ffb700 1 -- 172.21.15.57:0/3904780439 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564ddd2a7db0 con 0x564ddd375ff0 2023-10-06T22:17:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.648+0000 7f2651ffb700 10 client.26326 handle_command_reply: tid=0 2023-10-06T22:17:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.648+0000 7f2683e0fb80 1 client.26326 shutdown 2023-10-06T22:17:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 msgr2=0x564ddd3631e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd3631e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f266400c180 tx=0x7f266400c490 comp rx=0 tx=0).stop 2023-10-06T22:17:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 shutdown_connections 2023-10-06T22:17:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd375ff0 0x564ddd378410 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd363c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd3631e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3904780439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd363720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.649+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 >> 172.21.15.57:0/3904780439 conn(0x564ddd2d0200 msgr2=0x564ddd356c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.650+0000 7f2651ffb700 0 client.26326 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.676+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 shutdown_connections 2023-10-06T22:17:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.676+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3904780439 wait complete. 2023-10-06T22:17:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.677+0000 7f2683e0fb80 20 client.26326 trim_cache size 0 max 16384 2023-10-06T22:17:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.717+0000 7f2683e0fb80 1 Processor -- start 2023-10-06T22:17:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.717+0000 7f2683e0fb80 1 -- start start 2023-10-06T22:17:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd36cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd3639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd356b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd12f8c0 con 0x564ddd3663d0 2023-10-06T22:17:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd3736a0 con 0x564ddd35b520 2023-10-06T22:17:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd36d460 con 0x564ddd35ac00 2023-10-06T22:17:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f266d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd3639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.718+0000 7f266d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd3639a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:51238/0 (socket says 172.21.15.57:51238) 2023-10-06T22:17:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.719+0000 7f266d99a700 1 -- 172.21.15.57:0/3006047067 learned_addr learned my addr 172.21.15.57:0/3006047067 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.719+0000 7f266e19b700 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd36cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.720+0000 7f266e99c700 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd356b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.720+0000 7f266e99c700 1 -- 172.21.15.57:0/3006047067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 msgr2=0x564ddd36cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.720+0000 7f266e99c700 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd36cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.720+0000 7f266e99c700 1 -- 172.21.15.57:0/3006047067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 msgr2=0x564ddd3639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.720+0000 7f266e99c700 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd3639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.720+0000 7f266e99c700 1 -- 172.21.15.57:0/3006047067 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ddd1001f0 con 0x564ddd3663d0 2023-10-06T22:17:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.721+0000 7f266e99c700 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd356b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f266000b0a0 tx=0x7f266002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26cea6baf747ff7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.722+0000 7f265d3ff700 1 -- 172.21.15.57:0/3006047067 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000e030 con 0x564ddd3663d0 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.722+0000 7f265d3ff700 1 -- 172.21.15.57:0/3006047067 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f266000f040 con 0x564ddd3663d0 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.722+0000 7f265d3ff700 1 -- 172.21.15.57:0/3006047067 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600206b0 con 0x564ddd3663d0 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.723+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3006047067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 msgr2=0x564ddd356b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.723+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd356b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f266000b0a0 tx=0x7f266002d000 comp rx=0 tx=0).stop 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.724+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3006047067 shutdown_connections 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.724+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd3663d0 0x564ddd356b50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.724+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd35ac00 0x564ddd36cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.724+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/3006047067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd35b520 0x564ddd3639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.724+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3006047067 >> 172.21.15.57:0/3006047067 conn(0x564ddd2d0200 msgr2=0x564ddd371670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.730+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3006047067 shutdown_connections 2023-10-06T22:17:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.731+0000 7f2683e0fb80 1 -- 172.21.15.57:0/3006047067 wait complete. 2023-10-06T22:17:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.732+0000 7f2683e0fb80 1 -- start start 2023-10-06T22:17:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.732+0000 7f2683e0fb80 1 Processor -- start 2023-10-06T22:17:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.733+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 0x564ddd3bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.733+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.733+0000 7f2683e0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd3c2e70 0x564ddd3c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.733+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd3c3830 con 0x564ddd35ac00 2023-10-06T22:17:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.733+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd3c39a0 con 0x564ddd3c2e70 2023-10-06T22:17:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.733+0000 7f2683e0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ddd3c3b60 con 0x564ddd3663d0 2023-10-06T22:17:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.733+0000 7f2683e0fb80 10 client.0 _ll_register_callbacks cb 0x7f266400a830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.736+0000 7f266e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd3c2e70 0x564ddd3c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.737+0000 7f266d99a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.737+0000 7f266d99a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3be4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60770/0 (socket says 172.21.15.57:60770) 2023-10-06T22:17:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.737+0000 7f266d99a700 1 -- 172.21.15.57:0/2801891464 learned_addr learned my addr 172.21.15.57:0/2801891464 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.741+0000 7f266e19b700 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 0x564ddd3bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.743+0000 7f266d99a700 1 -- 172.21.15.57:0/2801891464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd3c2e70 msgr2=0x564ddd3c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.743+0000 7f266d99a700 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd3c2e70 0x564ddd3c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.743+0000 7f266d99a700 1 -- 172.21.15.57:0/2801891464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 msgr2=0x564ddd3bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.743+0000 7f266d99a700 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 0x564ddd3bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.743+0000 7f266d99a700 1 -- 172.21.15.57:0/2801891464 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ddd0d9d80 con 0x564ddd3663d0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.743+0000 7f266d99a700 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3be4e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2658099840 tx=0x7f26580b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40bb3c7f8616045a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.744+0000 7f2651ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.744+0000 7f2651ffb700 1 -- 172.21.15.57:0/2801891464 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265809dc90 con 0x564ddd3663d0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.744+0000 7f2651ffb700 1 -- 172.21.15.57:0/2801891464 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f26580a1ed0 con 0x564ddd3663d0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.744+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2660024070 con 0x564ddd3663d0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.744+0000 7f2651ffb700 1 -- 172.21.15.57:0/2801891464 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2658011990 con 0x564ddd3663d0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.744+0000 7f2683e0fb80 10 client.26329 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.744+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564ddd0d8250 con 0x564ddd3663d0 2023-10-06T22:17:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2651ffb700 1 -- 172.21.15.57:0/2801891464 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2660024070 con 0x564ddd3663d0 2023-10-06T22:17:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2651ffb700 1 -- 172.21.15.57:0/2801891464 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2658099030 con 0x564ddd3663d0 2023-10-06T22:17:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2683e0fb80 10 client.26329 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2683e0fb80 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2683e0fb80 20 client.26329 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd3cb370 0x564ddd3cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2683e0fb80 4 client.26329 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:17:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ddd2a7ad0 con 0x564ddd3cb370 2023-10-06T22:17:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.745+0000 7f266e19b700 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd3cb370 0x564ddd3cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.746+0000 7f266e19b700 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd3cb370 0x564ddd3cd790 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffa8f27b5b2b245 server_cookie=39619abd8867367 in_seq=0 out_seq=0 2023-10-06T22:17:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.746+0000 7f2651ffb700 10 client.26329 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2651ffb700 1 -- 172.21.15.57:0/2801891464 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ddd2a7ad0 con 0x564ddd3cb370 2023-10-06T22:17:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2651ffb700 10 client.26329 handle_command_reply: tid=0 2023-10-06T22:17:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 client.26329 shutdown 2023-10-06T22:17:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 msgr2=0x564ddd3be4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3be4e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2658099840 tx=0x7f26580b9040 comp rx=0 tx=0).stop 2023-10-06T22:17:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 shutdown_connections 2023-10-06T22:17:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564ddd3cb370 0x564ddd3cd790 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ddd3c2e70 0x564ddd3c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564ddd35ac00 0x564ddd3bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 --2- 172.21.15.57:0/2801891464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ddd3663d0 0x564ddd3be4e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2651ffb700 0 client.26329 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.927+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 >> 172.21.15.57:0/2801891464 conn(0x564ddd2d0200 msgr2=0x564ddd3639a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 shutdown_connections 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f2683e0fb80 1 -- 172.21.15.57:0/2801891464 wait complete. 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f2683e0fb80 20 client.26329 trim_cache size 0 max 16384 2023-10-06T22:17:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:17:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:17:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:17:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:17:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f26540a1800 msgr2=0x7f26540a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f26540a1800 0x7f26540a3cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2660027000 tx=0x7f2660020c60 comp rx=0 tx=0).stop 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 msgr2=0x7f2668119080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 0x7f2668119080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f265800b550 tx=0x7f265800b910 comp rx=0 tx=0).stop 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 shutdown_connections 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f26540a1800 0x7f26540a3cc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f266811a690 0x7f2668111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266810f620 0x7f2668118b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 --2- 172.21.15.57:0/3327463034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f266810fa90 0x7f2668119080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.928+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 >> 172.21.15.57:0/3327463034 conn(0x7f266806cfa0 msgr2=0x7f266806f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.929+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 shutdown_connections 2023-10-06T22:17:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:06.929+0000 7f26703ff700 1 -- 172.21.15.57:0/3327463034 wait complete. 2023-10-06T22:17:07.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:06 smithi190 ceph-mon[134891]: pgmap v74: 97 pgs: 97 active+clean; 184 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2023-10-06T22:17:07.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:06 smithi057 ceph-mon[98492]: pgmap v74: 97 pgs: 97 active+clean; 184 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2023-10-06T22:17:07.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:06 smithi156 ceph-mon[134358]: pgmap v74: 97 pgs: 97 active+clean; 184 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2023-10-06T22:17:09.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:08 smithi190 ceph-mon[134891]: pgmap v75: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2023-10-06T22:17:09.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:08 smithi057 ceph-mon[98492]: pgmap v75: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2023-10-06T22:17:09.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:08 smithi156 ceph-mon[134358]: pgmap v75: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2023-10-06T22:17:11.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:10 smithi190 ceph-mon[134891]: pgmap v76: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-10-06T22:17:11.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:10 smithi057 ceph-mon[98492]: pgmap v76: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-10-06T22:17:11.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:10 smithi156 ceph-mon[134358]: pgmap v76: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-10-06T22:17:12.033 INFO:tasks.workunit.client.0.smithi057.stdout: 66 107748 965 81512 1211 43021 28 2023-10-06T22:17:13.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:12 smithi190 ceph-mon[134891]: pgmap v77: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-10-06T22:17:13.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:12 smithi057 ceph-mon[98492]: pgmap v77: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-10-06T22:17:13.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:12 smithi156 ceph-mon[134358]: pgmap v77: 97 pgs: 97 active+clean; 191 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-10-06T22:17:15.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:14 smithi190 ceph-mon[134891]: pgmap v78: 97 pgs: 97 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2023-10-06T22:17:15.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:14 smithi057 ceph-mon[98492]: pgmap v78: 97 pgs: 97 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2023-10-06T22:17:15.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:14 smithi156 ceph-mon[134358]: pgmap v78: 97 pgs: 97 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2023-10-06T22:17:17.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:16 smithi190 ceph-mon[134891]: pgmap v79: 97 pgs: 97 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-10-06T22:17:17.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:16 smithi057 ceph-mon[98492]: pgmap v79: 97 pgs: 97 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-10-06T22:17:17.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:16 smithi156 ceph-mon[134358]: pgmap v79: 97 pgs: 97 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-10-06T22:17:19.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:18 smithi190 ceph-mon[134891]: pgmap v80: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 598 op/s 2023-10-06T22:17:19.176 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:18 smithi057 ceph-mon[98492]: pgmap v80: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 598 op/s 2023-10-06T22:17:19.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:18 smithi156 ceph-mon[134358]: pgmap v80: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 598 op/s 2023-10-06T22:17:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.417+0000 7f6aacdff700 1 -- 172.21.15.57:0/2806815395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6aa81061e0 msgr2=0x7f6aa810dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.417+0000 7f6aacdff700 1 --2- 172.21.15.57:0/2806815395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6aa81061e0 0x7f6aa810dd60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000b0a0 tx=0x7f6aa000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.418+0000 7f6aacdff700 1 -- 172.21.15.57:0/2806815395 shutdown_connections 2023-10-06T22:17:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.418+0000 7f6aacdff700 1 --2- 172.21.15.57:0/2806815395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6aa81061e0 0x7f6aa810dd60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.418+0000 7f6aacdff700 1 --2- 172.21.15.57:0/2806815395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa81010c0 0x7f6aa81014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.418+0000 7f6aacdff700 1 --2- 172.21.15.57:0/2806815395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa8101a70 0x7f6aa8105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.418+0000 7f6aacdff700 1 -- 172.21.15.57:0/2806815395 >> 172.21.15.57:0/2806815395 conn(0x7f6aa80fc940 msgr2=0x7f6aa80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.418+0000 7f6aacdff700 1 -- 172.21.15.57:0/2806815395 shutdown_connections 2023-10-06T22:17:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.418+0000 7f6aacdff700 1 -- 172.21.15.57:0/2806815395 wait complete. 2023-10-06T22:17:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.420+0000 7f6aacdff700 1 Processor -- start 2023-10-06T22:17:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.427+0000 7f6aacdff700 1 -- start start 2023-10-06T22:17:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.427+0000 7f6aacdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6aa81010c0 0x7f6aa819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.427+0000 7f6aacdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa81061e0 0x7f6aa819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.427+0000 7f6aacdff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa810a9e0 0x7f6aa810ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.427+0000 7f6aacdff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa810b570 con 0x7f6aa810a9e0 2023-10-06T22:17:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.428+0000 7f6aacdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa810b6b0 con 0x7f6aa81061e0 2023-10-06T22:17:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.428+0000 7f6aacdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa810b870 con 0x7f6aa81010c0 2023-10-06T22:17:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.428+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa810a9e0 0x7f6aa810ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.428+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa810a9e0 0x7f6aa810ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34154/0 (socket says 172.21.15.57:34154) 2023-10-06T22:17:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.428+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/3419704835 learned_addr learned my addr 172.21.15.57:0/3419704835 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.429+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/3419704835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6aa81010c0 msgr2=0x7f6aa819e020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:17:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.429+0000 7f6aa6d9d700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6aa81010c0 0x7f6aa819e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.429+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/3419704835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa81061e0 msgr2=0x7f6aa819e560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:17:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.429+0000 7f6aa6d9d700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa81061e0 0x7f6aa819e560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.429+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/3419704835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0009d00 con 0x7f6aa810a9e0 2023-10-06T22:17:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.429+0000 7f6aa6d9d700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa810a9e0 0x7f6aa810ae60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00062a0 tx=0x7f6aa000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.432+0000 7f6a977fe700 1 -- 172.21.15.57:0/3419704835 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000e040 con 0x7f6aa810a9e0 2023-10-06T22:17:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.433+0000 7f6a977fe700 1 -- 172.21.15.57:0/3419704835 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa0003e10 con 0x7f6aa810a9e0 2023-10-06T22:17:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.433+0000 7f6a977fe700 1 -- 172.21.15.57:0/3419704835 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0005280 con 0x7f6aa810a9e0 2023-10-06T22:17:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.442+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa8199c00 con 0x7f6aa810a9e0 2023-10-06T22:17:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.442+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa819a030 con 0x7f6aa810a9e0 2023-10-06T22:17:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.454+0000 7f6a977fe700 1 -- 172.21.15.57:0/3419704835 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6aa00054a0 con 0x7f6aa810a9e0 2023-10-06T22:17:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.456+0000 7f6a977fe700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6a900a1780 0x7f6a900a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.467+0000 7f6ac077ab80 1 Processor -- start 2023-10-06T22:17:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.467+0000 7f6a977fe700 1 -- 172.21.15.57:0/3419704835 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f6aa00cde20 con 0x7f6aa810a9e0 2023-10-06T22:17:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.467+0000 7f6ac077ab80 1 -- start start 2023-10-06T22:17:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.468+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 0x5622950bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.468+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.468+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.468+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295026400 con 0x5622950bd520 2023-10-06T22:17:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.468+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295017bc0 con 0x5622950c83d0 2023-10-06T22:17:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.468+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622950154e0 con 0x5622950bcc00 2023-10-06T22:17:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.470+0000 7f6aa659c700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6a900a1780 0x7f6a900a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.470+0000 7f6aa5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.470+0000 7f6aa5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34162/0 (socket says 172.21.15.57:34162) 2023-10-06T22:17:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.470+0000 7f6aa5d9b700 1 -- 172.21.15.57:0/2737697594 learned_addr learned my addr 172.21.15.57:0/2737697594 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.471+0000 7f6aa6d9d700 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.472+0000 7f6aa5d9b700 1 -- 172.21.15.57:0/2737697594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 msgr2=0x5622950bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.472+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 0x5622950bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.472+0000 7f6aa5d9b700 1 -- 172.21.15.57:0/2737697594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 msgr2=0x5622950ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.472+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.472+0000 7f6aa5d9b700 1 -- 172.21.15.57:0/2737697594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562294e3bd80 con 0x5622950bd520 2023-10-06T22:17:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.473+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c7e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ba30 tx=0x7f6a9c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319e9e3082cd41b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.476+0000 7f6a94ff9700 1 -- 172.21.15.57:0/2737697594 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c004a30 con 0x5622950bd520 2023-10-06T22:17:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.476+0000 7f6a94ff9700 1 -- 172.21.15.57:0/2737697594 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9c00dc90 con 0x5622950bd520 2023-10-06T22:17:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.476+0000 7f6a94ff9700 1 -- 172.21.15.57:0/2737697594 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c005440 con 0x5622950bd520 2023-10-06T22:17:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.478+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2737697594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 msgr2=0x5622950c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.478+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c7e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ba30 tx=0x7f6a9c00bd40 comp rx=0 tx=0).stop 2023-10-06T22:17:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6aa659c700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6a900a1780 0x7f6a900a3c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009b00 tx=0x7f6a98009ec0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2737697594 shutdown_connections 2023-10-06T22:17:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 0x5622950bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2737697594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c7e90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2737697594 >> 172.21.15.57:0/2737697594 conn(0x562295032200 msgr2=0x5622950b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2737697594 shutdown_connections 2023-10-06T22:17:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.480+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2737697594 wait complete. 2023-10-06T22:17:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.481+0000 7f6ac077ab80 1 -- start start 2023-10-06T22:17:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.481+0000 7f6ac077ab80 1 Processor -- start 2023-10-06T22:17:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.482+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 0x5622950c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.482+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.482+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.482+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562295010dd0 con 0x5622950bd520 2023-10-06T22:17:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.482+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562294fad6b0 con 0x5622950c83d0 2023-10-06T22:17:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.482+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562294fad350 con 0x5622950bcc00 2023-10-06T22:17:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.483+0000 7f6ac077ab80 10 client.0 _ll_register_callbacks cb 0x562294e0a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.489+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.489+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:58444/0 (socket says 172.21.15.57:58444) 2023-10-06T22:17:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.489+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/1067840780 learned_addr learned my addr 172.21.15.57:0/1067840780 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.489+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.490+0000 7f6aa659c700 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 0x5622950c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.490+0000 7f6aa5d9b700 1 -- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 msgr2=0x5622950c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.490+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 0x5622950c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.490+0000 7f6aa5d9b700 1 -- 172.21.15.57:0/1067840780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 msgr2=0x5622950c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.490+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.490+0000 7f6aa5d9b700 1 -- 172.21.15.57:0/1067840780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c00b660 con 0x5622950bd520 2023-10-06T22:17:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.491+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c56e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c00d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a81881b523a972 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.491+0000 7f6a8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:17:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.493+0000 7f6a8dffb700 1 -- 172.21.15.57:0/1067840780 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c01f070 con 0x5622950bd520 2023-10-06T22:17:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.493+0000 7f6a8dffb700 1 -- 172.21.15.57:0/1067840780 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f6a9c0050c0 con 0x5622950bd520 2023-10-06T22:17:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.494+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622950b0d50 con 0x5622950bd520 2023-10-06T22:17:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.495+0000 7f6a8dffb700 1 -- 172.21.15.57:0/1067840780 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a9c00db00 con 0x5622950bd520 2023-10-06T22:17:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.495+0000 7f6ac077ab80 10 client.16341 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.495+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562294e24c00 con 0x5622950bd520 2023-10-06T22:17:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.496+0000 7f6a8dffb700 1 -- 172.21.15.57:0/1067840780 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622950b0d50 con 0x5622950bd520 2023-10-06T22:17:20.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.496+0000 7f6a8dffb700 1 -- 172.21.15.57:0/1067840780 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6a9c02e470 con 0x5622950bd520 2023-10-06T22:17:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.497+0000 7f6ac077ab80 10 client.16341 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.497+0000 7f6ac077ab80 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.497+0000 7f6ac077ab80 20 client.16341 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.498+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5622950d7ea0 0x5622950da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.498+0000 7f6ac077ab80 4 client.16341 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:17:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.498+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562295009db0 con 0x5622950d7ea0 2023-10-06T22:17:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.498+0000 7f6aa659c700 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5622950d7ea0 0x5622950da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.499+0000 7f6aa659c700 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5622950d7ea0 0x5622950da2c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a826dc26813723 server_cookie=40250b7aff968af3 in_seq=0 out_seq=0 2023-10-06T22:17:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.499+0000 7f6a8dffb700 10 client.16341 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.502+0000 7f6a8dffb700 1 -- 172.21.15.57:0/1067840780 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562295009db0 con 0x5622950d7ea0 2023-10-06T22:17:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.502+0000 7f6a8dffb700 10 client.16341 handle_command_reply: tid=0 2023-10-06T22:17:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.502+0000 7f6ac077ab80 1 client.16341 shutdown 2023-10-06T22:17:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.503+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 msgr2=0x5622950c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.503+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c56e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c000c00 tx=0x7f6a9c00d7d0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.504+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 shutdown_connections 2023-10-06T22:17:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.504+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5622950d7ea0 0x5622950da2c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.504+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.504+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bcc00 0x5622950c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.505+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/1067840780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bd520 0x5622950c56e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.505+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 >> 172.21.15.57:0/1067840780 conn(0x562295032200 msgr2=0x5622950b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.505+0000 7f6a8dffb700 0 client.16341 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.506+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 shutdown_connections 2023-10-06T22:17:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.506+0000 7f6ac077ab80 1 -- 172.21.15.57:0/1067840780 wait complete. 2023-10-06T22:17:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.508+0000 7f6ac077ab80 20 client.16341 trim_cache size 0 max 16384 2023-10-06T22:17:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.546+0000 7f6ac077ab80 1 Processor -- start 2023-10-06T22:17:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.547+0000 7f6ac077ab80 1 -- start start 2023-10-06T22:17:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.547+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bcc00 0x5622950cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.547+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 0x5622950c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.547+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.547+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562294e918c0 con 0x5622950bcc00 2023-10-06T22:17:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.547+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622950ba610 con 0x5622950c83d0 2023-10-06T22:17:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.547+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622950bd900 con 0x5622950bd520 2023-10-06T22:17:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:58450/0 (socket says 172.21.15.57:58450) 2023-10-06T22:17:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/2343470868 learned_addr learned my addr 172.21.15.57:0/2343470868 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/2343470868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 msgr2=0x5622950c5710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:17:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 --2- 172.21.15.57:0/2343470868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 0x5622950c5710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/2343470868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bcc00 msgr2=0x5622950cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 --2- 172.21.15.57:0/2343470868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bcc00 0x5622950cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.548+0000 7f6aa6d9d700 1 -- 172.21.15.57:0/2343470868 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562294e621f0 con 0x5622950c83d0 2023-10-06T22:17:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.549+0000 7f6aa6d9d700 1 --2- 172.21.15.57:0/2343470868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6aa009ae30 tx=0x7f6aa00a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a01aed0211d90d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.550+0000 7f6a957fa700 1 -- 172.21.15.57:0/2343470868 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa009eff0 con 0x5622950c83d0 2023-10-06T22:17:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.550+0000 7f6a957fa700 1 -- 172.21.15.57:0/2343470868 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa00a31a0 con 0x5622950c83d0 2023-10-06T22:17:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.550+0000 7f6a957fa700 1 -- 172.21.15.57:0/2343470868 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa009b320 con 0x5622950c83d0 2023-10-06T22:17:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.551+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2343470868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 msgr2=0x5622950c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.551+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2343470868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6aa009ae30 tx=0x7f6aa00a1040 comp rx=0 tx=0).stop 2023-10-06T22:17:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.551+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2343470868 shutdown_connections 2023-10-06T22:17:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.551+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2343470868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950c83d0 0x5622950c5c50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.551+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2343470868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950bcc00 0x5622950cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.552+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/2343470868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 0x5622950c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.552+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2343470868 >> 172.21.15.57:0/2343470868 conn(0x562295032200 msgr2=0x5622950c04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.552+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2343470868 shutdown_connections 2023-10-06T22:17:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.552+0000 7f6ac077ab80 1 -- 172.21.15.57:0/2343470868 wait complete. 2023-10-06T22:17:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.553+0000 7f6ac077ab80 1 -- start start 2023-10-06T22:17:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.553+0000 7f6ac077ab80 1 Processor -- start 2023-10-06T22:17:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.554+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950bcc00 0x56229511fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.554+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 0x5622951201a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.554+0000 7f6ac077ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950c83d0 0x5622951206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.554+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56229512ad20 con 0x5622950c83d0 2023-10-06T22:17:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.554+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56229512ae60 con 0x5622950bcc00 2023-10-06T22:17:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.554+0000 7f6ac077ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56229512b020 con 0x5622950bd520 2023-10-06T22:17:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.554+0000 7f6ac077ab80 10 client.0 _ll_register_callbacks cb 0x7f6aa0009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.555+0000 7f6aa659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950bcc00 0x56229511fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.555+0000 7f6aa659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950bcc00 0x56229511fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:58454/0 (socket says 172.21.15.57:58454) 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.555+0000 7f6aa659c700 1 -- 172.21.15.57:0/6438620 learned_addr learned my addr 172.21.15.57:0/6438620 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.556+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 0x5622951201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.556+0000 7f6aa659c700 1 -- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 msgr2=0x5622951201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.556+0000 7f6aa659c700 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 0x5622951201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.556+0000 7f6aa659c700 1 -- 172.21.15.57:0/6438620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950c83d0 msgr2=0x5622951206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.556+0000 7f6aa659c700 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950c83d0 0x5622951206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.556+0000 7f6aa659c700 1 -- 172.21.15.57:0/6438620 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa819a030 con 0x5622950bcc00 2023-10-06T22:17:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.557+0000 7f6aa659c700 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950bcc00 0x56229511fc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a98005d90 tx=0x7f6a980047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa7fc823cc73e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.557+0000 7f6a8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:17:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.557+0000 7f6a8dffb700 1 -- 172.21.15.57:0/6438620 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004b10 con 0x5622950bcc00 2023-10-06T22:17:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.557+0000 7f6a8dffb700 1 -- 172.21.15.57:0/6438620 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f6a98004c70 con 0x5622950bcc00 2023-10-06T22:17:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.558+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a9c00dc90 con 0x5622950bcc00 2023-10-06T22:17:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.558+0000 7f6a8dffb700 1 -- 172.21.15.57:0/6438620 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a98004e30 con 0x5622950bcc00 2023-10-06T22:17:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.558+0000 7f6ac077ab80 10 client.26360 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.558+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562294e3a250 con 0x5622950bcc00 2023-10-06T22:17:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.559+0000 7f6a8dffb700 1 -- 172.21.15.57:0/6438620 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9c00dc90 con 0x5622950bcc00 2023-10-06T22:17:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.560+0000 7f6a8dffb700 1 -- 172.21.15.57:0/6438620 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6a98005620 con 0x5622950bcc00 2023-10-06T22:17:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.560+0000 7f6ac077ab80 10 client.26360 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.560+0000 7f6ac077ab80 10 client.26360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.561+0000 7f6ac077ab80 20 client.26360 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.561+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56229512c7e0 0x56229512ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.561+0000 7f6ac077ab80 4 client.26360 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:17:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.561+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562295009ad0 con 0x56229512c7e0 2023-10-06T22:17:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.561+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56229512c7e0 0x56229512ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.562+0000 7f6a8dffb700 10 client.26360 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:20.562+0000 7f6aa5d9b700 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56229512c7e0 0x56229512ec00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473fd52e428b78fa server_cookie=571fc8b672e6c838 in_seq=0 out_seq=0 2023-10-06T22:17:21.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:20 smithi190 ceph-mon[134891]: pgmap v81: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 559 op/s 2023-10-06T22:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.136+0000 7f6a8dffb700 1 -- 172.21.15.57:0/6438620 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562295009ad0 con 0x56229512c7e0 2023-10-06T22:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.136+0000 7f6a8dffb700 10 client.26360 handle_command_reply: tid=0 2023-10-06T22:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.136+0000 7f6ac077ab80 1 client.26360 shutdown 2023-10-06T22:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950bcc00 msgr2=0x56229511fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950bcc00 0x56229511fc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a98005d90 tx=0x7f6a980047c0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 shutdown_connections 2023-10-06T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56229512c7e0 0x56229512ec00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622950c83d0 0x5622951206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622950bcc00 0x56229511fc60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6a8dffb700 0 client.26360 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 --2- 172.21.15.57:0/6438620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622950bd520 0x5622951201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 >> 172.21.15.57:0/6438620 conn(0x562295032200 msgr2=0x5622950c04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 shutdown_connections 2023-10-06T22:17:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.137+0000 7f6ac077ab80 1 -- 172.21.15.57:0/6438620 wait complete. 2023-10-06T22:17:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.138+0000 7f6ac077ab80 20 client.26360 trim_cache size 0 max 16384 2023-10-06T22:17:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:17:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:17:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:17:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.138+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6a900a1780 msgr2=0x7f6a900a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.138+0000 7f6aacdff700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6a900a1780 0x7f6a900a3c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009b00 tx=0x7f6a98009ec0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.138+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa810a9e0 msgr2=0x7f6aa810ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.138+0000 7f6aacdff700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa810a9e0 0x7f6aa810ae60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00062a0 tx=0x7f6aa000bd30 comp rx=0 tx=0).stop 2023-10-06T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 shutdown_connections 2023-10-06T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f6a900a1780 0x7f6a900a3c40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa810a9e0 0x7f6aa810ae60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6aa81010c0 0x7f6aa819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 --2- 172.21.15.57:0/3419704835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aa81061e0 0x7f6aa819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 >> 172.21.15.57:0/3419704835 conn(0x7f6aa80fc940 msgr2=0x7f6aa810c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 shutdown_connections 2023-10-06T22:17:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:21.139+0000 7f6aacdff700 1 -- 172.21.15.57:0/3419704835 wait complete. 2023-10-06T22:17:21.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:20 smithi057 ceph-mon[98492]: pgmap v81: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 559 op/s 2023-10-06T22:17:21.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:20 smithi156 ceph-mon[134358]: pgmap v81: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 559 op/s 2023-10-06T22:17:22.033 INFO:tasks.workunit.client.0.smithi057.stdout: 78 321686 548 248546 704 118906 651 2023-10-06T22:17:23.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:22 smithi057 ceph-mon[98492]: pgmap v82: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.0 MiB/s wr, 565 op/s 2023-10-06T22:17:23.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:22 smithi156 ceph-mon[134358]: pgmap v82: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.0 MiB/s wr, 565 op/s 2023-10-06T22:17:23.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:22 smithi190 ceph-mon[134891]: pgmap v82: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.0 MiB/s wr, 565 op/s 2023-10-06T22:17:25.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:24 smithi057 ceph-mon[98492]: pgmap v83: 97 pgs: 97 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.7 MiB/s wr, 737 op/s 2023-10-06T22:17:25.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:24 smithi156 ceph-mon[134358]: pgmap v83: 97 pgs: 97 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.7 MiB/s wr, 737 op/s 2023-10-06T22:17:25.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:24 smithi190 ceph-mon[134891]: pgmap v83: 97 pgs: 97 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.7 MiB/s wr, 737 op/s 2023-10-06T22:17:26.907 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:17:27.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:26 smithi057 ceph-mon[98492]: pgmap v84: 97 pgs: 97 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2023-10-06T22:17:27.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:26 smithi156 ceph-mon[134358]: pgmap v84: 97 pgs: 97 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2023-10-06T22:17:27.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:26 smithi190 ceph-mon[134891]: pgmap v84: 97 pgs: 97 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2023-10-06T22:17:27.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.632+0000 7f9cc53ff700 1 -- 172.21.15.57:0/3879158491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc0108870 msgr2=0x7f9cc0079110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:27.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.632+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/3879158491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc0108870 0x7f9cc0079110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0008580 tx=0x7f9cb0008890 comp rx=0 tx=0).stop 2023-10-06T22:17:27.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.632+0000 7f9cc53ff700 1 -- 172.21.15.57:0/3879158491 shutdown_connections 2023-10-06T22:17:27.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.632+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/3879158491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc0108870 0x7f9cc0079110 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.632+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/3879158491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc010d560 0x7f9cc010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.632+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/3879158491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cc0107eb0 0x7f9cc0108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.632+0000 7f9cc53ff700 1 -- 172.21.15.57:0/3879158491 >> 172.21.15.57:0/3879158491 conn(0x7f9cc006d1a0 msgr2=0x7f9cc006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.637+0000 7f9cc53ff700 1 -- 172.21.15.57:0/3879158491 shutdown_connections 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.637+0000 7f9cc53ff700 1 -- 172.21.15.57:0/3879158491 wait complete. 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 Processor -- start 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 -- start start 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc0107eb0 0x7f9cc007c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cc0108870 0x7f9cc007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc010d560 0x7f9cc0081000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0081690 con 0x7f9cc0107eb0 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0081800 con 0x7f9cc010d560 2023-10-06T22:17:27.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.639+0000 7f9cc53ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc00819c0 con 0x7f9cc0108870 2023-10-06T22:17:27.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc0107eb0 0x7f9cc007c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc0107eb0 0x7f9cc007c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55138/0 (socket says 172.21.15.57:55138) 2023-10-06T22:17:27.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 -- 172.21.15.57:0/4137005398 learned_addr learned my addr 172.21.15.57:0/4137005398 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:27.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 -- 172.21.15.57:0/4137005398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cc0108870 msgr2=0x7f9cc007cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:17:27.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cc0108870 0x7f9cc007cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 -- 172.21.15.57:0/4137005398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc010d560 msgr2=0x7f9cc0081000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:17:27.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc010d560 0x7f9cc0081000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.641+0000 7f9cbeffd700 1 -- 172.21.15.57:0/4137005398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0008230 con 0x7f9cc0107eb0 2023-10-06T22:17:27.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.642+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc0107eb0 0x7f9cc007c5f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400abb0 tx=0x7f9cb400af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:27.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.650+0000 7f9ca7fff700 1 -- 172.21.15.57:0/4137005398 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4010070 con 0x7f9cc0107eb0 2023-10-06T22:17:27.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.650+0000 7f9ca7fff700 1 -- 172.21.15.57:0/4137005398 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb4007d80 con 0x7f9cc0107eb0 2023-10-06T22:17:27.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.651+0000 7f9ca7fff700 1 -- 172.21.15.57:0/4137005398 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40133b0 con 0x7f9cc0107eb0 2023-10-06T22:17:27.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.652+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc0081ca0 con 0x7f9cc0107eb0 2023-10-06T22:17:27.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.652+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc01cc540 con 0x7f9cc0107eb0 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.659+0000 7f9cd8dfeb80 1 Processor -- start 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.665+0000 7f9cd8dfeb80 1 -- start start 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.665+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.665+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.665+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8eec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.665+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc84a460 con 0x5637dc8e19e0 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.665+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc83bc20 con 0x5637dc8ec890 2023-10-06T22:17:27.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.665+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc839540 con 0x5637dc8e10c0 2023-10-06T22:17:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.666+0000 7f9ca7fff700 1 -- 172.21.15.57:0/4137005398 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9cb40078c0 con 0x7f9cc0107eb0 2023-10-06T22:17:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.669+0000 7f9ca7fff700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9ca80a17c0 0x7f9ca80a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.670+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.670+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45168/0 (socket says 172.21.15.57:45168) 2023-10-06T22:17:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.670+0000 7f9cbf7fe700 1 -- 172.21.15.57:0/4074306120 learned_addr learned my addr 172.21.15.57:0/4074306120 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.677+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.693+0000 7f9cbf7fe700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9ca80a17c0 0x7f9ca80a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.693+0000 7f9cbf7fe700 1 -- 172.21.15.57:0/4074306120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 msgr2=0x5637dc8eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.693+0000 7f9cbf7fe700 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8eec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.693+0000 7f9cbf7fe700 1 -- 172.21.15.57:0/4074306120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 msgr2=0x5637dc8ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.693+0000 7f9cbf7fe700 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.693+0000 7f9cbf7fe700 1 -- 172.21.15.57:0/4074306120 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637dc65fde0 con 0x5637dc8e10c0 2023-10-06T22:17:27.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.699+0000 7f9cbf7fe700 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e14a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000f240 tx=0x7f9cb000f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d404142d5f98c9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:27.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.702+0000 7f9ca7fff700 1 -- 172.21.15.57:0/4137005398 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9cb40cc560 con 0x7f9cc0107eb0 2023-10-06T22:17:27.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.710+0000 7f9ca57fa700 1 -- 172.21.15.57:0/4074306120 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000aa30 con 0x5637dc8e10c0 2023-10-06T22:17:27.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.710+0000 7f9ca57fa700 1 -- 172.21.15.57:0/4074306120 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb000f9c0 con 0x5637dc8e10c0 2023-10-06T22:17:27.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.710+0000 7f9ca57fa700 1 -- 172.21.15.57:0/4074306120 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0017ad0 con 0x5637dc8e10c0 2023-10-06T22:17:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.713+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/4074306120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 msgr2=0x5637dc8e14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.713+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e14a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000f240 tx=0x7f9cb000f270 comp rx=0 tx=0).stop 2023-10-06T22:17:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.713+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/4074306120 shutdown_connections 2023-10-06T22:17:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.713+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.713+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e14a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.713+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/4074306120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.713+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/4074306120 >> 172.21.15.57:0/4074306120 conn(0x5637dc856260 msgr2=0x5637dc8dd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:27.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.718+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/4074306120 shutdown_connections 2023-10-06T22:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.723+0000 7f9cbf7fe700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9ca80a17c0 0x7f9ca80a3c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0028000 tx=0x7f9cb0022710 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:27.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.726+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/4074306120 wait complete. 2023-10-06T22:17:27.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.727+0000 7f9cd8dfeb80 1 -- start start 2023-10-06T22:17:27.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.727+0000 7f9cd8dfeb80 1 Processor -- start 2023-10-06T22:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.728+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e10c0 0x5637dc8e9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.728+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.728+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8ec890 0x5637dc8ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.728+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc834e30 con 0x5637dc8e19e0 2023-10-06T22:17:27.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.728+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc7d1710 con 0x5637dc8e10c0 2023-10-06T22:17:27.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.728+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc7d13b0 con 0x5637dc8ec890 2023-10-06T22:17:27.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.728+0000 7f9cd8dfeb80 10 client.0 _ll_register_callbacks cb 0x5637dc62e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:27.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.729+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8ec890 0x5637dc8ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.729+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e10c0 0x5637dc8e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.730+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8ec890 0x5637dc8ea090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45170/0 (socket says 172.21.15.57:45170) 2023-10-06T22:17:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.730+0000 7f9cbf7fe700 1 -- 172.21.15.57:0/2283750958 learned_addr learned my addr 172.21.15.57:0/2283750958 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.730+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.740+0000 7f9cbeffd700 1 -- 172.21.15.57:0/2283750958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8ec890 msgr2=0x5637dc8ea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.740+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8ec890 0x5637dc8ea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.740+0000 7f9cbeffd700 1 -- 172.21.15.57:0/2283750958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 msgr2=0x5637dc8e9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.740+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8e9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.740+0000 7f9cbeffd700 1 -- 172.21.15.57:0/2283750958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc0079b40 con 0x5637dc8e10c0 2023-10-06T22:17:27.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.748+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e10c0 0x5637dc8e9610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40c9cc0 tx=0x7f9cb4099160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa4e7c7b647cbcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.753+0000 7f9ca2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:17:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.754+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2283750958 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4099550 con 0x5637dc8e10c0 2023-10-06T22:17:27.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.754+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2283750958 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9cb409d6a0 con 0x5637dc8e10c0 2023-10-06T22:17:27.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.756+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637dc8d4f10 con 0x5637dc8e10c0 2023-10-06T22:17:27.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.762+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2283750958 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb40a18e0 con 0x5637dc8e10c0 2023-10-06T22:17:27.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.762+0000 7f9cd8dfeb80 10 client.26369 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:27.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.762+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5637dc648c60 con 0x5637dc8e10c0 2023-10-06T22:17:27.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.768+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2283750958 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637dc8d4f10 con 0x5637dc8e10c0 2023-10-06T22:17:27.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.768+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2283750958 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9cb4098a40 con 0x5637dc8e10c0 2023-10-06T22:17:27.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.768+0000 7f9cd8dfeb80 10 client.26369 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:27.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.768+0000 7f9cd8dfeb80 10 client.26369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:27.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.769+0000 7f9cd8dfeb80 20 client.26369 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:27.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.769+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc8fc2a0 0x5637dc8fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.769+0000 7f9cd8dfeb80 4 client.26369 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:17:27.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.769+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637dc82de10 con 0x5637dc8fc2a0 2023-10-06T22:17:27.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.769+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc8fc2a0 0x5637dc8fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.790+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc8fc2a0 0x5637dc8fe6c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe9efd76f5e10f6 server_cookie=62857603c5f994d0 in_seq=0 out_seq=0 2023-10-06T22:17:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.794+0000 7f9ca2ff5700 10 client.26369 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.794+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2283750958 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5637dc82de10 con 0x5637dc8fc2a0 2023-10-06T22:17:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.794+0000 7f9ca2ff5700 10 client.26369 handle_command_reply: tid=0 2023-10-06T22:17:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.794+0000 7f9cd8dfeb80 1 client.26369 shutdown 2023-10-06T22:17:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.794+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e10c0 msgr2=0x5637dc8e9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:27.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.794+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e10c0 0x5637dc8e9610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40c9cc0 tx=0x7f9cb4099160 comp rx=0 tx=0).stop 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.795+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 shutdown_connections 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.795+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc8fc2a0 0x5637dc8fe6c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.795+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8ec890 0x5637dc8ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.795+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e10c0 0x5637dc8e9610 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.795+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2283750958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.795+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 >> 172.21.15.57:0/2283750958 conn(0x5637dc856260 msgr2=0x5637dc8dcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.795+0000 7f9ca2ff5700 0 client.26369 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.799+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 shutdown_connections 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.799+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2283750958 wait complete. 2023-10-06T22:17:27.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.803+0000 7f9cd8dfeb80 20 client.26369 trim_cache size 0 max 16384 2023-10-06T22:17:27.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.845+0000 7f9cd8dfeb80 1 Processor -- start 2023-10-06T22:17:27.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.846+0000 7f9cd8dfeb80 1 -- start start 2023-10-06T22:17:27.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.846+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.846+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e19e0 0x5637dc8f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.846+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8ec890 0x5637dc8e9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.846+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc6b5920 con 0x5637dc8ec890 2023-10-06T22:17:27.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.846+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc8ec6c0 con 0x5637dc8e19e0 2023-10-06T22:17:27.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.846+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc8d4b30 con 0x5637dc8e10c0 2023-10-06T22:17:27.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45172/0 (socket says 172.21.15.57:45172) 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 -- 172.21.15.57:0/260161755 learned_addr learned my addr 172.21.15.57:0/260161755 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e19e0 0x5637dc8f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 -- 172.21.15.57:0/260161755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e19e0 msgr2=0x5637dc8f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e19e0 0x5637dc8f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 -- 172.21.15.57:0/260161755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8ec890 msgr2=0x5637dc8e9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8ec890 0x5637dc8e9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.847+0000 7f9cbeffd700 1 -- 172.21.15.57:0/260161755 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637dc9032f0 con 0x5637dc8e10c0 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.848+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e4010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4095f70 tx=0x7f9cb4097b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c4ce28606246b6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.848+0000 7f9ca5ffb700 1 -- 172.21.15.57:0/260161755 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4098eb0 con 0x5637dc8e10c0 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.848+0000 7f9ca5ffb700 1 -- 172.21.15.57:0/260161755 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb40a16d0 con 0x5637dc8e10c0 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.848+0000 7f9ca5ffb700 1 -- 172.21.15.57:0/260161755 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4098a40 con 0x5637dc8e10c0 2023-10-06T22:17:27.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/260161755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 msgr2=0x5637dc8e4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e4010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4095f70 tx=0x7f9cb4097b00 comp rx=0 tx=0).stop 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/260161755 shutdown_connections 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8ec890 0x5637dc8e9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8e4010 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/260161755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8e19e0 0x5637dc8f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/260161755 >> 172.21.15.57:0/260161755 conn(0x5637dc856260 msgr2=0x5637dc8e48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/260161755 shutdown_connections 2023-10-06T22:17:27.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.849+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/260161755 wait complete. 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.850+0000 7f9cd8dfeb80 1 -- start start 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.850+0000 7f9cd8dfeb80 1 Processor -- start 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.851+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.851+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.851+0000 7f9cd8dfeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.851+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc94f200 con 0x5637dc8e19e0 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.851+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc94f370 con 0x5637dc8ec890 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.851+0000 7f9cd8dfeb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637dc94f530 con 0x5637dc8e10c0 2023-10-06T22:17:27.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.851+0000 7f9cd8dfeb80 10 client.0 _ll_register_callbacks cb 0x5637dc62e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:27.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.852+0000 7f9cbe7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.852+0000 7f9cbe7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55176/0 (socket says 172.21.15.57:55176) 2023-10-06T22:17:27.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.852+0000 7f9cbe7fc700 1 -- 172.21.15.57:0/2429074142 learned_addr learned my addr 172.21.15.57:0/2429074142 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:27.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.853+0000 7f9cbf7fe700 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.853+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.853+0000 7f9cbe7fc700 1 -- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 msgr2=0x5637dc8f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.853+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.853+0000 7f9cbe7fc700 1 -- 172.21.15.57:0/2429074142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 msgr2=0x5637dc8f16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:27.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.853+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8f16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.853+0000 7f9cbe7fc700 1 -- 172.21.15.57:0/2429074142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637dc648c60 con 0x5637dc8e19e0 2023-10-06T22:17:27.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.856+0000 7f9cbe7fc700 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8f1170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5637dc8fefd0 tx=0x7f9cb801bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecb146f522a5e90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:27.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.856+0000 7f9ca2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:17:27.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.856+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2429074142 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8010040 con 0x5637dc8e19e0 2023-10-06T22:17:27.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.856+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cb000f9c0 con 0x5637dc8e19e0 2023-10-06T22:17:27.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.857+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2429074142 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f9cb800bb70 con 0x5637dc8e19e0 2023-10-06T22:17:27.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.860+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2429074142 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb800bd30 con 0x5637dc8e19e0 2023-10-06T22:17:27.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.860+0000 7f9cd8dfeb80 10 client.16353 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:27.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.860+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5637dc686250 con 0x5637dc8e19e0 2023-10-06T22:17:27.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2429074142 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb000f9c0 con 0x5637dc8e19e0 2023-10-06T22:17:27.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2429074142 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9cb802c070 con 0x5637dc8e19e0 2023-10-06T22:17:27.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9cd8dfeb80 10 client.16353 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:27.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9cd8dfeb80 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:27.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9cd8dfeb80 20 client.16353 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:27.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc950db0 0x5637dc9531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:27.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9cd8dfeb80 4 client.16353 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:17:27.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637dc82db30 con 0x5637dc950db0 2023-10-06T22:17:27.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.861+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc950db0 0x5637dc9531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:27.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.862+0000 7f9cbeffd700 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc950db0 0x5637dc9531d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc647e2e81caf08 server_cookie=57288b717114c34f in_seq=0 out_seq=0 2023-10-06T22:17:27.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.862+0000 7f9ca2ff5700 10 client.16353 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:27.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.862+0000 7f9ca2ff5700 1 -- 172.21.15.57:0/2429074142 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5637dc82db30 con 0x5637dc950db0 2023-10-06T22:17:27.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.862+0000 7f9ca2ff5700 10 client.16353 handle_command_reply: tid=0 2023-10-06T22:17:27.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.862+0000 7f9cd8dfeb80 1 client.16353 shutdown 2023-10-06T22:17:27.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 msgr2=0x5637dc8f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:27.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8f1170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5637dc8fefd0 tx=0x7f9cb801bd50 comp rx=0 tx=0).stop 2023-10-06T22:17:27.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 shutdown_connections 2023-10-06T22:17:27.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5637dc950db0 0x5637dc9531d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637dc8ec890 0x5637dc8f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637dc8e10c0 0x5637dc8f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9ca2ff5700 0 client.16353 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 --2- 172.21.15.57:0/2429074142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5637dc8e19e0 0x5637dc8f1170 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 >> 172.21.15.57:0/2429074142 conn(0x5637dc856260 msgr2=0x5637dc8e7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.863+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 shutdown_connections 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.864+0000 7f9cd8dfeb80 1 -- 172.21.15.57:0/2429074142 wait complete. 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.864+0000 7f9cd8dfeb80 20 client.16353 trim_cache size 0 max 16384 2023-10-06T22:17:27.867 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:17:27.868 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9ca80a17c0 msgr2=0x7f9ca80a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9ca80a17c0 0x7f9ca80a3c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0028000 tx=0x7f9cb0022710 comp rx=0 tx=0).stop 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc0107eb0 msgr2=0x7f9cc007c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc0107eb0 0x7f9cc007c5f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400abb0 tx=0x7f9cb400af70 comp rx=0 tx=0).stop 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 shutdown_connections 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9ca80a17c0 0x7f9ca80a3c80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc010d560 0x7f9cc0081000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc0107eb0 0x7f9cc007c5f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 --2- 172.21.15.57:0/4137005398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cc0108870 0x7f9cc007cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:27.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 >> 172.21.15.57:0/4137005398 conn(0x7f9cc006d1a0 msgr2=0x7f9cc00712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:27.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.865+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 shutdown_connections 2023-10-06T22:17:27.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:17:27.866+0000 7f9cc53ff700 1 -- 172.21.15.57:0/4137005398 wait complete. 2023-10-06T22:17:27.904 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:17:29.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:28 smithi057 ceph-mon[98492]: pgmap v85: 97 pgs: 97 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 690 op/s 2023-10-06T22:17:29.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:28 smithi156 ceph-mon[134358]: pgmap v85: 97 pgs: 97 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 690 op/s 2023-10-06T22:17:29.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:28 smithi190 ceph-mon[134891]: pgmap v85: 97 pgs: 97 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 690 op/s 2023-10-06T22:17:31.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:30 smithi057 ceph-mon[98492]: pgmap v86: 97 pgs: 97 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 368 op/s 2023-10-06T22:17:31.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:30 smithi156 ceph-mon[134358]: pgmap v86: 97 pgs: 97 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 368 op/s 2023-10-06T22:17:31.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:30 smithi190 ceph-mon[134891]: pgmap v86: 97 pgs: 97 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 368 op/s 2023-10-06T22:17:32.033 INFO:tasks.workunit.client.0.smithi057.stdout: 95 428299 805 314499 808 144720 461 2023-10-06T22:17:33.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:32 smithi057 ceph-mon[98492]: pgmap v87: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 416 op/s 2023-10-06T22:17:33.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:32 smithi156 ceph-mon[134358]: pgmap v87: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 416 op/s 2023-10-06T22:17:33.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:32 smithi190 ceph-mon[134891]: pgmap v87: 97 pgs: 97 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 416 op/s 2023-10-06T22:17:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 -- 172.21.15.57:0/632618309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d4073ba0 msgr2=0x7f06d410d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 --2- 172.21.15.57:0/632618309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d4073ba0 0x7f06d410d1f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009990 tx=0x7f06c8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 -- 172.21.15.57:0/632618309 shutdown_connections 2023-10-06T22:17:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 --2- 172.21.15.57:0/632618309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d410d730 0x7f06d410fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 --2- 172.21.15.57:0/632618309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d4073280 0x7f06d4073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 --2- 172.21.15.57:0/632618309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d4073ba0 0x7f06d410d1f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 -- 172.21.15.57:0/632618309 >> 172.21.15.57:0/632618309 conn(0x7f06d40fc9a0 msgr2=0x7f06d40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 -- 172.21.15.57:0/632618309 shutdown_connections 2023-10-06T22:17:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.354+0000 7f06d9bff700 1 -- 172.21.15.57:0/632618309 wait complete. 2023-10-06T22:17:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 Processor -- start 2023-10-06T22:17:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 -- start start 2023-10-06T22:17:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d4073280 0x7f06d4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d4073ba0 0x7f06d4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d410d730 0x7f06d419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d419e430 con 0x7f06d4073280 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d419e5a0 con 0x7f06d410d730 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.355+0000 7f06d9bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d419e760 con 0x7f06d4073ba0 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d4073ba0 0x7f06d4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d4073ba0 0x7f06d4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52602/0 (socket says 172.21.15.57:52602) 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3478810947 learned_addr learned my addr 172.21.15.57:0/3478810947 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d410d730 0x7f06d419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d37fe700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d4073280 0x7f06d4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3478810947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d410d730 msgr2=0x7f06d419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d410d730 0x7f06d419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d4073280 msgr2=0x7f06d4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d4073280 0x7f06d4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3478810947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80095f0 con 0x7f06d4073ba0 2023-10-06T22:17:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.356+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d4073ba0 0x7f06d4199780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005910 tx=0x7f06c8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.357+0000 7f06d0ff9700 1 -- 172.21.15.57:0/3478810947 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c801d070 con 0x7f06d4073ba0 2023-10-06T22:17:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.357+0000 7f06d0ff9700 1 -- 172.21.15.57:0/3478810947 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06c8022470 con 0x7f06d4073ba0 2023-10-06T22:17:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.357+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d419e9e0 con 0x7f06d4073ba0 2023-10-06T22:17:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.357+0000 7f06d0ff9700 1 -- 172.21.15.57:0/3478810947 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8017650 con 0x7f06d4073ba0 2023-10-06T22:17:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.357+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d41a6280 con 0x7f06d4073ba0 2023-10-06T22:17:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.360+0000 7f06d0ff9700 1 -- 172.21.15.57:0/3478810947 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f06c800fad0 con 0x7f06d4073ba0 2023-10-06T22:17:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.362+0000 7f06d0ff9700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06bc0a1670 0x7f06bc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.362+0000 7f06d0ff9700 1 -- 172.21.15.57:0/3478810947 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f06c80ce5e0 con 0x7f06d4073ba0 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.362+0000 7f06d37fe700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06bc0a1670 0x7f06bc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 Processor -- start 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 -- start start 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3410 0x5639b44f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3d30 0x5639b44f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b4509a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b4454400 con 0x5639b4507650 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b4445bc0 con 0x5639b44f3d30 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.363+0000 7f06ed574b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b44434e0 con 0x5639b44f3410 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3d30 0x5639b44f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3d30 0x5639b44f0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:52426/0 (socket says 172.21.15.57:52426) 2023-10-06T22:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 -- 172.21.15.57:0/2319015028 learned_addr learned my addr 172.21.15.57:0/2319015028 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d37fe700 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3410 0x5639b44f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 -- 172.21.15.57:0/2319015028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3410 msgr2=0x5639b44f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3410 0x5639b44f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d3fff700 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b4509a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 -- 172.21.15.57:0/2319015028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 msgr2=0x5639b4509a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b4509a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d37fe700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06bc0a1670 0x7f06bc0a3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06c4005950 tx=0x7f06c400b410 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 -- 172.21.15.57:0/2319015028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b4269d80 con 0x5639b44f3d30 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d3fff700 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b4509a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3d30 0x5639b44f0be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06c80981f0 tx=0x7f06c809b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04476e73ca5f9e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.364+0000 7f06b9ffb700 1 -- 172.21.15.57:0/2319015028 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c809be80 con 0x5639b44f3d30 2023-10-06T22:17:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06b9ffb700 1 -- 172.21.15.57:0/2319015028 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06c809b780 con 0x5639b44f3d30 2023-10-06T22:17:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06b9ffb700 1 -- 172.21.15.57:0/2319015028 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c809b9e0 con 0x5639b44f3d30 2023-10-06T22:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06ed574b80 1 -- 172.21.15.57:0/2319015028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3d30 msgr2=0x5639b44f0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3d30 0x5639b44f0be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06c80981f0 tx=0x7f06c809b550 comp rx=0 tx=0).stop 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06ed574b80 1 -- 172.21.15.57:0/2319015028 shutdown_connections 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b4509a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3410 0x5639b44f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2319015028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3d30 0x5639b44f0be0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.365+0000 7f06ed574b80 1 -- 172.21.15.57:0/2319015028 >> 172.21.15.57:0/2319015028 conn(0x5639b4460200 msgr2=0x5639b44e6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.366+0000 7f06ed574b80 1 -- 172.21.15.57:0/2319015028 shutdown_connections 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.366+0000 7f06ed574b80 1 -- 172.21.15.57:0/2319015028 wait complete. 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.366+0000 7f06ed574b80 1 -- start start 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.366+0000 7f06ed574b80 1 Processor -- start 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b450c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b44f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06ed574b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b443edd0 con 0x5639b4507650 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06ed574b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b43db6b0 con 0x5639b44f3410 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06ed574b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b43db350 con 0x5639b44f3d30 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06ed574b80 10 client.0 _ll_register_callbacks cb 0x5639b42387a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06d37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b450c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06d37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b450c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:52428/0 (socket says 172.21.15.57:52428) 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.367+0000 7f06d37fe700 1 -- 172.21.15.57:0/301180369 learned_addr learned my addr 172.21.15.57:0/301180369 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06d37fe700 1 -- 172.21.15.57:0/301180369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 msgr2=0x5639b450cbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06d37fe700 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06d37fe700 1 -- 172.21.15.57:0/301180369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 msgr2=0x5639b44f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06d37fe700 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b44f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06d37fe700 1 -- 172.21.15.57:0/301180369 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d41a6280 con 0x5639b44f3410 2023-10-06T22:17:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06d37fe700 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b450c680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06c401e6c0 tx=0x7f06c40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3388dbe3ee54030 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:17:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06b2ffd700 1 -- 172.21.15.57:0/301180369 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4011850 con 0x5639b44f3410 2023-10-06T22:17:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639b44ded50 con 0x5639b44f3410 2023-10-06T22:17:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.368+0000 7f06b2ffd700 1 -- 172.21.15.57:0/301180369 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f06c4017070 con 0x5639b44f3410 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06b2ffd700 1 -- 172.21.15.57:0/301180369 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06c4010ea0 con 0x5639b44f3410 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 10 client.26387 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5639b4252c00 con 0x5639b44f3410 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06b2ffd700 1 -- 172.21.15.57:0/301180369 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639b44ded50 con 0x5639b44f3410 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06b2ffd700 1 -- 172.21.15.57:0/301180369 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f06c4026b80 con 0x5639b44f3410 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 10 client.26387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 20 client.26387 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b4516c80 0x5639b45190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 4 client.26387 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.369+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639b4437db0 con 0x5639b4516c80 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.370+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b4516c80 0x5639b45190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.370+0000 7f06b2ffd700 10 client.26387 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.370+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b4516c80 0x5639b45190a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a019259f25a10ff server_cookie=607bc9253bb7bee8 in_seq=0 out_seq=0 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06b2ffd700 1 -- 172.21.15.57:0/301180369 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5639b4437db0 con 0x5639b4516c80 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06b2ffd700 10 client.26387 handle_command_reply: tid=0 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 client.26387 shutdown 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 msgr2=0x5639b450c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b450c680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06c401e6c0 tx=0x7f06c40047c0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 shutdown_connections 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b4516c80 0x5639b45190a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b44f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06b2ffd700 0 client.26387 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b450c680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 --2- 172.21.15.57:0/301180369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.371+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 >> 172.21.15.57:0/301180369 conn(0x5639b4460200 msgr2=0x5639b44e6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.372+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 shutdown_connections 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.372+0000 7f06ed574b80 1 -- 172.21.15.57:0/301180369 wait complete. 2023-10-06T22:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.372+0000 7f06ed574b80 20 client.26387 trim_cache size 0 max 16384 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.388+0000 7f06ed574b80 1 Processor -- start 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.388+0000 7f06ed574b80 1 -- start start 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b44eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b44efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b44effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06ed574b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b42bf8c0 con 0x5639b4507650 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06ed574b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b44e83c0 con 0x5639b44f3410 2023-10-06T22:17:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06ed574b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b44e7db0 con 0x5639b44f3d30 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b44eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b44efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b44eab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:52438/0 (socket says 172.21.15.57:52438) 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b44efa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52620/0 (socket says 172.21.15.57:52620) 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d37fe700 1 -- 172.21.15.57:0/2920583333 learned_addr learned my addr 172.21.15.57:0/2920583333 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d3fff700 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b44effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d2ffd700 1 -- 172.21.15.57:0/2920583333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 msgr2=0x5639b44eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.389+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b44eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06d2ffd700 1 -- 172.21.15.57:0/2920583333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 msgr2=0x5639b44effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b44effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06d2ffd700 1 -- 172.21.15.57:0/2920583333 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b42901f0 con 0x5639b44f3d30 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06d37fe700 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b44eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:17:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b44efa60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5639b4507e80 tx=0x7f06c8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e228b0ba36a5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06ba7fc700 1 -- 172.21.15.57:0/2920583333 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c809f8d0 con 0x5639b44f3d30 2023-10-06T22:17:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06ba7fc700 1 -- 172.21.15.57:0/2920583333 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06c80a3a80 con 0x5639b44f3d30 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.390+0000 7f06ba7fc700 1 -- 172.21.15.57:0/2920583333 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c809ac70 con 0x5639b44f3d30 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 -- 172.21.15.57:0/2920583333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 msgr2=0x5639b44efa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b44efa60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5639b4507e80 tx=0x7f06c8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 -- 172.21.15.57:0/2920583333 shutdown_connections 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b4507650 0x5639b44effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b44f3410 0x5639b44eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 --2- 172.21.15.57:0/2920583333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b44efa60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 -- 172.21.15.57:0/2920583333 >> 172.21.15.57:0/2920583333 conn(0x5639b4460200 msgr2=0x5639b44e6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 -- 172.21.15.57:0/2920583333 shutdown_connections 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.391+0000 7f06ed574b80 1 -- 172.21.15.57:0/2920583333 wait complete. 2023-10-06T22:17:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.392+0000 7f06ed574b80 1 -- start start 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.392+0000 7f06ed574b80 1 Processor -- start 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b44f3410 0x5639b450a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06ed574b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b4507650 0x5639b4569af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06ed574b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b450b020 con 0x5639b44f3410 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06ed574b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b450b190 con 0x5639b4507650 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06ed574b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b456a060 con 0x5639b44f3d30 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06ed574b80 10 client.0 _ll_register_callbacks cb 0x7f06c4010d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06d2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:52634/0 (socket says 172.21.15.57:52634) 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3903477538 learned_addr learned my addr 172.21.15.57:0/3903477538 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06d37fe700 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b44f3410 0x5639b450a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.393+0000 7f06d3fff700 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b4507650 0x5639b4569af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3903477538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b4507650 msgr2=0x5639b4569af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b4507650 0x5639b4569af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3903477538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b44f3410 msgr2=0x5639b450a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b44f3410 0x5639b450a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06d2ffd700 1 -- 172.21.15.57:0/3903477538 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639b4252c00 con 0x5639b44f3d30 2023-10-06T22:17:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06d2ffd700 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450a870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06c80a3550 tx=0x7f06c8022b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45b0f9a934eaf3ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06b2ffd700 1 -- 172.21.15.57:0/3903477538 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c809f8d0 con 0x5639b44f3d30 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06b2ffd700 1 -- 172.21.15.57:0/3903477538 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f06c8099b20 con 0x5639b44f3d30 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06c809b780 con 0x5639b44f3d30 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06b2ffd700 1 -- 172.21.15.57:0/3903477538 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06c800fd80 con 0x5639b44f3d30 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06ed574b80 10 client.26362 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.394+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5639b4269d80 con 0x5639b44f3d30 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06b2ffd700 1 -- 172.21.15.57:0/3903477538 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06c809b780 con 0x5639b44f3d30 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06b2ffd700 1 -- 172.21.15.57:0/3903477538 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f06c80b7070 con 0x5639b44f3d30 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06ed574b80 10 client.26362 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06ed574b80 10 client.26362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06ed574b80 20 client.26362 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06ed574b80 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b456b3c0 0x5639b456d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06ed574b80 4 client.26362 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:17:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639b4437ad0 con 0x5639b456b3c0 2023-10-06T22:17:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.395+0000 7f06d37fe700 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b456b3c0 0x5639b456d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.396+0000 7f06d37fe700 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b456b3c0 0x5639b456d7e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98182cb355df2df1 server_cookie=1837d69c92f6793b in_seq=0 out_seq=0 2023-10-06T22:17:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.396+0000 7f06b2ffd700 10 client.26362 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06b2ffd700 1 -- 172.21.15.57:0/3903477538 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639b4437ad0 con 0x5639b456b3c0 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06b2ffd700 10 client.26362 handle_command_reply: tid=0 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 client.26362 shutdown 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 msgr2=0x5639b450a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450a870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06c80a3550 tx=0x7f06c8022b20 comp rx=0 tx=0).stop 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 shutdown_connections 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5639b456b3c0 0x5639b456d7e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639b4507650 0x5639b4569af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5639b44f3410 0x5639b450a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 --2- 172.21.15.57:0/3903477538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5639b44f3d30 0x5639b450a870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06b2ffd700 0 client.26362 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 >> 172.21.15.57:0/3903477538 conn(0x5639b4460200 msgr2=0x5639b44e6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 shutdown_connections 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 1 -- 172.21.15.57:0/3903477538 wait complete. 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.590+0000 7f06ed574b80 20 client.26362 trim_cache size 0 max 16384 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:17:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:17:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06bc0a1670 msgr2=0x7f06bc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06bc0a1670 0x7f06bc0a3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06c4005950 tx=0x7f06c400b410 comp rx=0 tx=0).stop 2023-10-06T22:17:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d4073ba0 msgr2=0x7f06d4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d4073ba0 0x7f06d4199780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005910 tx=0x7f06c8005740 comp rx=0 tx=0).stop 2023-10-06T22:17:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 shutdown_connections 2023-10-06T22:17:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06bc0a1670 0x7f06bc0a3b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06d410d730 0x7f06d419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06d4073280 0x7f06d4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 --2- 172.21.15.57:0/3478810947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06d4073ba0 0x7f06d4199780 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.591+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 >> 172.21.15.57:0/3478810947 conn(0x7f06d40fc9a0 msgr2=0x7f06d410f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.592+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 shutdown_connections 2023-10-06T22:17:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:34.592+0000 7f06d9bff700 1 -- 172.21.15.57:0/3478810947 wait complete. 2023-10-06T22:17:35.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:34 smithi057 ceph-mon[98492]: pgmap v88: 97 pgs: 97 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2023-10-06T22:17:35.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:34 smithi156 ceph-mon[134358]: pgmap v88: 97 pgs: 97 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2023-10-06T22:17:35.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:34 smithi190 ceph-mon[134891]: pgmap v88: 97 pgs: 97 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2023-10-06T22:17:37.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:36 smithi057 ceph-mon[98492]: pgmap v89: 97 pgs: 97 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 264 op/s 2023-10-06T22:17:37.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:36 smithi156 ceph-mon[134358]: pgmap v89: 97 pgs: 97 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 264 op/s 2023-10-06T22:17:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:36 smithi190 ceph-mon[134891]: pgmap v89: 97 pgs: 97 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 264 op/s 2023-10-06T22:17:39.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:38 smithi057 ceph-mon[98492]: pgmap v90: 97 pgs: 97 active+clean; 338 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2023-10-06T22:17:39.254 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:38 smithi156 ceph-mon[134358]: pgmap v90: 97 pgs: 97 active+clean; 338 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2023-10-06T22:17:39.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:38 smithi190 ceph-mon[134891]: pgmap v90: 97 pgs: 97 active+clean; 338 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2023-10-06T22:17:41.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:40 smithi156 ceph-mon[134358]: pgmap v91: 97 pgs: 97 active+clean; 338 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-10-06T22:17:41.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:40 smithi190 ceph-mon[134891]: pgmap v91: 97 pgs: 97 active+clean; 338 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-10-06T22:17:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:40 smithi057 ceph-mon[98492]: pgmap v91: 97 pgs: 97 active+clean; 338 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-10-06T22:17:42.043 INFO:tasks.workunit.client.0.smithi057.stdout: 107 404780 718 288962 576 108589 37 2023-10-06T22:17:43.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:42 smithi156 ceph-mon[134358]: pgmap v92: 97 pgs: 97 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 491 op/s 2023-10-06T22:17:43.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:42 smithi190 ceph-mon[134891]: pgmap v92: 97 pgs: 97 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 491 op/s 2023-10-06T22:17:43.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:42 smithi057 ceph-mon[98492]: pgmap v92: 97 pgs: 97 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 491 op/s 2023-10-06T22:17:44.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:44 smithi057 ceph-mon[98492]: pgmap v93: 97 pgs: 97 active+clean; 433 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 549 op/s 2023-10-06T22:17:44.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:44 smithi156 ceph-mon[134358]: pgmap v93: 97 pgs: 97 active+clean; 433 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 549 op/s 2023-10-06T22:17:44.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:44 smithi190 ceph-mon[134891]: pgmap v93: 97 pgs: 97 active+clean; 433 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 549 op/s 2023-10-06T22:17:46.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:45 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:17:46.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:45 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:17:46.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:45 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:17:47.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:46 smithi190 ceph-mon[134891]: pgmap v94: 97 pgs: 97 active+clean; 433 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 521 op/s 2023-10-06T22:17:47.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:46 smithi057 ceph-mon[98492]: pgmap v94: 97 pgs: 97 active+clean; 433 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 521 op/s 2023-10-06T22:17:47.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:46 smithi156 ceph-mon[134358]: pgmap v94: 97 pgs: 97 active+clean; 433 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 521 op/s 2023-10-06T22:17:49.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:48 smithi057 ceph-mon[98492]: pgmap v95: 97 pgs: 97 active+clean; 440 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 566 op/s 2023-10-06T22:17:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.180+0000 7f66f499d700 1 -- 172.21.15.57:0/2233716987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f010d8b0 msgr2=0x7f66f0119e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.180+0000 7f66f499d700 1 --2- 172.21.15.57:0/2233716987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f010d8b0 0x7f66f0119e00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f66e800b0a0 tx=0x7f66e800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.180+0000 7f66f499d700 1 -- 172.21.15.57:0/2233716987 shutdown_connections 2023-10-06T22:17:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.180+0000 7f66f499d700 1 --2- 172.21.15.57:0/2233716987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f011a340 0x7f66f011c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.180+0000 7f66f499d700 1 --2- 172.21.15.57:0/2233716987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f010cf90 0x7f66f010d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.180+0000 7f66f499d700 1 --2- 172.21.15.57:0/2233716987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f010d8b0 0x7f66f0119e00 secure :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f66e800b0a0 tx=0x7f66e800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.180+0000 7f66f499d700 1 -- 172.21.15.57:0/2233716987 >> 172.21.15.57:0/2233716987 conn(0x7f66f00721d0 msgr2=0x7f66f00725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.181+0000 7f66f499d700 1 -- 172.21.15.57:0/2233716987 shutdown_connections 2023-10-06T22:17:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.181+0000 7f66f499d700 1 -- 172.21.15.57:0/2233716987 wait complete. 2023-10-06T22:17:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.183+0000 7f66f499d700 1 Processor -- start 2023-10-06T22:17:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.183+0000 7f66f499d700 1 -- start start 2023-10-06T22:17:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.184+0000 7f66f499d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f010cf90 0x7f66f01a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.184+0000 7f66f499d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f011a340 0x7f66f01a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.184+0000 7f66f499d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f01aa8c0 0x7f66f01aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.184+0000 7f66f499d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ab340 con 0x7f66f01aa8c0 2023-10-06T22:17:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.184+0000 7f66f499d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ab480 con 0x7f66f010cf90 2023-10-06T22:17:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.184+0000 7f66f499d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ab5f0 con 0x7f66f011a340 2023-10-06T22:17:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.185+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f010cf90 0x7f66f01a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.185+0000 7f66effff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f01aa8c0 0x7f66f01aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.185+0000 7f66effff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f01aa8c0 0x7f66f01aad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55276/0 (socket says 172.21.15.57:55276) 2023-10-06T22:17:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.185+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f010cf90 0x7f66f01a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41062/0 (socket says 172.21.15.57:41062) 2023-10-06T22:17:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.185+0000 7f66effff700 1 -- 172.21.15.57:0/2980359155 learned_addr learned my addr 172.21.15.57:0/2980359155 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.185+0000 7f66eeffd700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f011a340 0x7f66f01a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.186+0000 7f66effff700 1 -- 172.21.15.57:0/2980359155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f011a340 msgr2=0x7f66f01a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.186+0000 7f66effff700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f011a340 0x7f66f01a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.186+0000 7f66effff700 1 -- 172.21.15.57:0/2980359155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f010cf90 msgr2=0x7f66f01a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.186+0000 7f66effff700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f010cf90 0x7f66f01a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.186+0000 7f66effff700 1 -- 172.21.15.57:0/2980359155 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009d00 con 0x7f66f01aa8c0 2023-10-06T22:17:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.187+0000 7f66effff700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f01aa8c0 0x7f66f01aad40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e970 tx=0x7f66e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.188+0000 7f66ecff9700 1 -- 172.21.15.57:0/2980359155 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0004a30 con 0x7f66f01aa8c0 2023-10-06T22:17:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.188+0000 7f66ecff9700 1 -- 172.21.15.57:0/2980359155 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e00054e0 con 0x7f66f01aa8c0 2023-10-06T22:17:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.188+0000 7f66ecff9700 1 -- 172.21.15.57:0/2980359155 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0010be0 con 0x7f66f01aa8c0 2023-10-06T22:17:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.188+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f010f5e0 con 0x7f66f01aa8c0 2023-10-06T22:17:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.188+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f010faa0 con 0x7f66f01aa8c0 2023-10-06T22:17:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.195+0000 7f6709514b80 1 Processor -- start 2023-10-06T22:17:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.199+0000 7f6709514b80 1 -- start start 2023-10-06T22:17:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.199+0000 7f6709514b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682cd60 0x55b48682d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.199+0000 7f6709514b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b486837ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.199+0000 7f6709514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b486838530 0x55b48683a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.199+0000 7f6709514b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486796400 con 0x55b48682cd60 2023-10-06T22:17:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.199+0000 7f6709514b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486787bc0 con 0x55b486838530 2023-10-06T22:17:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.199+0000 7f6709514b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4867854e0 con 0x55b48682d680 2023-10-06T22:17:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.204+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b486837ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.204+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b486837ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:55332/0 (socket says 172.21.15.57:55332) 2023-10-06T22:17:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.204+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3266113938 learned_addr learned my addr 172.21.15.57:0/3266113938 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.204+0000 7f66eeffd700 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b486838530 0x55b48683a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.204+0000 7f66eeffd700 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682cd60 0x55b48682d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.205+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3266113938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b486838530 msgr2=0x55b48683a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.205+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b486838530 0x55b48683a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.205+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3266113938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682cd60 msgr2=0x55b48682d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.205+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682cd60 0x55b48682d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.205+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3266113938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4865abd80 con 0x55b48682d680 2023-10-06T22:17:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.210+0000 7f66ecff9700 1 -- 172.21.15.57:0/2980359155 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f66e0004b90 con 0x7f66f01aa8c0 2023-10-06T22:17:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.217+0000 7f66ecff9700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f66d80a1800 0x7f66d80a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.226+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b486837ff0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009d80 tx=0x7f66e4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5784a41b2c52e2d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.228+0000 7f66ecff9700 1 -- 172.21.15.57:0/2980359155 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f66e0014070 con 0x7f66f01aa8c0 2023-10-06T22:17:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.228+0000 7f66d5ffb700 1 -- 172.21.15.57:0/3266113938 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4004230 con 0x55b48682d680 2023-10-06T22:17:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.229+0000 7f66d5ffb700 1 -- 172.21.15.57:0/3266113938 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e4004390 con 0x55b48682d680 2023-10-06T22:17:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.229+0000 7f66d5ffb700 1 -- 172.21.15.57:0/3266113938 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e40198c0 con 0x55b48682d680 2023-10-06T22:17:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.231+0000 7f66eeffd700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f66d80a1800 0x7f66d80a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.240+0000 7f66eeffd700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f66d80a1800 0x7f66d80a3cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66e800b070 tx=0x7f66e800b700 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.245+0000 7f6709514b80 1 -- 172.21.15.57:0/3266113938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 msgr2=0x55b486837ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.245+0000 7f6709514b80 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b486837ff0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009d80 tx=0x7f66e4011720 comp rx=0 tx=0).stop 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.246+0000 7f6709514b80 1 -- 172.21.15.57:0/3266113938 shutdown_connections 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.246+0000 7f6709514b80 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b486838530 0x55b48683a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.246+0000 7f6709514b80 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682cd60 0x55b48682d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.246+0000 7f6709514b80 1 --2- 172.21.15.57:0/3266113938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b486837ff0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.246+0000 7f6709514b80 1 -- 172.21.15.57:0/3266113938 >> 172.21.15.57:0/3266113938 conn(0x55b4867a2200 msgr2=0x55b4868291b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.246+0000 7f6709514b80 1 -- 172.21.15.57:0/3266113938 shutdown_connections 2023-10-06T22:17:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.246+0000 7f6709514b80 1 -- 172.21.15.57:0/3266113938 wait complete. 2023-10-06T22:17:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.247+0000 7f6709514b80 1 -- start start 2023-10-06T22:17:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.247+0000 7f6709514b80 1 Processor -- start 2023-10-06T22:17:49.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:48 smithi156 ceph-mon[134358]: pgmap v95: 97 pgs: 97 active+clean; 440 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 566 op/s 2023-10-06T22:17:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.255+0000 7f6709514b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682cd60 0x55b486837ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.255+0000 7f6709514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682d680 0x55b486835170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.255+0000 7f6709514b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 0x55b4868357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.255+0000 7f6709514b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486780dd0 con 0x55b486838530 2023-10-06T22:17:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.255+0000 7f6709514b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48671d6b0 con 0x55b48682d680 2023-10-06T22:17:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.255+0000 7f6709514b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48671d350 con 0x55b48682cd60 2023-10-06T22:17:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.255+0000 7f6709514b80 10 client.0 _ll_register_callbacks cb 0x55b48657a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.257+0000 7f66eeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682d680 0x55b486835170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.257+0000 7f66eeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682d680 0x55b486835170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41082/0 (socket says 172.21.15.57:41082) 2023-10-06T22:17:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.257+0000 7f66eeffd700 1 -- 172.21.15.57:0/2012123751 learned_addr learned my addr 172.21.15.57:0/2012123751 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.257+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682cd60 0x55b486837ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.267+0000 7f66effff700 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 0x55b4868357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.268+0000 7f66ef7fe700 1 -- 172.21.15.57:0/2012123751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682d680 msgr2=0x55b486835170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.268+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682d680 0x55b486835170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.268+0000 7f66ef7fe700 1 -- 172.21.15.57:0/2012123751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 msgr2=0x55b4868357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.268+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 0x55b4868357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.268+0000 7f66ef7fe700 1 -- 172.21.15.57:0/2012123751 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e40099e0 con 0x55b48682cd60 2023-10-06T22:17:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.271+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682cd60 0x55b486837ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009d80 tx=0x7f66e4003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f93c09b944588de9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.284+0000 7f66ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:17:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.284+0000 7f66ceffd700 1 -- 172.21.15.57:0/2012123751 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4005450 con 0x55b48682cd60 2023-10-06T22:17:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.284+0000 7f66ceffd700 1 -- 172.21.15.57:0/2012123751 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f66e40055b0 con 0x55b48682cd60 2023-10-06T22:17:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.284+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b486820eb0 con 0x55b48682cd60 2023-10-06T22:17:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.304+0000 7f66ceffd700 1 -- 172.21.15.57:0/2012123751 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66e4004500 con 0x55b48682cd60 2023-10-06T22:17:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.304+0000 7f6709514b80 10 client.26374 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.304+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b486594c00 con 0x55b48682cd60 2023-10-06T22:17:49.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:48 smithi190 ceph-mon[134891]: pgmap v95: 97 pgs: 97 active+clean; 440 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 566 op/s 2023-10-06T22:17:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.312+0000 7f66ceffd700 1 -- 172.21.15.57:0/2012123751 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b486820eb0 con 0x55b48682cd60 2023-10-06T22:17:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.312+0000 7f66ceffd700 1 -- 172.21.15.57:0/2012123751 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f66e4021040 con 0x55b48682cd60 2023-10-06T22:17:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.313+0000 7f6709514b80 10 client.26374 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.313+0000 7f6709514b80 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.314+0000 7f6709514b80 20 client.26374 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.314+0000 7f6709514b80 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b486847f80 0x55b48684a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.314+0000 7f6709514b80 4 client.26374 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:17:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.314+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b486779db0 con 0x55b486847f80 2023-10-06T22:17:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.321+0000 7f66eeffd700 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b486847f80 0x55b48684a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.323+0000 7f66eeffd700 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b486847f80 0x55b48684a3a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddebc412cb7fda9e server_cookie=e80e088cc2721e47 in_seq=0 out_seq=0 2023-10-06T22:17:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.323+0000 7f66ceffd700 10 client.26374 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.334+0000 7f66ceffd700 1 -- 172.21.15.57:0/2012123751 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b486779db0 con 0x55b486847f80 2023-10-06T22:17:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.334+0000 7f66ceffd700 10 client.26374 handle_command_reply: tid=0 2023-10-06T22:17:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.334+0000 7f6709514b80 1 client.26374 shutdown 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.334+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682cd60 msgr2=0x55b486837ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.334+0000 7f6709514b80 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682cd60 0x55b486837ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009d80 tx=0x7f66e4003960 comp rx=0 tx=0).stop 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 shutdown_connections 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f6709514b80 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b486847f80 0x55b48684a3a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f6709514b80 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 0x55b4868357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f6709514b80 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682cd60 0x55b486837ef0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f6709514b80 1 --2- 172.21.15.57:0/2012123751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682d680 0x55b486835170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 >> 172.21.15.57:0/2012123751 conn(0x55b4867a2200 msgr2=0x55b486828d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f66ceffd700 0 client.26374 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.335+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 shutdown_connections 2023-10-06T22:17:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.336+0000 7f6709514b80 1 -- 172.21.15.57:0/2012123751 wait complete. 2023-10-06T22:17:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.337+0000 7f6709514b80 20 client.26374 trim_cache size 0 max 16384 2023-10-06T22:17:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.636+0000 7f6709514b80 1 Processor -- start 2023-10-06T22:17:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.637+0000 7f6709514b80 1 -- start start 2023-10-06T22:17:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.637+0000 7f6709514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b486837a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.638+0000 7f6709514b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682d680 0x55b486834950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.638+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b486837a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.639+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b486837a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41084/0 (socket says 172.21.15.57:41084) 2023-10-06T22:17:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.639+0000 7f66ef7fe700 1 -- 172.21.15.57:0/4040268350 learned_addr learned my addr 172.21.15.57:0/4040268350 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.639+0000 7f6709514b80 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b486838530 0x55b486834e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.639+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4866018c0 con 0x55b48682d680 2023-10-06T22:17:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.640+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48682a590 con 0x55b48682cd60 2023-10-06T22:17:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.640+0000 7f66effff700 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b486838530 0x55b486834e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.640+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4868364c0 con 0x55b486838530 2023-10-06T22:17:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.641+0000 7f66eeffd700 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682d680 0x55b486834950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.644+0000 7f66effff700 1 -- 172.21.15.57:0/4040268350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 msgr2=0x55b486837a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.644+0000 7f66effff700 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b486837a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.644+0000 7f66effff700 1 -- 172.21.15.57:0/4040268350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682d680 msgr2=0x55b486834950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.644+0000 7f66effff700 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682d680 0x55b486834950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.644+0000 7f66effff700 1 -- 172.21.15.57:0/4040268350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4865d21f0 con 0x55b486838530 2023-10-06T22:17:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.655+0000 7f66effff700 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b486838530 0x55b486834e90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ca790 tx=0x7f66e0099c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460974a9156f824a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.737+0000 7f66d67fc700 1 -- 172.21.15.57:0/4040268350 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e009a720 con 0x55b486838530 2023-10-06T22:17:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.737+0000 7f66d67fc700 1 -- 172.21.15.57:0/4040268350 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e009a020 con 0x55b486838530 2023-10-06T22:17:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.737+0000 7f66d67fc700 1 -- 172.21.15.57:0/4040268350 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00caa10 con 0x55b486838530 2023-10-06T22:17:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.749+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b486838530 msgr2=0x55b486834e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.749+0000 7f6709514b80 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b486838530 0x55b486834e90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ca790 tx=0x7f66e0099c30 comp rx=0 tx=0).stop 2023-10-06T22:17:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.750+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 shutdown_connections 2023-10-06T22:17:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.751+0000 7f6709514b80 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b486838530 0x55b486834e90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.751+0000 7f6709514b80 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b486837a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.751+0000 7f6709514b80 1 --2- 172.21.15.57:0/4040268350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b48682d680 0x55b486834950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.751+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 >> 172.21.15.57:0/4040268350 conn(0x55b4867a2200 msgr2=0x55b486830490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.751+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 shutdown_connections 2023-10-06T22:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.751+0000 7f6709514b80 1 -- 172.21.15.57:0/4040268350 wait complete. 2023-10-06T22:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.752+0000 7f6709514b80 1 -- start start 2023-10-06T22:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.752+0000 7f6709514b80 1 Processor -- start 2023-10-06T22:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.753+0000 7f6709514b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b48683f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f6709514b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b48683d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f6709514b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 0x55b48683da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f6709514b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48683e160 con 0x55b486838530 2023-10-06T22:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f6709514b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48683e2d0 con 0x55b48682cd60 2023-10-06T22:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f6709514b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b48689bde0 con 0x55b48682d680 2023-10-06T22:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f6709514b80 10 client.0 _ll_register_callbacks cb 0x7f66e8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b48683f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b48683f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41094/0 (socket says 172.21.15.57:41094) 2023-10-06T22:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.754+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3817518907 learned_addr learned my addr 172.21.15.57:0/3817518907 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.755+0000 7f66eeffd700 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b48683d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.756+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 msgr2=0x55b48683d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.756+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b48683d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.756+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3817518907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 msgr2=0x55b48683da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.756+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 0x55b48683da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.756+0000 7f66ef7fe700 1 -- 172.21.15.57:0/3817518907 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f010faa0 con 0x55b48682cd60 2023-10-06T22:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.763+0000 7f66ef7fe700 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b48683f0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f66e4003f20 tx=0x7f66e4019420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41b7884dea04f81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:17:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.769+0000 7f66ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:17:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.769+0000 7f66ceffd700 1 -- 172.21.15.57:0/3817518907 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4009cb0 con 0x55b48682cd60 2023-10-06T22:17:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.769+0000 7f66ceffd700 1 -- 172.21.15.57:0/3817518907 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f66e4019df0 con 0x55b48682cd60 2023-10-06T22:17:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.786+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66e4004390 con 0x55b48682cd60 2023-10-06T22:17:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.787+0000 7f66ceffd700 1 -- 172.21.15.57:0/3817518907 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66e4004500 con 0x55b48682cd60 2023-10-06T22:17:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.787+0000 7f6709514b80 10 client.26411 fetch_fsmap learned FSMap version 33 2023-10-06T22:17:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.787+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b4865aa250 con 0x55b48682cd60 2023-10-06T22:17:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.797+0000 7f66ceffd700 1 -- 172.21.15.57:0/3817518907 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e4043030 con 0x55b48682cd60 2023-10-06T22:17:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.797+0000 7f66ceffd700 1 -- 172.21.15.57:0/3817518907 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f66e4011790 con 0x55b48682cd60 2023-10-06T22:17:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.804+0000 7f6709514b80 10 client.26411 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:17:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.804+0000 7f6709514b80 10 client.26411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:17:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.804+0000 7f6709514b80 20 client.26411 populate_metadata read hostname 'smithi057' 2023-10-06T22:17:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.804+0000 7f6709514b80 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b48689cf80 0x55b48689f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:17:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.804+0000 7f6709514b80 4 client.26411 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:17:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.804+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b486779ad0 con 0x55b48689cf80 2023-10-06T22:17:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.805+0000 7f66eeffd700 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b48689cf80 0x55b48689f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:17:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.819+0000 7f66eeffd700 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b48689cf80 0x55b48689f370 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5517feafe2889b server_cookie=315cdf9bcb462d07 in_seq=0 out_seq=0 2023-10-06T22:17:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:49.819+0000 7f66ceffd700 10 client.26411 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.502+0000 7f66ceffd700 1 -- 172.21.15.57:0/3817518907 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b486779ad0 con 0x55b48689cf80 2023-10-06T22:17:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.502+0000 7f66ceffd700 10 client.26411 handle_command_reply: tid=0 2023-10-06T22:17:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.502+0000 7f6709514b80 1 client.26411 shutdown 2023-10-06T22:17:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.517+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 msgr2=0x55b48683f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.517+0000 7f6709514b80 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b48683f0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f66e4003f20 tx=0x7f66e4019420 comp rx=0 tx=0).stop 2023-10-06T22:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.530+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 shutdown_connections 2023-10-06T22:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.530+0000 7f6709514b80 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b48689cf80 0x55b48689f370 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.530+0000 7f6709514b80 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b486838530 0x55b48683da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.530+0000 7f6709514b80 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b48682cd60 0x55b48683f0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.530+0000 7f6709514b80 1 --2- 172.21.15.57:0/3817518907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b48682d680 0x55b48683d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.530+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 >> 172.21.15.57:0/3817518907 conn(0x55b4867a2200 msgr2=0x55b4868283b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.530+0000 7f66ceffd700 0 client.26411 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.532+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 shutdown_connections 2023-10-06T22:17:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.534+0000 7f6709514b80 1 -- 172.21.15.57:0/3817518907 wait complete. 2023-10-06T22:17:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.535+0000 7f6709514b80 20 client.26411 trim_cache size 0 max 16384 2023-10-06T22:17:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:17:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:17:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:17:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:17:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.577+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f66d80a1800 msgr2=0x7f66d80a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.577+0000 7f66f499d700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f66d80a1800 0x7f66d80a3cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66e800b070 tx=0x7f66e800b700 comp rx=0 tx=0).stop 2023-10-06T22:17:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.577+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f01aa8c0 msgr2=0x7f66f01aad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:17:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.577+0000 7f66f499d700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f01aa8c0 0x7f66f01aad40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e970 tx=0x7f66e000ec80 comp rx=0 tx=0).stop 2023-10-06T22:17:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 shutdown_connections 2023-10-06T22:17:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f66d80a1800 0x7f66d80a3cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66f01aa8c0 0x7f66f01aad40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f010cf90 0x7f66f01a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 --2- 172.21.15.57:0/2980359155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66f011a340 0x7f66f01a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:17:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 >> 172.21.15.57:0/2980359155 conn(0x7f66f00721d0 msgr2=0x7f66f006b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:17:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 shutdown_connections 2023-10-06T22:17:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:17:50.578+0000 7f66f499d700 1 -- 172.21.15.57:0/2980359155 wait complete. 2023-10-06T22:17:51.186 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:50 smithi057 ceph-mon[98492]: pgmap v96: 97 pgs: 97 active+clean; 440 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 431 op/s 2023-10-06T22:17:51.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:50 smithi156 ceph-mon[134358]: pgmap v96: 97 pgs: 97 active+clean; 440 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 431 op/s 2023-10-06T22:17:51.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:50 smithi190 ceph-mon[134891]: pgmap v96: 97 pgs: 97 active+clean; 440 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 431 op/s 2023-10-06T22:17:52.046 INFO:tasks.workunit.client.0.smithi057.stdout: 118 829717 571 579400 498 212480 298 2023-10-06T22:17:53.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:52 smithi156 ceph-mon[134358]: pgmap v97: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.3 MiB/s wr, 660 op/s 2023-10-06T22:17:53.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:52 smithi190 ceph-mon[134891]: pgmap v97: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.3 MiB/s wr, 660 op/s 2023-10-06T22:17:53.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:52 smithi057 ceph-mon[98492]: pgmap v97: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.3 MiB/s wr, 660 op/s 2023-10-06T22:17:55.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:54 smithi156 ceph-mon[134358]: pgmap v98: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 381 op/s 2023-10-06T22:17:55.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:54 smithi190 ceph-mon[134891]: pgmap v98: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 381 op/s 2023-10-06T22:17:55.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:54 smithi057 ceph-mon[98492]: pgmap v98: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 381 op/s 2023-10-06T22:17:57.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:57 smithi190 ceph-mon[134891]: pgmap v99: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 275 op/s 2023-10-06T22:17:57.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:57 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:17:57.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:57 smithi057 ceph-mon[98492]: pgmap v99: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 275 op/s 2023-10-06T22:17:57.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:57 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:17:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:57 smithi156 ceph-mon[134358]: pgmap v99: 97 pgs: 97 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 275 op/s 2023-10-06T22:17:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:57 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:17:57.906 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:17:59.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:17:59 smithi190 ceph-mon[134891]: pgmap v100: 97 pgs: 97 active+clean; 449 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 333 op/s 2023-10-06T22:17:59.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:17:59 smithi057 ceph-mon[98492]: pgmap v100: 97 pgs: 97 active+clean; 449 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 333 op/s 2023-10-06T22:17:59.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:17:59 smithi156 ceph-mon[134358]: pgmap v100: 97 pgs: 97 active+clean; 449 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 333 op/s 2023-10-06T22:18:01.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.385+0000 7f79f93ff700 1 -- 172.21.15.57:0/1107717407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79f410f640 msgr2=0x7f79f4107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.385+0000 7f79f93ff700 1 --2- 172.21.15.57:0/1107717407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79f410f640 0x7f79f4107eb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009990 tx=0x7f79e4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.385+0000 7f79f93ff700 1 -- 172.21.15.57:0/1107717407 shutdown_connections 2023-10-06T22:18:01.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.385+0000 7f79f93ff700 1 --2- 172.21.15.57:0/1107717407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79f411a690 0x7f79f411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.385+0000 7f79f93ff700 1 --2- 172.21.15.57:0/1107717407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79f410f640 0x7f79f4107eb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.385+0000 7f79f93ff700 1 --2- 172.21.15.57:0/1107717407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 0x7f79f4108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.385+0000 7f79f93ff700 1 -- 172.21.15.57:0/1107717407 >> 172.21.15.57:0/1107717407 conn(0x7f79f406d1a0 msgr2=0x7f79f406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:01.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.386+0000 7f79f93ff700 1 -- 172.21.15.57:0/1107717407 shutdown_connections 2023-10-06T22:18:01.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.386+0000 7f79f93ff700 1 -- 172.21.15.57:0/1107717407 wait complete. 2023-10-06T22:18:01.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.387+0000 7f79f93ff700 1 Processor -- start 2023-10-06T22:18:01.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.388+0000 7f79f93ff700 1 -- start start 2023-10-06T22:18:01.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.388+0000 7f79f93ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 0x7f79f41119f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.388+0000 7f79f93ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79f410f640 0x7f79f4111f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.388+0000 7f79f93ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79f411a690 0x7f79f41ba360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.388+0000 7f79f93ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41ba930 con 0x7f79f411a690 2023-10-06T22:18:01.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.388+0000 7f79f93ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41baaa0 con 0x7f79f410f640 2023-10-06T22:18:01.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.388+0000 7f79f93ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41bac60 con 0x7f79f41083f0 2023-10-06T22:18:01.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.389+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 0x7f79f41119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.389+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 0x7f79f41119f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:54850/0 (socket says 172.21.15.57:54850) 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.389+0000 7f79f2ffd700 1 -- 172.21.15.57:0/2074160702 learned_addr learned my addr 172.21.15.57:0/2074160702 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.390+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79f410f640 0x7f79f4111f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.390+0000 7f79f2ffd700 1 -- 172.21.15.57:0/2074160702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79f410f640 msgr2=0x7f79f4111f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.390+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79f410f640 0x7f79f4111f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.390+0000 7f79f2ffd700 1 -- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79f411a690 msgr2=0x7f79f41ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.390+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79f411a690 0x7f79f41ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.390+0000 7f79f2ffd700 1 -- 172.21.15.57:0/2074160702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40095f0 con 0x7f79f41083f0 2023-10-06T22:18:01.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.390+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79f410f640 0x7f79f4111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:18:01.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.406+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 0x7f79f41119f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005910 tx=0x7f79e4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:01.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:01 smithi057 ceph-mon[98492]: pgmap v101: 97 pgs: 97 active+clean; 449 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 288 op/s 2023-10-06T22:18:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.422+0000 7f79dbfff700 1 -- 172.21.15.57:0/2074160702 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e401d070 con 0x7f79f41083f0 2023-10-06T22:18:01.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.422+0000 7f79dbfff700 1 -- 172.21.15.57:0/2074160702 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e40173f0 con 0x7f79f41083f0 2023-10-06T22:18:01.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.422+0000 7f79dbfff700 1 -- 172.21.15.57:0/2074160702 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400e640 con 0x7f79f41083f0 2023-10-06T22:18:01.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.428+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f41baee0 con 0x7f79f41083f0 2023-10-06T22:18:01.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.428+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f41bb250 con 0x7f79f41083f0 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.435+0000 7f7a0ccb7b80 1 Processor -- start 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.435+0000 7f7a0ccb7b80 1 -- start start 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.436+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.436+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcea9e0 0x55d81bcf5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.436+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcf5890 0x55d81bcf7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.436+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bc53460 con 0x55d81bcea0c0 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.436+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bc44c20 con 0x55d81bcea9e0 2023-10-06T22:18:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.436+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bc42540 con 0x55d81bcf5890 2023-10-06T22:18:01.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.440+0000 7f79f37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.441+0000 7f79f37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcea4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42132/0 (socket says 172.21.15.57:42132) 2023-10-06T22:18:01.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.441+0000 7f79f37fe700 1 -- 172.21.15.57:0/2805091619 learned_addr learned my addr 172.21.15.57:0/2805091619 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:01.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.445+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcea9e0 0x55d81bcf5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.445+0000 7f79dbfff700 1 -- 172.21.15.57:0/2074160702 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f79e4017560 con 0x7f79f41083f0 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.447+0000 7f79dbfff700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f79dc0a1800 0x7f79dc0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.448+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcf5890 0x55d81bcf7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.448+0000 7f79dbfff700 1 -- 172.21.15.57:0/2074160702 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f79e40ce560 con 0x7f79f41083f0 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.450+0000 7f79f27fc700 1 -- 172.21.15.57:0/2805091619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcea9e0 msgr2=0x55d81bcf5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.450+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcea9e0 0x55d81bcf5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.450+0000 7f79f27fc700 1 -- 172.21.15.57:0/2805091619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 msgr2=0x55d81bcea4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.450+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcea4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.450+0000 7f79f27fc700 1 -- 172.21.15.57:0/2805091619 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81ba68de0 con 0x55d81bcf5890 2023-10-06T22:18:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.461+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcf5890 0x55d81bcf7c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79f410d170 tx=0x7f79e8009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef26b8077160d01d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.461+0000 7f79f37fe700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f79dc0a1800 0x7f79dc0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.465+0000 7f79f37fe700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f79dc0a1800 0x7f79dc0a3cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009fd0 tx=0x7f79ec004ad0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:01.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.496+0000 7f79d97fa700 1 -- 172.21.15.57:0/2805091619 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8004230 con 0x55d81bcf5890 2023-10-06T22:18:01.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.496+0000 7f79d97fa700 1 -- 172.21.15.57:0/2805091619 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e8010d80 con 0x55d81bcf5890 2023-10-06T22:18:01.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.496+0000 7f79d97fa700 1 -- 172.21.15.57:0/2805091619 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8003d30 con 0x55d81bcf5890 2023-10-06T22:18:01.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.497+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2805091619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcf5890 msgr2=0x55d81bcf7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.497+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcf5890 0x55d81bcf7c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79f410d170 tx=0x7f79e8009f80 comp rx=0 tx=0).stop 2023-10-06T22:18:01.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.497+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2805091619 shutdown_connections 2023-10-06T22:18:01.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.497+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcf5890 0x55d81bcf7c80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.497+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.497+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2805091619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcea9e0 0x55d81bcf5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.497+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2805091619 >> 172.21.15.57:0/2805091619 conn(0x55d81bc5f260 msgr2=0x55d81bce6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:01.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:01 smithi156 ceph-mon[134358]: pgmap v101: 97 pgs: 97 active+clean; 449 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 288 op/s 2023-10-06T22:18:01.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.501+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2805091619 shutdown_connections 2023-10-06T22:18:01.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.501+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2805091619 wait complete. 2023-10-06T22:18:01.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.503+0000 7f7a0ccb7b80 1 -- start start 2023-10-06T22:18:01.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.503+0000 7f7a0ccb7b80 1 Processor -- start 2023-10-06T22:18:01.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.505+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea0c0 0x55d81bcf2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.505+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.505+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcf30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.505+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bc3de30 con 0x55d81bcea9e0 2023-10-06T22:18:01.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.505+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bbda710 con 0x55d81bcf5890 2023-10-06T22:18:01.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.505+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bbda3b0 con 0x55d81bcea0c0 2023-10-06T22:18:01.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.505+0000 7f7a0ccb7b80 10 client.0 _ll_register_callbacks cb 0x55d81ba37800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:01.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.507+0000 7f79f27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.507+0000 7f79f27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42134/0 (socket says 172.21.15.57:42134) 2023-10-06T22:18:01.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.507+0000 7f79f27fc700 1 -- 172.21.15.57:0/10800059 learned_addr learned my addr 172.21.15.57:0/10800059 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:01.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.508+0000 7f79f27fc700 1 -- 172.21.15.57:0/10800059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea0c0 msgr2=0x55d81bcf2670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:18:01.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.508+0000 7f79f27fc700 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea0c0 0x55d81bcf2670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.508+0000 7f79f27fc700 1 -- 172.21.15.57:0/10800059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 msgr2=0x55d81bcf30f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:18:01.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.508+0000 7f79f27fc700 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcf30f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.508+0000 7f79f27fc700 1 -- 172.21.15.57:0/10800059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e8009900 con 0x55d81bcea9e0 2023-10-06T22:18:01.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.508+0000 7f79f27fc700 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2bb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f79e80126b0 tx=0x7f79e8012790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6200a4daac113e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:01.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.511+0000 7f79d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:18:01.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.511+0000 7f79d6ff5700 1 -- 172.21.15.57:0/10800059 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8012df0 con 0x55d81bcea9e0 2023-10-06T22:18:01.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.511+0000 7f79d6ff5700 1 -- 172.21.15.57:0/10800059 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f79e8016070 con 0x55d81bcea9e0 2023-10-06T22:18:01.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.511+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d81bcddf10 con 0x55d81bcea9e0 2023-10-06T22:18:01.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.511+0000 7f79d6ff5700 1 -- 172.21.15.57:0/10800059 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79e8003680 con 0x55d81bcea9e0 2023-10-06T22:18:01.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 10 client.16389 fetch_fsmap learned FSMap version 33 2023-10-06T22:18:01.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d81ba51c60 con 0x55d81bcea9e0 2023-10-06T22:18:01.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f79d6ff5700 1 -- 172.21.15.57:0/10800059 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d81bcddf10 con 0x55d81bcea9e0 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f79d6ff5700 1 -- 172.21.15.57:0/10800059 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f79e802d070 con 0x55d81bcea9e0 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 20 client.16389 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd051b0 0x55d81bd075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 4 client.16389 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.512+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d81bc36e10 con 0x55d81bd051b0 2023-10-06T22:18:01.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.513+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd051b0 0x55d81bd075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.513+0000 7f79d6ff5700 10 client.16389 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:01.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.513+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd051b0 0x55d81bd075d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc475138eb317a63 server_cookie=1c7e853a7abfd14a in_seq=0 out_seq=0 2023-10-06T22:18:01.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.514+0000 7f79d6ff5700 1 -- 172.21.15.57:0/10800059 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d81bc36e10 con 0x55d81bd051b0 2023-10-06T22:18:01.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.514+0000 7f79d6ff5700 10 client.16389 handle_command_reply: tid=0 2023-10-06T22:18:01.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.514+0000 7f7a0ccb7b80 1 client.16389 shutdown 2023-10-06T22:18:01.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.514+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea9e0 msgr2=0x55d81bcf2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.514+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2bb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f79e80126b0 tx=0x7f79e8012790 comp rx=0 tx=0).stop 2023-10-06T22:18:01.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 shutdown_connections 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd051b0 0x55d81bd075d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcf30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea0c0 0x55d81bcf2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/10800059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2bb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 >> 172.21.15.57:0/10800059 conn(0x55d81bc5f260 msgr2=0x55d81bce6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f79d6ff5700 0 client.16389 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 shutdown_connections 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/10800059 wait complete. 2023-10-06T22:18:01.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.515+0000 7f7a0ccb7b80 20 client.16389 trim_cache size 0 max 16384 2023-10-06T22:18:01.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.535+0000 7f7a0ccb7b80 1 Processor -- start 2023-10-06T22:18:01.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.535+0000 7f7a0ccb7b80 1 -- start start 2023-10-06T22:18:01.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcf1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81babe920 con 0x55d81bcea0c0 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bced520 con 0x55d81bcf5890 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bcee550 con 0x55d81bcea9e0 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:54880/0 (socket says 172.21.15.57:54880) 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f27fc700 1 -- 172.21.15.57:0/2753919633 learned_addr learned my addr 172.21.15.57:0/2753919633 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f37fe700 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcf1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f27fc700 1 -- 172.21.15.57:0/2753919633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 msgr2=0x55d81bcf1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcf1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.536+0000 7f79f27fc700 1 -- 172.21.15.57:0/2753919633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 msgr2=0x55d81bcecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.537+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.537+0000 7f79f27fc700 1 -- 172.21.15.57:0/2753919633 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81bce2140 con 0x55d81bcea9e0 2023-10-06T22:18:01.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.537+0000 7f79f27fc700 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2f20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005720 tx=0x7f79e8005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f0d3a5e9ec8e1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.537+0000 7f79d9ffb700 1 -- 172.21.15.57:0/2753919633 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8003680 con 0x55d81bcea9e0 2023-10-06T22:18:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.537+0000 7f79d9ffb700 1 -- 172.21.15.57:0/2753919633 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e801f070 con 0x55d81bcea9e0 2023-10-06T22:18:01.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.537+0000 7f79d9ffb700 1 -- 172.21.15.57:0/2753919633 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80257f0 con 0x55d81bcea9e0 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.538+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2753919633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 msgr2=0x55d81bcf2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.538+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2f20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005720 tx=0x7f79e8005750 comp rx=0 tx=0).stop 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.538+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2753919633 shutdown_connections 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.538+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcf1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.538+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.538+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/2753919633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcf2f20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.538+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2753919633 >> 172.21.15.57:0/2753919633 conn(0x55d81bc5f260 msgr2=0x55d81bcf0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:01.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.539+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2753919633 shutdown_connections 2023-10-06T22:18:01.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.539+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/2753919633 wait complete. 2023-10-06T22:18:01.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.539+0000 7f7a0ccb7b80 1 -- start start 2023-10-06T22:18:01.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.539+0000 7f7a0ccb7b80 1 Processor -- start 2023-10-06T22:18:01.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.540+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcfbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.540+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcfc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.540+0000 7f7a0ccb7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcfa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.540+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bcfa9d0 con 0x55d81bcea0c0 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.540+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bcfab40 con 0x55d81bcf5890 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.540+0000 7f7a0ccb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81bcfad00 con 0x55d81bcea9e0 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.540+0000 7f7a0ccb7b80 10 client.0 _ll_register_callbacks cb 0x7f79f406a0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.541+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcfbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.541+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcfbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42148/0 (socket says 172.21.15.57:42148) 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.541+0000 7f79f2ffd700 1 -- 172.21.15.57:0/77608856 learned_addr learned my addr 172.21.15.57:0/77608856 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:01.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79f2ffd700 1 -- 172.21.15.57:0/77608856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 msgr2=0x55d81bcfc2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:18:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcfc2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79f2ffd700 1 -- 172.21.15.57:0/77608856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 msgr2=0x55d81bcfa3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:18:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcfa3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79f2ffd700 1 -- 172.21.15.57:0/77608856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d81ba51c60 con 0x55d81bcea0c0 2023-10-06T22:18:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79f2ffd700 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcfbd60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f79e4000c00 tx=0x7f79e409b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ace24ea3ccf26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:18:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79d6ff5700 1 -- 172.21.15.57:0/77608856 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e409f730 con 0x55d81bcea0c0 2023-10-06T22:18:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.542+0000 7f79d6ff5700 1 -- 172.21.15.57:0/77608856 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f79e40a38e0 con 0x55d81bcea0c0 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.543+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79e8010d80 con 0x55d81bcea0c0 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.543+0000 7f79d6ff5700 1 -- 172.21.15.57:0/77608856 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79e4017860 con 0x55d81bcea0c0 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.543+0000 7f7a0ccb7b80 10 client.16395 fetch_fsmap learned FSMap version 33 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.543+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d81ba8f250 con 0x55d81bcea0c0 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.543+0000 7f79d6ff5700 1 -- 172.21.15.57:0/77608856 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e8010d80 con 0x55d81bcea0c0 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.543+0000 7f79d6ff5700 1 -- 172.21.15.57:0/77608856 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f79e409b840 con 0x55d81bcea0c0 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f7a0ccb7b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f7a0ccb7b80 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f7a0ccb7b80 20 client.16395 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd59e90 0x55d81bd5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f7a0ccb7b80 4 client.16395 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d81bc36b30 con 0x55d81bd59e90 2023-10-06T22:18:01.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f79f27fc700 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd59e90 0x55d81bd5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:01.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f79f27fc700 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd59e90 0x55d81bd5c2d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b5c0c09c2dfc2e server_cookie=5b775a4983b51a2a in_seq=0 out_seq=0 2023-10-06T22:18:01.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.544+0000 7f79d6ff5700 10 client.16395 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:01.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.545+0000 7f79d6ff5700 1 -- 172.21.15.57:0/77608856 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d81bc36b30 con 0x55d81bd59e90 2023-10-06T22:18:01.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.545+0000 7f79d6ff5700 10 client.16395 handle_command_reply: tid=0 2023-10-06T22:18:01.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.545+0000 7f7a0ccb7b80 1 client.16395 shutdown 2023-10-06T22:18:01.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.545+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 msgr2=0x55d81bcfbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.545+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcfbd60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f79e4000c00 tx=0x7f79e409b1d0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 shutdown_connections 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d81bd59e90 0x55d81bd5c2d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d81bcf5890 0x55d81bcfa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d81bcea0c0 0x55d81bcfbd60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 --2- 172.21.15.57:0/77608856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d81bcea9e0 0x55d81bcfc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 >> 172.21.15.57:0/77608856 conn(0x55d81bc5f260 msgr2=0x55d81bcf1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f79d6ff5700 0 client.16395 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 shutdown_connections 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 1 -- 172.21.15.57:0/77608856 wait complete. 2023-10-06T22:18:01.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.547+0000 7f7a0ccb7b80 20 client.16395 trim_cache size 0 max 16384 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:18:01.551 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:18:01.552 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:18:01.552 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:18:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f79dc0a1800 msgr2=0x7f79dc0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f79dc0a1800 0x7f79dc0a3cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009fd0 tx=0x7f79ec004ad0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 msgr2=0x7f79f41119f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 0x7f79f41119f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005910 tx=0x7f79e4009580 comp rx=0 tx=0).stop 2023-10-06T22:18:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 shutdown_connections 2023-10-06T22:18:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f79dc0a1800 0x7f79dc0a3cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f79f411a690 0x7f79f41ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f41083f0 0x7f79f41119f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 --2- 172.21.15.57:0/2074160702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79f410f640 0x7f79f4111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.553+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 >> 172.21.15.57:0/2074160702 conn(0x7f79f406d1a0 msgr2=0x7f79f406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:01.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.555+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 shutdown_connections 2023-10-06T22:18:01.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:01.555+0000 7f79f93ff700 1 -- 172.21.15.57:0/2074160702 wait complete. 2023-10-06T22:18:01.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:01 smithi190 ceph-mon[134891]: pgmap v101: 97 pgs: 97 active+clean; 449 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 288 op/s 2023-10-06T22:18:01.588 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:18:02.046 INFO:tasks.workunit.client.0.smithi057.stdout: 128 845353 569 589557 507 209158 215 2023-10-06T22:18:03.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:03 smithi057 ceph-mon[98492]: pgmap v102: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 337 op/s 2023-10-06T22:18:03.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:03 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:18:03.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:03 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:18:03.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:03 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:18:03.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:03 smithi156 ceph-mon[134358]: pgmap v102: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 337 op/s 2023-10-06T22:18:03.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:03 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:18:03.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:03 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:18:03.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:03 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:18:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:03 smithi190 ceph-mon[134891]: pgmap v102: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 337 op/s 2023-10-06T22:18:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:03 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:18:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:03 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:18:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:03 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:18:04.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:04 smithi156 ceph-mon[134358]: pgmap v103: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-06T22:18:04.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:04 smithi190 ceph-mon[134891]: pgmap v103: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-06T22:18:04.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:04 smithi057 ceph-mon[98492]: pgmap v103: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-06T22:18:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.876+0000 7f8901fff700 1 -- 172.21.15.57:0/4280471738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88fc11a690 msgr2=0x7f88fc11ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.876+0000 7f8901fff700 1 --2- 172.21.15.57:0/4280471738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88fc11a690 0x7f88fc11ca80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b0a0 tx=0x7f88f400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.891+0000 7f8901fff700 1 -- 172.21.15.57:0/4280471738 shutdown_connections 2023-10-06T22:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.891+0000 7f8901fff700 1 --2- 172.21.15.57:0/4280471738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88fc11a690 0x7f88fc11ca80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.891+0000 7f8901fff700 1 --2- 172.21.15.57:0/4280471738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc10f620 0x7f88fc107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.891+0000 7f8901fff700 1 --2- 172.21.15.57:0/4280471738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88fc10fa90 0x7f88fc1083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.891+0000 7f8901fff700 1 -- 172.21.15.57:0/4280471738 >> 172.21.15.57:0/4280471738 conn(0x7f88fc06cfa0 msgr2=0x7f88fc06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.891+0000 7f8901fff700 1 -- 172.21.15.57:0/4280471738 shutdown_connections 2023-10-06T22:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.891+0000 7f8901fff700 1 -- 172.21.15.57:0/4280471738 wait complete. 2023-10-06T22:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.893+0000 7f8901fff700 1 Processor -- start 2023-10-06T22:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.893+0000 7f8901fff700 1 -- start start 2023-10-06T22:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.894+0000 7f8901fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88fc10f620 0x7f88fc119b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.894+0000 7f8901fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88fc10fa90 0x7f88fc11a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.894+0000 7f8901fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc11a690 0x7f88fc113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.894+0000 7f8901fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc1142a0 con 0x7f88fc10f620 2023-10-06T22:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.894+0000 7f8901fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc114410 con 0x7f88fc10fa90 2023-10-06T22:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.894+0000 7f8901fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc1145d0 con 0x7f88fc11a690 2023-10-06T22:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.895+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc11a690 0x7f88fc113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.895+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc11a690 0x7f88fc113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43428/0 (socket says 172.21.15.57:43428) 2023-10-06T22:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.895+0000 7f88fbfff700 1 -- 172.21.15.57:0/1012340737 learned_addr learned my addr 172.21.15.57:0/1012340737 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.914+0000 7f88fbfff700 1 -- 172.21.15.57:0/1012340737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88fc10fa90 msgr2=0x7f88fc11a0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:18:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.914+0000 7f88fbfff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88fc10fa90 0x7f88fc11a0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.914+0000 7f88fbfff700 1 -- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88fc10f620 msgr2=0x7f88fc119b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.914+0000 7f88fbfff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88fc10f620 0x7f88fc119b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.914+0000 7f88fbfff700 1 -- 172.21.15.57:0/1012340737 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4009d00 con 0x7f88fc11a690 2023-10-06T22:18:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:06.926+0000 7f88fbfff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc11a690 0x7f88fc113ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009fd0 tx=0x7f88f40123f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.112+0000 7f88f8ff9700 1 -- 172.21.15.57:0/1012340737 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400e040 con 0x7f88fc11a690 2023-10-06T22:18:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.112+0000 7f88f8ff9700 1 -- 172.21.15.57:0/1012340737 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f4012c60 con 0x7f88fc11a690 2023-10-06T22:18:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.112+0000 7f88f8ff9700 1 -- 172.21.15.57:0/1012340737 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4005690 con 0x7f88fc11a690 2023-10-06T22:18:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.112+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc1bb310 con 0x7f88fc11a690 2023-10-06T22:18:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.112+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc1bb680 con 0x7f88fc11a690 2023-10-06T22:18:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.133+0000 7f89158e2b80 1 Processor -- start 2023-10-06T22:18:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.133+0000 7f89158e2b80 1 -- start start 2023-10-06T22:18:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.134+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x561766669fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.134+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 0x561766674e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.134+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x5617666777c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.134+0000 7f89158e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617665d3400 con 0x56176666a520 2023-10-06T22:18:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.134+0000 7f89158e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617665c4bc0 con 0x5617666753d0 2023-10-06T22:18:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.134+0000 7f89158e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617665c24e0 con 0x561766669c00 2023-10-06T22:18:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.134+0000 7f88faffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x561766669fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x561766669fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43432/0 (socket says 172.21.15.57:43432) 2023-10-06T22:18:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 -- 172.21.15.57:0/488111170 learned_addr learned my addr 172.21.15.57:0/488111170 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x5617666777c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 -- 172.21.15.57:0/488111170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 msgr2=0x5617666777c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x5617666777c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 -- 172.21.15.57:0/488111170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 msgr2=0x561766674e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 0x561766674e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.135+0000 7f88faffd700 1 -- 172.21.15.57:0/488111170 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617663e8d80 con 0x561766669c00 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.136+0000 7f88f8ff9700 1 -- 172.21.15.57:0/1012340737 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f88f40127a0 con 0x7f88fc11a690 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.138+0000 7f88f8ff9700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f88e40a1800 0x7f88e40a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.138+0000 7f88f8ff9700 1 -- 172.21.15.57:0/1012340737 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f88f40ce590 con 0x7f88fc11a690 2023-10-06T22:18:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.144+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f88e40a1800 0x7f88e40a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.144+0000 7f88faffd700 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x561766669fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f88f00143d0 tx=0x7f88f00090d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aef9a436d3345d92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.165+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f88e40a1800 0x7f88e40a3cc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009770 tx=0x7f88ec006d40 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:07.188 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:06 smithi057 ceph-mon[98492]: pgmap v104: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-06T22:18:07.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:06 smithi156 ceph-mon[134358]: pgmap v104: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-06T22:18:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.252+0000 7f88e9ffb700 1 -- 172.21.15.57:0/488111170 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0014bf0 con 0x561766669c00 2023-10-06T22:18:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.252+0000 7f88e9ffb700 1 -- 172.21.15.57:0/488111170 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f0014d50 con 0x561766669c00 2023-10-06T22:18:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.252+0000 7f88e9ffb700 1 -- 172.21.15.57:0/488111170 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0016600 con 0x561766669c00 2023-10-06T22:18:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.262+0000 7f89158e2b80 1 -- 172.21.15.57:0/488111170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 msgr2=0x561766669fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.262+0000 7f89158e2b80 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x561766669fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f88f00143d0 tx=0x7f88f00090d0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.278+0000 7f89158e2b80 1 -- 172.21.15.57:0/488111170 shutdown_connections 2023-10-06T22:18:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.278+0000 7f89158e2b80 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x5617666777c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.278+0000 7f89158e2b80 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x561766669fe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.278+0000 7f89158e2b80 1 --2- 172.21.15.57:0/488111170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 0x561766674e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.278+0000 7f89158e2b80 1 -- 172.21.15.57:0/488111170 >> 172.21.15.57:0/488111170 conn(0x5617665df200 msgr2=0x561766666080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.282+0000 7f89158e2b80 1 -- 172.21.15.57:0/488111170 shutdown_connections 2023-10-06T22:18:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.282+0000 7f89158e2b80 1 -- 172.21.15.57:0/488111170 wait complete. 2023-10-06T22:18:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.283+0000 7f89158e2b80 1 -- start start 2023-10-06T22:18:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.283+0000 7f89158e2b80 1 Processor -- start 2023-10-06T22:18:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.284+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766674e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.284+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.284+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.284+0000 7f89158e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617665bddd0 con 0x5617666753d0 2023-10-06T22:18:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.284+0000 7f89158e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176655a6b0 con 0x561766669c00 2023-10-06T22:18:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.284+0000 7f89158e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176655a350 con 0x56176666a520 2023-10-06T22:18:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.284+0000 7f89158e2b80 10 client.0 _ll_register_callbacks cb 0x5617663b77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.292+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.292+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666727c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43842/0 (socket says 172.21.15.57:43842) 2023-10-06T22:18:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.292+0000 7f88fbfff700 1 -- 172.21.15.57:0/780898059 learned_addr learned my addr 172.21.15.57:0/780898059 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.295+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766674e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.295+0000 7f88faffd700 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.298+0000 7f88fbfff700 1 -- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 msgr2=0x5617666720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.298+0000 7f88fbfff700 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.299+0000 7f88fbfff700 1 -- 172.21.15.57:0/780898059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 msgr2=0x561766674e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.299+0000 7f88fbfff700 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766674e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.299+0000 7f88fbfff700 1 -- 172.21.15.57:0/780898059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f0009e20 con 0x5617666753d0 2023-10-06T22:18:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.306+0000 7f88fbfff700 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666727c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f88f40a33e0 tx=0x7f88f4011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae344eb01f658ec1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.306+0000 7f88deffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:18:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.311+0000 7f88deffd700 1 -- 172.21.15.57:0/780898059 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f409f760 con 0x5617666753d0 2023-10-06T22:18:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.311+0000 7f88deffd700 1 -- 172.21.15.57:0/780898059 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f88f40a3910 con 0x5617666753d0 2023-10-06T22:18:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.312+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56176665dd50 con 0x5617666753d0 2023-10-06T22:18:07.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:06 smithi190 ceph-mon[134891]: pgmap v104: 97 pgs: 97 active+clean; 454 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-06T22:18:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.320+0000 7f88deffd700 1 -- 172.21.15.57:0/780898059 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f40999b0 con 0x5617666753d0 2023-10-06T22:18:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.330+0000 7f89158e2b80 10 client.16401 fetch_fsmap learned FSMap version 33 2023-10-06T22:18:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.330+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5617663d1c00 con 0x5617666753d0 2023-10-06T22:18:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.346+0000 7f88deffd700 1 -- 172.21.15.57:0/780898059 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f40b8070 con 0x5617666753d0 2023-10-06T22:18:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.346+0000 7f88deffd700 1 -- 172.21.15.57:0/780898059 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f88f409b870 con 0x5617666753d0 2023-10-06T22:18:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.347+0000 7f89158e2b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:18:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.347+0000 7f89158e2b80 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.347+0000 7f89158e2b80 20 client.16401 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.347+0000 7f89158e2b80 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561766684ce0 0x561766687100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.347+0000 7f89158e2b80 4 client.16401 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:18:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.347+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617665b6db0 con 0x561766684ce0 2023-10-06T22:18:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.347+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561766684ce0 0x561766687100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.379+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561766684ce0 0x561766687100 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc858829534af10 server_cookie=99eb87da849c7e16 in_seq=0 out_seq=0 2023-10-06T22:18:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.379+0000 7f88deffd700 10 client.16401 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.379+0000 7f88deffd700 1 -- 172.21.15.57:0/780898059 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5617665b6db0 con 0x561766684ce0 2023-10-06T22:18:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.379+0000 7f88deffd700 10 client.16401 handle_command_reply: tid=0 2023-10-06T22:18:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.380+0000 7f89158e2b80 1 client.16401 shutdown 2023-10-06T22:18:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.573+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 msgr2=0x5617666727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.573+0000 7f89158e2b80 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666727c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f88f40a33e0 tx=0x7f88f4011e70 comp rx=0 tx=0).stop 2023-10-06T22:18:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.576+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 shutdown_connections 2023-10-06T22:18:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.576+0000 7f89158e2b80 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561766684ce0 0x561766687100 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.576+0000 7f89158e2b80 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666727c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.576+0000 7f89158e2b80 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766674e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.576+0000 7f89158e2b80 1 --2- 172.21.15.57:0/780898059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.576+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 >> 172.21.15.57:0/780898059 conn(0x5617665df200 msgr2=0x561766665c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.576+0000 7f88deffd700 0 client.16401 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.577+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 shutdown_connections 2023-10-06T22:18:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.577+0000 7f89158e2b80 1 -- 172.21.15.57:0/780898059 wait complete. 2023-10-06T22:18:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.578+0000 7f89158e2b80 20 client.16401 trim_cache size 0 max 16384 2023-10-06T22:18:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.636+0000 7f89158e2b80 1 Processor -- start 2023-10-06T22:18:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.636+0000 7f89158e2b80 1 -- start start 2023-10-06T22:18:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.637+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766669fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.637+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.637+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.637+0000 7f89158e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176643e8c0 con 0x5617666753d0 2023-10-06T22:18:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.637+0000 7f89158e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561766666dc0 con 0x561766669c00 2023-10-06T22:18:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.637+0000 7f89158e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176666bae0 con 0x56176666a520 2023-10-06T22:18:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.639+0000 7f88faffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.639+0000 7f88fb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766669fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.639+0000 7f88faffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43458/0 (socket says 172.21.15.57:43458) 2023-10-06T22:18:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.639+0000 7f88faffd700 1 -- 172.21.15.57:0/4004164652 learned_addr learned my addr 172.21.15.57:0/4004164652 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.640+0000 7f88fbfff700 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.640+0000 7f88fb7fe700 1 -- 172.21.15.57:0/4004164652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 msgr2=0x5617666749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.640+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.640+0000 7f88fb7fe700 1 -- 172.21.15.57:0/4004164652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 msgr2=0x5617666718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.640+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.640+0000 7f88fb7fe700 1 -- 172.21.15.57:0/4004164652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56176640f1f0 con 0x561766669c00 2023-10-06T22:18:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.641+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766669fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5617666879a0 tx=0x7f88ec00e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6b6012153357f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.641+0000 7f88ea7fc700 1 -- 172.21.15.57:0/4004164652 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec013c80 con 0x561766669c00 2023-10-06T22:18:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.641+0000 7f88ea7fc700 1 -- 172.21.15.57:0/4004164652 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88ec021530 con 0x561766669c00 2023-10-06T22:18:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.642+0000 7f88ea7fc700 1 -- 172.21.15.57:0/4004164652 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec020b80 con 0x561766669c00 2023-10-06T22:18:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 -- 172.21.15.57:0/4004164652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 msgr2=0x561766669fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766669fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5617666879a0 tx=0x7f88ec00e000 comp rx=0 tx=0).stop 2023-10-06T22:18:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 -- 172.21.15.57:0/4004164652 shutdown_connections 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617666753d0 0x5617666718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561766669c00 0x561766669fe0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4004164652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56176666a520 0x5617666749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 -- 172.21.15.57:0/4004164652 >> 172.21.15.57:0/4004164652 conn(0x5617665df200 msgr2=0x56176666d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 -- 172.21.15.57:0/4004164652 shutdown_connections 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.643+0000 7f89158e2b80 1 -- 172.21.15.57:0/4004164652 wait complete. 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.644+0000 7f89158e2b80 1 -- start start 2023-10-06T22:18:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.644+0000 7f89158e2b80 1 Processor -- start 2023-10-06T22:18:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.646+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x56176667bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.646+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 0x561766679fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.646+0000 7f89158e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x56176667a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.646+0000 7f89158e2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176667aa40 con 0x56176666a520 2023-10-06T22:18:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.646+0000 7f89158e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176667abb0 con 0x5617666753d0 2023-10-06T22:18:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.646+0000 7f89158e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176667ad20 con 0x561766669c00 2023-10-06T22:18:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.646+0000 7f89158e2b80 10 client.0 _ll_register_callbacks cb 0x7f88f409b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.647+0000 7f88fb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x56176667bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.647+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x56176667a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.647+0000 7f88fb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x56176667bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43474/0 (socket says 172.21.15.57:43474) 2023-10-06T22:18:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.647+0000 7f88fb7fe700 1 -- 172.21.15.57:0/4239136608 learned_addr learned my addr 172.21.15.57:0/4239136608 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.648+0000 7f88faffd700 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 0x561766679fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.648+0000 7f88fbfff700 1 -- 172.21.15.57:0/4239136608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 msgr2=0x56176667bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.648+0000 7f88fbfff700 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x56176667bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.648+0000 7f88fbfff700 1 -- 172.21.15.57:0/4239136608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 msgr2=0x561766679fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.648+0000 7f88fbfff700 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 0x561766679fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.648+0000 7f88fbfff700 1 -- 172.21.15.57:0/4239136608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec0135d0 con 0x5617666753d0 2023-10-06T22:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.657+0000 7f88fbfff700 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x56176667a500 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b830 tx=0x7f88f409b150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f146a79ae6689d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.667+0000 7f88deffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:18:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.678+0000 7f88deffd700 1 -- 172.21.15.57:0/4239136608 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4097950 con 0x5617666753d0 2023-10-06T22:18:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.678+0000 7f88deffd700 1 -- 172.21.15.57:0/4239136608 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f88f40a3910 con 0x5617666753d0 2023-10-06T22:18:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.678+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88f0014d50 con 0x5617666753d0 2023-10-06T22:18:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.681+0000 7f88deffd700 1 -- 172.21.15.57:0/4239136608 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f409f760 con 0x5617666753d0 2023-10-06T22:18:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.681+0000 7f89158e2b80 10 client.26429 fetch_fsmap learned FSMap version 33 2023-10-06T22:18:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.681+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5617663e7250 con 0x5617666753d0 2023-10-06T22:18:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.692+0000 7f88deffd700 1 -- 172.21.15.57:0/4239136608 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f0014d50 con 0x5617666753d0 2023-10-06T22:18:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.692+0000 7f88deffd700 1 -- 172.21.15.57:0/4239136608 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 33) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f88f40a3480 con 0x5617666753d0 2023-10-06T22:18:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.693+0000 7f89158e2b80 10 client.26429 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T22:18:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.693+0000 7f89158e2b80 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.693+0000 7f89158e2b80 20 client.26429 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.693+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5617666d9a10 0x5617666dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.693+0000 7f89158e2b80 4 client.26429 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:18:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.693+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617665b6ad0 con 0x5617666d9a10 2023-10-06T22:18:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.700+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5617666d9a10 0x5617666dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.729+0000 7f88fb7fe700 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5617666d9a10 0x5617666dbe00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1feab6a7d55126f4 server_cookie=beed7aaab66e4ad in_seq=0 out_seq=0 2023-10-06T22:18:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:07.730+0000 7f88deffd700 10 client.26429 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:09.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:08 smithi156 ceph-mon[134358]: pgmap v105: 97 pgs: 97 active+clean; 476 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2023-10-06T22:18:09.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:08 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:09.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:08 smithi190 ceph-mon[134891]: pgmap v105: 97 pgs: 97 active+clean; 476 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2023-10-06T22:18:09.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:08 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:08 smithi057 ceph-mon[98492]: pgmap v105: 97 pgs: 97 active+clean; 476 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2023-10-06T22:18:09.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:08 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:11.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:10 smithi156 ceph-mon[134358]: pgmap v106: 97 pgs: 97 active+clean; 476 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.4 MiB/s wr, 169 op/s 2023-10-06T22:18:11.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:10 smithi190 ceph-mon[134891]: pgmap v106: 97 pgs: 97 active+clean; 476 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.4 MiB/s wr, 169 op/s 2023-10-06T22:18:11.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:10 smithi057 ceph-mon[98492]: pgmap v106: 97 pgs: 97 active+clean; 476 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.4 MiB/s wr, 169 op/s 2023-10-06T22:18:12.047 INFO:tasks.workunit.client.0.smithi057.stdout: 143 727169 944 498254 805 162728 62 2023-10-06T22:18:13.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:12 smithi156 ceph-mon[134358]: pgmap v107: 97 pgs: 97 active+clean; 488 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 323 op/s 2023-10-06T22:18:13.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:12 smithi190 ceph-mon[134891]: pgmap v107: 97 pgs: 97 active+clean; 488 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 323 op/s 2023-10-06T22:18:13.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:12 smithi057 ceph-mon[98492]: pgmap v107: 97 pgs: 97 active+clean; 488 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 323 op/s 2023-10-06T22:18:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.852+0000 7f88deffd700 1 -- 172.21.15.57:0/4239136608 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617665b6ad0 con 0x5617666d9a10 2023-10-06T22:18:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.852+0000 7f88deffd700 10 client.26429 handle_command_reply: tid=0 2023-10-06T22:18:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.852+0000 7f89158e2b80 1 client.26429 shutdown 2023-10-06T22:18:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.852+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 msgr2=0x56176667a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.852+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x56176667a500 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b830 tx=0x7f88f409b150 comp rx=0 tx=0).stop 2023-10-06T22:18:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 shutdown_connections 2023-10-06T22:18:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5617666d9a10 0x5617666dbe00 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617666753d0 0x56176667a500 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561766669c00 0x56176667bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f89158e2b80 1 --2- 172.21.15.57:0/4239136608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56176666a520 0x561766679fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 >> 172.21.15.57:0/4239136608 conn(0x5617665df200 msgr2=0x56176666bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f88deffd700 0 client.26429 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.853+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 shutdown_connections 2023-10-06T22:18:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.854+0000 7f89158e2b80 1 -- 172.21.15.57:0/4239136608 wait complete. 2023-10-06T22:18:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.854+0000 7f89158e2b80 20 client.26429 trim_cache size 0 max 16384 2023-10-06T22:18:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:18:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:18:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:18:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.911+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f88e40a1800 msgr2=0x7f88e40a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.911+0000 7f8901fff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f88e40a1800 0x7f88e40a3cc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009770 tx=0x7f88ec006d40 comp rx=0 tx=0).stop 2023-10-06T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.911+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc11a690 msgr2=0x7f88fc113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.911+0000 7f8901fff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc11a690 0x7f88fc113ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009fd0 tx=0x7f88f40123f0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.921+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 shutdown_connections 2023-10-06T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.921+0000 7f8901fff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f88e40a1800 0x7f88e40a3cc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.921+0000 7f8901fff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc11a690 0x7f88fc113ad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.921+0000 7f8901fff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88fc10f620 0x7f88fc119b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.921+0000 7f8901fff700 1 --2- 172.21.15.57:0/1012340737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88fc10fa90 0x7f88fc11a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.921+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 >> 172.21.15.57:0/1012340737 conn(0x7f88fc06cfa0 msgr2=0x7f88fc0718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.923+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 shutdown_connections 2023-10-06T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:13.923+0000 7f8901fff700 1 -- 172.21.15.57:0/1012340737 wait complete. 2023-10-06T22:18:14.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:14 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:14.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:14 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:14.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:14 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:15.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:15 smithi190 ceph-mon[134891]: pgmap v108: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-10-06T22:18:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:15 smithi057 ceph-mon[98492]: pgmap v108: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-10-06T22:18:15.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:15 smithi156 ceph-mon[134358]: pgmap v108: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-10-06T22:18:17.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:17 smithi057 ceph-mon[98492]: pgmap v109: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-10-06T22:18:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:17 smithi156 ceph-mon[134358]: pgmap v109: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-10-06T22:18:17.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:17 smithi190 ceph-mon[134891]: pgmap v109: 97 pgs: 97 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-10-06T22:18:19.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:19 smithi057 ceph-mon[98492]: pgmap v110: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 614 op/s 2023-10-06T22:18:19.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:19 smithi156 ceph-mon[134358]: pgmap v110: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 614 op/s 2023-10-06T22:18:19.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:19 smithi190 ceph-mon[134891]: pgmap v110: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 614 op/s 2023-10-06T22:18:21.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:21 smithi057 ceph-mon[98492]: pgmap v111: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 494 op/s 2023-10-06T22:18:21.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:21 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:18:21.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:21 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:18:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:21 smithi156 ceph-mon[134358]: pgmap v111: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 494 op/s 2023-10-06T22:18:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:21 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:18:21.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:21 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:18:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:21 smithi190 ceph-mon[134891]: pgmap v111: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 494 op/s 2023-10-06T22:18:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:21 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:18:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:21 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:18:22.053 INFO:tasks.workunit.client.0.smithi057.stdout: 156 804071 827 542175 552 154438 39 2023-10-06T22:18:22.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:22 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:18:22.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:22 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:18:22.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:22 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:18:22.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:22 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:18:22.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:22 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:18:22.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:22 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:18:23.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:23 smithi156 ceph-mon[134358]: pgmap v112: 97 pgs: 97 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.6 MiB/s wr, 557 op/s 2023-10-06T22:18:23.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:23 smithi190 ceph-mon[134891]: pgmap v112: 97 pgs: 97 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.6 MiB/s wr, 557 op/s 2023-10-06T22:18:23.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:23 smithi057 ceph-mon[98492]: pgmap v112: 97 pgs: 97 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.6 MiB/s wr, 557 op/s 2023-10-06T22:18:24.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:24 smithi190 ceph-mon[134891]: pgmap v113: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 407 op/s 2023-10-06T22:18:24.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:24 smithi057 ceph-mon[98492]: pgmap v113: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 407 op/s 2023-10-06T22:18:24.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:24 smithi156 ceph-mon[134358]: pgmap v113: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 407 op/s 2023-10-06T22:18:26.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:26 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:26.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:26 smithi057 ceph-mon[98492]: pgmap v114: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 391 op/s 2023-10-06T22:18:26.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:26 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:26.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:26 smithi156 ceph-mon[134358]: pgmap v114: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 391 op/s 2023-10-06T22:18:26.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:26 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:26.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:26 smithi190 ceph-mon[134891]: pgmap v114: 97 pgs: 97 active+clean; 570 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 391 op/s 2023-10-06T22:18:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.499+0000 7f6780b9b700 1 --2- 172.21.15.57:0/2492695286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10f620 0x7f677c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:18:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.500+0000 7f6782dff700 1 -- 172.21.15.57:0/2492695286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 msgr2=0x7f677c11ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.500+0000 7f6782dff700 1 --2- 172.21.15.57:0/2492695286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 0x7f677c11ca80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f676c009990 tx=0x7f676c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.500+0000 7f6782dff700 1 -- 172.21.15.57:0/2492695286 shutdown_connections 2023-10-06T22:18:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.500+0000 7f6782dff700 1 --2- 172.21.15.57:0/2492695286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 0x7f677c11ca80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.500+0000 7f6782dff700 1 --2- 172.21.15.57:0/2492695286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10f620 0x7f677c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.500+0000 7f6782dff700 1 --2- 172.21.15.57:0/2492695286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10fa90 0x7f677c1083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.500+0000 7f6782dff700 1 -- 172.21.15.57:0/2492695286 >> 172.21.15.57:0/2492695286 conn(0x7f677c06cfa0 msgr2=0x7f677c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.501+0000 7f6782dff700 1 -- 172.21.15.57:0/2492695286 shutdown_connections 2023-10-06T22:18:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.501+0000 7f6782dff700 1 -- 172.21.15.57:0/2492695286 wait complete. 2023-10-06T22:18:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.503+0000 7f6782dff700 1 Processor -- start 2023-10-06T22:18:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.504+0000 7f6782dff700 1 -- start start 2023-10-06T22:18:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.504+0000 7f6782dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10f620 0x7f677c117900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.504+0000 7f6782dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10fa90 0x7f677c117e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.504+0000 7f6782dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 0x7f677c110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.504+0000 7f6782dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c110e40 con 0x7f677c11a690 2023-10-06T22:18:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.504+0000 7f6782dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c110fb0 con 0x7f677c10f620 2023-10-06T22:18:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.504+0000 7f6782dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c111170 con 0x7f677c10fa90 2023-10-06T22:18:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.507+0000 7f677bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10fa90 0x7f677c117e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.507+0000 7f677bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10fa90 0x7f677c117e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45496/0 (socket says 172.21.15.57:45496) 2023-10-06T22:18:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.507+0000 7f677bfff700 1 -- 172.21.15.57:0/3103660475 learned_addr learned my addr 172.21.15.57:0/3103660475 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.507+0000 7f678139c700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 0x7f677c110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.507+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10f620 0x7f677c117900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.508+0000 7f677bfff700 1 -- 172.21.15.57:0/3103660475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10f620 msgr2=0x7f677c117900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.508+0000 7f677bfff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10f620 0x7f677c117900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.508+0000 7f677bfff700 1 -- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 msgr2=0x7f677c110700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.508+0000 7f677bfff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 0x7f677c110700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.508+0000 7f677bfff700 1 -- 172.21.15.57:0/3103660475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0095f0 con 0x7f677c10fa90 2023-10-06T22:18:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.508+0000 7f677bfff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10fa90 0x7f677c117e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6774007f10 tx=0x7f677400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.508+0000 7f6779ffb700 1 -- 172.21.15.57:0/3103660475 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400de90 con 0x7f677c10fa90 2023-10-06T22:18:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.509+0000 7f6779ffb700 1 -- 172.21.15.57:0/3103660475 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f677400f040 con 0x7f677c10fa90 2023-10-06T22:18:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.509+0000 7f6779ffb700 1 -- 172.21.15.57:0/3103660475 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774015400 con 0x7f677c10fa90 2023-10-06T22:18:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.509+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c111450 con 0x7f677c10fa90 2023-10-06T22:18:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.509+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c1156d0 con 0x7f677c10fa90 2023-10-06T22:18:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.518+0000 7f6796708b80 1 Processor -- start 2023-10-06T22:18:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.519+0000 7f6796708b80 1 -- start start 2023-10-06T22:18:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.519+0000 7f6796708b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea77fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.519+0000 7f6796708b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea78520 0x55ed2ea82e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.519+0000 7f6796708b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea833d0 0x55ed2ea857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.519+0000 7f6796708b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2e9e1400 con 0x55ed2ea77c00 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.519+0000 7f6796708b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2e9d2bc0 con 0x55ed2ea78520 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.519+0000 7f6796708b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2e9d04e0 con 0x55ed2ea833d0 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.520+0000 7f6780b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea78520 0x55ed2ea82e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.520+0000 7f6780b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea78520 0x55ed2ea82e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55488/0 (socket says 172.21.15.57:55488) 2023-10-06T22:18:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.520+0000 7f6780b9b700 1 -- 172.21.15.57:0/1760871107 learned_addr learned my addr 172.21.15.57:0/1760871107 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.520+0000 7f678139c700 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea77fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.521+0000 7f6780b9b700 1 -- 172.21.15.57:0/1760871107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea833d0 msgr2=0x55ed2ea857c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:18:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.521+0000 7f6780b9b700 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea833d0 0x55ed2ea857c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.521+0000 7f6780b9b700 1 -- 172.21.15.57:0/1760871107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 msgr2=0x55ed2ea77fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.521+0000 7f6780b9b700 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea77fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.521+0000 7f6780b9b700 1 -- 172.21.15.57:0/1760871107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed2e7f6d80 con 0x55ed2ea78520 2023-10-06T22:18:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.522+0000 7f6780b9b700 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea78520 0x55ed2ea82e90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f677000bbc0 tx=0x7f677000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a23075eaae5bc8d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.522+0000 7f6762ffd700 1 -- 172.21.15.57:0/1760871107 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677000dd00 con 0x55ed2ea78520 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.522+0000 7f6762ffd700 1 -- 172.21.15.57:0/1760871107 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f677000de60 con 0x55ed2ea78520 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.523+0000 7f6762ffd700 1 -- 172.21.15.57:0/1760871107 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677000f710 con 0x55ed2ea78520 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.523+0000 7f6796708b80 1 -- 172.21.15.57:0/1760871107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea78520 msgr2=0x55ed2ea82e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.523+0000 7f6796708b80 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea78520 0x55ed2ea82e90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f677000bbc0 tx=0x7f677000bed0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.524+0000 7f6796708b80 1 -- 172.21.15.57:0/1760871107 shutdown_connections 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.524+0000 7f6796708b80 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea833d0 0x55ed2ea857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.524+0000 7f6796708b80 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea77fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.524+0000 7f6796708b80 1 --2- 172.21.15.57:0/1760871107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea78520 0x55ed2ea82e90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.524+0000 7f6796708b80 1 -- 172.21.15.57:0/1760871107 >> 172.21.15.57:0/1760871107 conn(0x55ed2e9ed200 msgr2=0x55ed2ea74080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.528+0000 7f6779ffb700 1 -- 172.21.15.57:0/3103660475 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6774004000 con 0x7f677c10fa90 2023-10-06T22:18:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.531+0000 7f6796708b80 1 -- 172.21.15.57:0/1760871107 shutdown_connections 2023-10-06T22:18:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.531+0000 7f6779ffb700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f67640a1710 0x7f67640a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.532+0000 7f6779ffb700 1 -- 172.21.15.57:0/3103660475 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f67740cc9d0 con 0x7f677c10fa90 2023-10-06T22:18:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.533+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f67640a1710 0x7f67640a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.534+0000 7f6796708b80 1 -- 172.21.15.57:0/1760871107 wait complete. 2023-10-06T22:18:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.534+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f67640a1710 0x7f67640a3bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6770017df0 tx=0x7f677000dfd0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.535+0000 7f6796708b80 1 -- start start 2023-10-06T22:18:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.535+0000 7f6796708b80 1 Processor -- start 2023-10-06T22:18:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f6796708b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea80250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f6796708b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f6796708b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f6796708b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2e9cbdd0 con 0x55ed2ea77c00 2023-10-06T22:18:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f6796708b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2e9686b0 con 0x55ed2ea80cd0 2023-10-06T22:18:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f6796708b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2e968350 con 0x55ed2ea78520 2023-10-06T22:18:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f6796708b80 10 client.0 _ll_register_callbacks cb 0x55ed2e7c57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.536+0000 7f677bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea80790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45504/0 (socket says 172.21.15.57:45504) 2023-10-06T22:18:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 -- 172.21.15.57:0/3516395362 learned_addr learned my addr 172.21.15.57:0/3516395362 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f678139c700 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 -- 172.21.15.57:0/3516395362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 msgr2=0x55ed2ea8f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 -- 172.21.15.57:0/3516395362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 msgr2=0x55ed2ea80250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:18:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea80250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 -- 172.21.15.57:0/3516395362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677000b7f0 con 0x55ed2ea78520 2023-10-06T22:18:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f677bfff700 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea80790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f67740996c0 tx=0x7f677409a0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5313e3feb16c7ee1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.537+0000 7f675bfff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:18:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.538+0000 7f675bfff700 1 -- 172.21.15.57:0/3516395362 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774099e20 con 0x55ed2ea78520 2023-10-06T22:18:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.538+0000 7f675bfff700 1 -- 172.21.15.57:0/3516395362 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f67740b2260 con 0x55ed2ea78520 2023-10-06T22:18:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.538+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed2ea6bd50 con 0x55ed2ea78520 2023-10-06T22:18:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f675bfff700 1 -- 172.21.15.57:0/3516395362 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67740a1d50 con 0x55ed2ea78520 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 10 client.26413 fetch_fsmap learned FSMap version 36 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ed2e7dfc00 con 0x55ed2ea78520 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f675bfff700 1 -- 172.21.15.57:0/3516395362 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed2ea6bd50 con 0x55ed2ea78520 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f675bfff700 1 -- 172.21.15.57:0/3516395362 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f67740a1950 con 0x55ed2ea78520 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 10 client.26413 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 10 client.26413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 20 client.26413 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2ea934b0 0x55ed2ea958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 4 client.26413 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:18:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.539+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed2e9c4db0 con 0x55ed2ea934b0 2023-10-06T22:18:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.540+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2ea934b0 0x55ed2ea958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.541+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2ea934b0 0x55ed2ea958d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f89e555ee48fe3 server_cookie=6a9cf545a2e32c31 in_seq=0 out_seq=0 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.541+0000 7f675bfff700 10 client.26413 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.542+0000 7f675bfff700 1 -- 172.21.15.57:0/3516395362 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed2e9c4db0 con 0x55ed2ea934b0 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.542+0000 7f675bfff700 10 client.26413 handle_command_reply: tid=0 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.542+0000 7f6796708b80 1 client.26413 shutdown 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 msgr2=0x55ed2ea80790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea80790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f67740996c0 tx=0x7f677409a0c0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 shutdown_connections 2023-10-06T22:18:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2ea934b0 0x55ed2ea958d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea80250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f675bfff700 0 client.26413 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 --2- 172.21.15.57:0/3516395362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea80790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 >> 172.21.15.57:0/3516395362 conn(0x55ed2e9ed200 msgr2=0x55ed2ea73c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 shutdown_connections 2023-10-06T22:18:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 1 -- 172.21.15.57:0/3516395362 wait complete. 2023-10-06T22:18:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.543+0000 7f6796708b80 20 client.26413 trim_cache size 0 max 16384 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 Processor -- start 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 -- start start 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea89f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2e84c8c0 con 0x55ed2ea77c00 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2ea79710 con 0x55ed2ea80cd0 2023-10-06T22:18:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.568+0000 7f6796708b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2ea78e20 con 0x55ed2ea78520 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45514/0 (socket says 172.21.15.57:45514) 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 -- 172.21.15.57:0/2962336334 learned_addr learned my addr 172.21.15.57:0/2962336334 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f6780b9b700 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea89f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f678139c700 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 -- 172.21.15.57:0/2962336334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 msgr2=0x55ed2ea8e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 -- 172.21.15.57:0/2962336334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 msgr2=0x55ed2ea89f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea89f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 -- 172.21.15.57:0/2962336334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed2e81d1f0 con 0x55ed2ea78520 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f678139c700 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:18:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.569+0000 7f677bfff700 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea727b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f677400d7b0 tx=0x7f6774097f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe65a2df8bdfa72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.570+0000 7f67637fe700 1 -- 172.21.15.57:0/2962336334 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774099a40 con 0x55ed2ea78520 2023-10-06T22:18:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.570+0000 7f67637fe700 1 -- 172.21.15.57:0/2962336334 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67740a1ac0 con 0x55ed2ea78520 2023-10-06T22:18:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.570+0000 7f67637fe700 1 -- 172.21.15.57:0/2962336334 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774099c20 con 0x55ed2ea78520 2023-10-06T22:18:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.570+0000 7f6796708b80 1 -- 172.21.15.57:0/2962336334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 msgr2=0x55ed2ea727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.570+0000 7f6796708b80 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea727b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f677400d7b0 tx=0x7f6774097f70 comp rx=0 tx=0).stop 2023-10-06T22:18:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 -- 172.21.15.57:0/2962336334 shutdown_connections 2023-10-06T22:18:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2ea8e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea77c00 0x55ed2ea89f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 --2- 172.21.15.57:0/2962336334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea78520 0x55ed2ea727b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 -- 172.21.15.57:0/2962336334 >> 172.21.15.57:0/2962336334 conn(0x55ed2e9ed200 msgr2=0x55ed2ea9a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 -- 172.21.15.57:0/2962336334 shutdown_connections 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 -- 172.21.15.57:0/2962336334 wait complete. 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.571+0000 7f6796708b80 1 -- start start 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 1 Processor -- start 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea77c00 0x55ed2eadb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea78520 0x55ed2eadb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2eadbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2eae6380 con 0x55ed2ea78520 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2eae64c0 con 0x55ed2ea80cd0 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2eae6680 con 0x55ed2ea77c00 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6796708b80 10 client.0 _ll_register_callbacks cb 0x7f6774099880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6780b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea77c00 0x55ed2eadb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6780b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea77c00 0x55ed2eadb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45518/0 (socket says 172.21.15.57:45518) 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.572+0000 7f6780b9b700 1 -- 172.21.15.57:0/3796845010 learned_addr learned my addr 172.21.15.57:0/3796845010 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f677bfff700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea78520 0x55ed2eadb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f678139c700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2eadbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f6780b9b700 1 -- 172.21.15.57:0/3796845010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 msgr2=0x55ed2eadbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2eadbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f6780b9b700 1 -- 172.21.15.57:0/3796845010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea78520 msgr2=0x55ed2eadb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea78520 0x55ed2eadb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f6780b9b700 1 -- 172.21.15.57:0/3796845010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed2e7dfc00 con 0x55ed2ea77c00 2023-10-06T22:18:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.573+0000 7f677bfff700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea78520 0x55ed2eadb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:18:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f6780b9b700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea77c00 0x55ed2eadb2b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6770017f60 tx=0x7f6770017e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2f052b8b3f7ddb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f675bfff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:18:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f675bfff700 1 -- 172.21.15.57:0/3796845010 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770030a00 con 0x55ed2ea77c00 2023-10-06T22:18:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f677000de60 con 0x55ed2ea77c00 2023-10-06T22:18:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f675bfff700 1 -- 172.21.15.57:0/3796845010 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f6770013070 con 0x55ed2ea77c00 2023-10-06T22:18:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f675bfff700 1 -- 172.21.15.57:0/3796845010 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6770030b60 con 0x55ed2ea77c00 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f6796708b80 10 client.26419 fetch_fsmap learned FSMap version 36 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.574+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ed2e7f5250 con 0x55ed2ea77c00 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.575+0000 7f675bfff700 1 -- 172.21.15.57:0/3796845010 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f677000de60 con 0x55ed2ea77c00 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.575+0000 7f675bfff700 1 -- 172.21.15.57:0/3796845010 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f6770004c60 con 0x55ed2ea77c00 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.576+0000 7f6796708b80 10 client.26419 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.576+0000 7f6796708b80 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.576+0000 7f6796708b80 20 client.26419 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.576+0000 7f6796708b80 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2eae7cf0 0x55ed2eaea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.576+0000 7f6796708b80 4 client.26419 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.576+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed2e9c4ad0 con 0x55ed2eae7cf0 2023-10-06T22:18:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.576+0000 7f677bfff700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2eae7cf0 0x55ed2eaea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.577+0000 7f677bfff700 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2eae7cf0 0x55ed2eaea0e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b0fc8b6466c2c9 server_cookie=3adefe5b209f18a3 in_seq=0 out_seq=0 2023-10-06T22:18:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:27.577+0000 7f675bfff700 10 client.26419 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.223+0000 7f675bfff700 1 -- 172.21.15.57:0/3796845010 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed2e9c4ad0 con 0x55ed2eae7cf0 2023-10-06T22:18:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.223+0000 7f675bfff700 10 client.26419 handle_command_reply: tid=0 2023-10-06T22:18:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.223+0000 7f6796708b80 1 client.26419 shutdown 2023-10-06T22:18:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.223+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea77c00 msgr2=0x55ed2eadb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.223+0000 7f6796708b80 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea77c00 0x55ed2eadb2b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6770017f60 tx=0x7f6770017e50 comp rx=0 tx=0).stop 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.224+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 shutdown_connections 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.224+0000 7f6796708b80 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ed2eae7cf0 0x55ed2eaea0e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.224+0000 7f6796708b80 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed2ea80cd0 0x55ed2eadbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.224+0000 7f6796708b80 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed2ea77c00 0x55ed2eadb2b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.224+0000 7f6796708b80 1 --2- 172.21.15.57:0/3796845010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ed2ea78520 0x55ed2eadb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.224+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 >> 172.21.15.57:0/3796845010 conn(0x55ed2e9ed200 msgr2=0x55ed2ea7b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.229+0000 7f675bfff700 0 client.26419 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.238+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 shutdown_connections 2023-10-06T22:18:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.238+0000 7f6796708b80 1 -- 172.21.15.57:0/3796845010 wait complete. 2023-10-06T22:18:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.239+0000 7f6796708b80 20 client.26419 trim_cache size 0 max 16384 2023-10-06T22:18:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:18:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:18:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:18:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.496+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f67640a1710 msgr2=0x7f67640a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.496+0000 7f6782dff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f67640a1710 0x7f67640a3bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6770017df0 tx=0x7f677000dfd0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.497+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10fa90 msgr2=0x7f677c117e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.497+0000 7f6782dff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10fa90 0x7f677c117e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6774007f10 tx=0x7f677400d3b0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.501+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 shutdown_connections 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.501+0000 7f6782dff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f67640a1710 0x7f67640a3bd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.501+0000 7f6782dff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f677c11a690 0x7f677c110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.501+0000 7f6782dff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c10f620 0x7f677c117900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.501+0000 7f6782dff700 1 --2- 172.21.15.57:0/3103660475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f677c10fa90 0x7f677c117e80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.502+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 >> 172.21.15.57:0/3103660475 conn(0x7f677c06cfa0 msgr2=0x7f677c10d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.502+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 shutdown_connections 2023-10-06T22:18:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:28.502+0000 7f6782dff700 1 -- 172.21.15.57:0/3103660475 wait complete. 2023-10-06T22:18:29.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:28 smithi156 ceph-mon[134358]: pgmap v115: 97 pgs: 97 active+clean; 575 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 423 op/s 2023-10-06T22:18:29.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:28 smithi190 ceph-mon[134891]: pgmap v115: 97 pgs: 97 active+clean; 575 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 423 op/s 2023-10-06T22:18:29.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:28 smithi057 ceph-mon[98492]: pgmap v115: 97 pgs: 97 active+clean; 575 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 423 op/s 2023-10-06T22:18:31.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:30 smithi156 ceph-mon[134358]: pgmap v116: 97 pgs: 97 active+clean; 575 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1010 KiB/s wr, 99 op/s 2023-10-06T22:18:31.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:30 smithi190 ceph-mon[134891]: pgmap v116: 97 pgs: 97 active+clean; 575 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1010 KiB/s wr, 99 op/s 2023-10-06T22:18:31.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:30 smithi057 ceph-mon[98492]: pgmap v116: 97 pgs: 97 active+clean; 575 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1010 KiB/s wr, 99 op/s 2023-10-06T22:18:31.589 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:18:32.058 INFO:tasks.workunit.client.0.smithi057.stdout: 168 868139 543 590796 666 160654 44 2023-10-06T22:18:33.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:33 smithi190 ceph-mon[134891]: pgmap v117: 97 pgs: 97 active+clean; 576 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 461 op/s 2023-10-06T22:18:33.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:33 smithi057 ceph-mon[98492]: pgmap v117: 97 pgs: 97 active+clean; 576 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 461 op/s 2023-10-06T22:18:33.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:33 smithi156 ceph-mon[134358]: pgmap v117: 97 pgs: 97 active+clean; 576 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 461 op/s 2023-10-06T22:18:35.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:35 smithi190 ceph-mon[134891]: pgmap v118: 97 pgs: 97 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 437 op/s 2023-10-06T22:18:35.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:35 smithi057 ceph-mon[98492]: pgmap v118: 97 pgs: 97 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 437 op/s 2023-10-06T22:18:35.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:35 smithi156 ceph-mon[134358]: pgmap v118: 97 pgs: 97 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 437 op/s 2023-10-06T22:18:37.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.182+0000 7fdd97dff700 1 -- 172.21.15.57:0/2253453196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 msgr2=0x7fdd9011ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.182+0000 7fdd97dff700 1 --2- 172.21.15.57:0/2253453196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 0x7fdd9011ca80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdd8801c0d0 tx=0x7fdd8801c3e0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.192+0000 7fdd97dff700 1 -- 172.21.15.57:0/2253453196 shutdown_connections 2023-10-06T22:18:37.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.192+0000 7fdd97dff700 1 --2- 172.21.15.57:0/2253453196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 0x7fdd9011ca80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.192+0000 7fdd97dff700 1 --2- 172.21.15.57:0/2253453196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd9010f640 0x7fdd90107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.192+0000 7fdd97dff700 1 --2- 172.21.15.57:0/2253453196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd901083f0 0x7fdd90108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.192+0000 7fdd97dff700 1 -- 172.21.15.57:0/2253453196 >> 172.21.15.57:0/2253453196 conn(0x7fdd9006d1a0 msgr2=0x7fdd9006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:37.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.206+0000 7fdd97dff700 1 -- 172.21.15.57:0/2253453196 shutdown_connections 2023-10-06T22:18:37.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.206+0000 7fdd97dff700 1 -- 172.21.15.57:0/2253453196 wait complete. 2023-10-06T22:18:37.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.208+0000 7fdd97dff700 1 Processor -- start 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.208+0000 7fdd97dff700 1 -- start start 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.209+0000 7fdd97dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd901083f0 0x7fdd90115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.209+0000 7fdd97dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd9010f640 0x7fdd90115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.209+0000 7fdd97dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 0x7fdd901a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.209+0000 7fdd97dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd901a39a0 con 0x7fdd901083f0 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.209+0000 7fdd97dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd901a3b10 con 0x7fdd9011a690 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.209+0000 7fdd97dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd901a3cd0 con 0x7fdd9010f640 2023-10-06T22:18:37.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.210+0000 7fdd95b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd901083f0 0x7fdd90115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.210+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd9010f640 0x7fdd90115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.210+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd9010f640 0x7fdd90115f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33582/0 (socket says 172.21.15.57:33582) 2023-10-06T22:18:37.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.210+0000 7fdd9539a700 1 -- 172.21.15.57:0/4039261123 learned_addr learned my addr 172.21.15.57:0/4039261123 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:37.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.210+0000 7fdd9639c700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 0x7fdd901a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.221+0000 7fdd9539a700 1 -- 172.21.15.57:0/4039261123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 msgr2=0x7fdd901a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.221+0000 7fdd9539a700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 0x7fdd901a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.221+0000 7fdd9539a700 1 -- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd901083f0 msgr2=0x7fdd90115a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.221+0000 7fdd9539a700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd901083f0 0x7fdd90115a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.222+0000 7fdd9539a700 1 -- 172.21.15.57:0/4039261123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8801ad10 con 0x7fdd9010f640 2023-10-06T22:18:37.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.225+0000 7fdd9539a700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd9010f640 0x7fdd90115f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000c920 tx=0x7fdd8000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:37.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.225+0000 7fdd86ffd700 1 -- 172.21.15.57:0/4039261123 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004a30 con 0x7fdd9010f640 2023-10-06T22:18:37.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.225+0000 7fdd86ffd700 1 -- 172.21.15.57:0/4039261123 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd800054e0 con 0x7fdd9010f640 2023-10-06T22:18:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.225+0000 7fdd86ffd700 1 -- 172.21.15.57:0/4039261123 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80007bd0 con 0x7fdd9010f640 2023-10-06T22:18:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.226+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd901a3fb0 con 0x7fdd9010f640 2023-10-06T22:18:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.226+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd901aa630 con 0x7fdd9010f640 2023-10-06T22:18:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.234+0000 7fddab7dbb80 1 Processor -- start 2023-10-06T22:18:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.235+0000 7fddab7dbb80 1 -- start start 2023-10-06T22:18:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.235+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d51f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.235+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d5204350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.235+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5206c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.235+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d5162460 con 0x5555d51f90c0 2023-10-06T22:18:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.235+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d5153c20 con 0x5555d51f99e0 2023-10-06T22:18:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.235+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d5151540 con 0x5555d5204890 2023-10-06T22:18:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.239+0000 7fdd9639c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d51f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.239+0000 7fdd9639c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d51f94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55916/0 (socket says 172.21.15.57:55916) 2023-10-06T22:18:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.239+0000 7fdd9639c700 1 -- 172.21.15.57:0/1127814789 learned_addr learned my addr 172.21.15.57:0/1127814789 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:37.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.245+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d5204350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.254+0000 7fdd9539a700 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5206c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.254+0000 7fdd86ffd700 1 -- 172.21.15.57:0/4039261123 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdd80007d50 con 0x7fdd9010f640 2023-10-06T22:18:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.258+0000 7fdd9639c700 1 -- 172.21.15.57:0/1127814789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 msgr2=0x5555d5206c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.258+0000 7fdd9639c700 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5206c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.258+0000 7fdd9639c700 1 -- 172.21.15.57:0/1127814789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 msgr2=0x5555d5204350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.258+0000 7fdd9639c700 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d5204350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.258+0000 7fdd9639c700 1 -- 172.21.15.57:0/1127814789 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555d4f77de0 con 0x5555d51f90c0 2023-10-06T22:18:37.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.258+0000 7fdd86ffd700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd7c0a1800 0x7fdd7c0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.258+0000 7fdd86ffd700 1 -- 172.21.15.57:0/4039261123 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fdd800ccfb0 con 0x7fdd9010f640 2023-10-06T22:18:37.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.267+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d5204350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:18:37.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.267+0000 7fdd9639c700 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d51f94a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdd88019ca0 tx=0x7fdd88019cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f077b6e4c539aec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:37.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.268+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd7c0a1800 0x7fdd7c0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.274+0000 7fdd7bfff700 1 -- 172.21.15.57:0/1127814789 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8801f040 con 0x5555d51f90c0 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.274+0000 7fdd7bfff700 1 -- 172.21.15.57:0/1127814789 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd8801a2d0 con 0x5555d51f90c0 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.274+0000 7fdd7bfff700 1 -- 172.21.15.57:0/1127814789 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8802d440 con 0x5555d51f90c0 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.275+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1127814789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 msgr2=0x5555d51f94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.275+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d51f94a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdd88019ca0 tx=0x7fdd88019cd0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.275+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1127814789 shutdown_connections 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.275+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5206c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.275+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d51f94a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.275+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1127814789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d5204350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.275+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1127814789 >> 172.21.15.57:0/1127814789 conn(0x5555d516e260 msgr2=0x5555d51f5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:37.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.277+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1127814789 shutdown_connections 2023-10-06T22:18:37.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.277+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1127814789 wait complete. 2023-10-06T22:18:37.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.278+0000 7fddab7dbb80 1 -- start start 2023-10-06T22:18:37.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.278+0000 7fddab7dbb80 1 Processor -- start 2023-10-06T22:18:37.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.279+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d5201570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.279+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d51f99e0 0x5555d5201ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.279+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d5204890 0x5555d5201ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.279+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d514ce30 con 0x5555d5204890 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.279+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d50e9710 con 0x5555d51f90c0 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.279+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d50e93b0 con 0x5555d51f99e0 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.279+0000 7fddab7dbb80 10 client.0 _ll_register_callbacks cb 0x5555d4f46800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.281+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d51f99e0 0x5555d5201ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.281+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d51f99e0 0x5555d5201ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33600/0 (socket says 172.21.15.57:33600) 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.281+0000 7fdd9539a700 1 -- 172.21.15.57:0/1041607916 learned_addr learned my addr 172.21.15.57:0/1041607916 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:37.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.281+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d5201570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.282+0000 7fdd9539a700 1 -- 172.21.15.57:0/1041607916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 msgr2=0x5555d5201570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.282+0000 7fdd9539a700 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d5201570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.282+0000 7fdd9539a700 1 -- 172.21.15.57:0/1041607916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d5204890 msgr2=0x5555d5201ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.282+0000 7fdd9539a700 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d5204890 0x5555d5201ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.282+0000 7fdd9539a700 1 -- 172.21.15.57:0/1041607916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9011d430 con 0x5555d51f99e0 2023-10-06T22:18:37.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.286+0000 7fdd9539a700 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d51f99e0 0x5555d5201ab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd800a1e00 tx=0x7fdd8009a9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcb1ebe2173c76d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:37.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.287+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd7c0a1800 0x7fdd7c0a3cc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c005950 tx=0x7fdd8c00b410 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.312+0000 7fdd797fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:18:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.312+0000 7fdd797fa700 1 -- 172.21.15.57:0/1041607916 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8009a230 con 0x5555d51f99e0 2023-10-06T22:18:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.312+0000 7fdd797fa700 1 -- 172.21.15.57:0/1041607916 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fdd8009b240 con 0x5555d51f99e0 2023-10-06T22:18:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.312+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555d51ecf10 con 0x5555d51f99e0 2023-10-06T22:18:37.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.333+0000 7fdd797fa700 1 -- 172.21.15.57:0/1041607916 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd800a2330 con 0x5555d51f99e0 2023-10-06T22:18:37.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.333+0000 7fddab7dbb80 10 client.26428 fetch_fsmap learned FSMap version 36 2023-10-06T22:18:37.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.333+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5555d4f60c60 con 0x5555d51f99e0 2023-10-06T22:18:37.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fdd797fa700 1 -- 172.21.15.57:0/1041607916 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555d51ecf10 con 0x5555d51f99e0 2023-10-06T22:18:37.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fdd797fa700 1 -- 172.21.15.57:0/1041607916 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdd80099aa0 con 0x5555d51f99e0 2023-10-06T22:18:37.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fddab7dbb80 10 client.26428 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T22:18:37.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fddab7dbb80 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:37.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fddab7dbb80 20 client.26428 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:37.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d5214370 0x5555d5216790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fddab7dbb80 4 client.26428 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:18:37.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.341+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555d5145e10 con 0x5555d5214370 2023-10-06T22:18:37.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.353+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d5214370 0x5555d5216790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.379+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d5214370 0x5555d5216790 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2596c5408485b000 server_cookie=aa86fb77ddb5d4cc in_seq=0 out_seq=0 2023-10-06T22:18:37.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.384+0000 7fdd797fa700 10 client.26428 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.389+0000 7fdd797fa700 1 -- 172.21.15.57:0/1041607916 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555d5145e10 con 0x5555d5214370 2023-10-06T22:18:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.389+0000 7fdd797fa700 10 client.26428 handle_command_reply: tid=0 2023-10-06T22:18:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.389+0000 7fddab7dbb80 1 client.26428 shutdown 2023-10-06T22:18:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d51f99e0 msgr2=0x5555d5201ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d51f99e0 0x5555d5201ab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd800a1e00 tx=0x7fdd8009a9c0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 shutdown_connections 2023-10-06T22:18:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d5214370 0x5555d5216790 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d5204890 0x5555d5201ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d5201570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/1041607916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d51f99e0 0x5555d5201ab0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 >> 172.21.15.57:0/1041607916 conn(0x5555d516e260 msgr2=0x5555d51f4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.390+0000 7fdd797fa700 0 client.26428 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.391+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 shutdown_connections 2023-10-06T22:18:37.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.391+0000 7fddab7dbb80 1 -- 172.21.15.57:0/1041607916 wait complete. 2023-10-06T22:18:37.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.391+0000 7fddab7dbb80 20 client.26428 trim_cache size 0 max 16384 2023-10-06T22:18:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:37 smithi057 ceph-mon[98492]: pgmap v119: 97 pgs: 97 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 433 op/s 2023-10-06T22:18:37.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:37 smithi156 ceph-mon[134358]: pgmap v119: 97 pgs: 97 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 433 op/s 2023-10-06T22:18:37.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:37 smithi190 ceph-mon[134891]: pgmap v119: 97 pgs: 97 active+clean; 577 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 433 op/s 2023-10-06T22:18:37.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.593+0000 7fddab7dbb80 1 Processor -- start 2023-10-06T22:18:37.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.600+0000 7fddab7dbb80 1 -- start start 2023-10-06T22:18:37.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.600+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d5201e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.600+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d51fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.600+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d51fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.600+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d4fcd920 con 0x5555d51f90c0 2023-10-06T22:18:37.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.600+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d51fd3c0 con 0x5555d51f99e0 2023-10-06T22:18:37.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.600+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d52028c0 con 0x5555d5204890 2023-10-06T22:18:37.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.603+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d51fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.603+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d51fe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49830/0 (socket says 172.21.15.57:49830) 2023-10-06T22:18:37.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.603+0000 7fdd9539a700 1 -- 172.21.15.57:0/2608058309 learned_addr learned my addr 172.21.15.57:0/2608058309 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:37.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.603+0000 7fdd9639c700 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d51fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.603+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d5201e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.606+0000 7fdd9539a700 1 -- 172.21.15.57:0/2608058309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 msgr2=0x5555d51fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.606+0000 7fdd9539a700 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d51fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.606+0000 7fdd9539a700 1 -- 172.21.15.57:0/2608058309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 msgr2=0x5555d5201e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.606+0000 7fdd9539a700 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d5201e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.606+0000 7fdd9539a700 1 -- 172.21.15.57:0/2608058309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555d4f9e250 con 0x5555d51f99e0 2023-10-06T22:18:37.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.616+0000 7fdd9539a700 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d51fe7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000cf80 tx=0x7fdd8009a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dd7a9037bd612f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:37.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.617+0000 7fdd84bff700 1 -- 172.21.15.57:0/2608058309 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800a2330 con 0x5555d51f99e0 2023-10-06T22:18:37.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.617+0000 7fdd84bff700 1 -- 172.21.15.57:0/2608058309 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd80099aa0 con 0x5555d51f99e0 2023-10-06T22:18:37.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.617+0000 7fdd84bff700 1 -- 172.21.15.57:0/2608058309 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8009a290 con 0x5555d51f99e0 2023-10-06T22:18:37.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.629+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2608058309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 msgr2=0x5555d51fe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.629+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d51fe7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000cf80 tx=0x7fdd8009a730 comp rx=0 tx=0).stop 2023-10-06T22:18:37.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.633+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2608058309 shutdown_connections 2023-10-06T22:18:37.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.633+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d51fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.633+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f90c0 0x5555d5201e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.633+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2608058309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f99e0 0x5555d51fe7e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.633+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2608058309 >> 172.21.15.57:0/2608058309 conn(0x5555d516e260 msgr2=0x5555d5200360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:37.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.633+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2608058309 shutdown_connections 2023-10-06T22:18:37.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.633+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2608058309 wait complete. 2023-10-06T22:18:37.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.634+0000 7fddab7dbb80 1 -- start start 2023-10-06T22:18:37.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.634+0000 7fddab7dbb80 1 Processor -- start 2023-10-06T22:18:37.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.635+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d520b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.635+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f99e0 0x5555d5209720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.635+0000 7fddab7dbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5209c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.635+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d520a370 con 0x5555d51f99e0 2023-10-06T22:18:37.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.635+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d520a4e0 con 0x5555d51f90c0 2023-10-06T22:18:37.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.635+0000 7fddab7dbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555d5268170 con 0x5555d5204890 2023-10-06T22:18:37.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.635+0000 7fddab7dbb80 10 client.0 _ll_register_callbacks cb 0x7fdd8000c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:37.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.641+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f99e0 0x5555d5209720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.641+0000 7fdd9539a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f99e0 0x5555d5209720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55940/0 (socket says 172.21.15.57:55940) 2023-10-06T22:18:37.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.641+0000 7fdd9539a700 1 -- 172.21.15.57:0/2800909307 learned_addr learned my addr 172.21.15.57:0/2800909307 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:37.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.642+0000 7fdd9639c700 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5209c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.642+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d520b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.645+0000 7fdd95b9b700 1 -- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 msgr2=0x5555d5209c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.645+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5209c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.645+0000 7fdd95b9b700 1 -- 172.21.15.57:0/2800909307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f99e0 msgr2=0x5555d5209720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:37.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.645+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f99e0 0x5555d5209720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.645+0000 7fdd95b9b700 1 -- 172.21.15.57:0/2800909307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555d4f60c60 con 0x5555d51f90c0 2023-10-06T22:18:37.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.650+0000 7fdd95b9b700 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d520b3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c011a50 tx=0x7fdd8c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5387e0a0ad3a6c2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:37.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.658+0000 7fdd797fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:18:37.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.658+0000 7fdd797fa700 1 -- 172.21.15.57:0/2800909307 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c025040 con 0x5555d51f90c0 2023-10-06T22:18:37.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.658+0000 7fdd797fa700 1 -- 172.21.15.57:0/2800909307 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fdd8c018070 con 0x5555d51f90c0 2023-10-06T22:18:37.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.658+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd8801a2d0 con 0x5555d51f90c0 2023-10-06T22:18:37.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.681+0000 7fdd797fa700 1 -- 172.21.15.57:0/2800909307 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd8c0275b0 con 0x5555d51f90c0 2023-10-06T22:18:37.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.682+0000 7fddab7dbb80 10 client.26456 fetch_fsmap learned FSMap version 36 2023-10-06T22:18:37.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.682+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5555d4f762b0 con 0x5555d51f90c0 2023-10-06T22:18:37.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fdd797fa700 1 -- 172.21.15.57:0/2800909307 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd8801a2d0 con 0x5555d51f90c0 2023-10-06T22:18:37.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fdd797fa700 1 -- 172.21.15.57:0/2800909307 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdd8c012560 con 0x5555d51f90c0 2023-10-06T22:18:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fddab7dbb80 10 client.26456 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T22:18:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fddab7dbb80 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fddab7dbb80 20 client.26456 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d52698d0 0x5555d526bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fddab7dbb80 4 client.26456 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:18:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.689+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5555d5145b30 con 0x5555d52698d0 2023-10-06T22:18:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.693+0000 7fdd9539a700 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d52698d0 0x5555d526bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.712+0000 7fdd9539a700 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d52698d0 0x5555d526bcf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca4392d50bb2eaf server_cookie=c3e06bf576212ee0 in_seq=0 out_seq=0 2023-10-06T22:18:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.712+0000 7fdd797fa700 10 client.26456 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.712+0000 7fdd797fa700 1 -- 172.21.15.57:0/2800909307 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5555d5145b30 con 0x5555d52698d0 2023-10-06T22:18:37.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.712+0000 7fdd797fa700 10 client.26456 handle_command_reply: tid=0 2023-10-06T22:18:37.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.712+0000 7fddab7dbb80 1 client.26456 shutdown 2023-10-06T22:18:37.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.713+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 msgr2=0x5555d520b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.713+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d520b3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c011a50 tx=0x7fdd8c004750 comp rx=0 tx=0).stop 2023-10-06T22:18:37.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 shutdown_connections 2023-10-06T22:18:37.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5555d52698d0 0x5555d526bcf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5555d5204890 0x5555d5209c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5555d51f90c0 0x5555d520b3f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fddab7dbb80 1 --2- 172.21.15.57:0/2800909307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5555d51f99e0 0x5555d5209720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 >> 172.21.15.57:0/2800909307 conn(0x5555d516e260 msgr2=0x5555d5202e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:37.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fdd797fa700 0 client.26456 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:37.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.726+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 shutdown_connections 2023-10-06T22:18:37.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.742+0000 7fddab7dbb80 1 -- 172.21.15.57:0/2800909307 wait complete. 2023-10-06T22:18:37.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.742+0000 7fddab7dbb80 20 client.26456 trim_cache size 0 max 16384 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (3 inodes in the stack)", 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:18:37.746 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:18:37.747 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:18:37.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.754+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd7c0a1800 msgr2=0x7fdd7c0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.754+0000 7fdd97dff700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd7c0a1800 0x7fdd7c0a3cc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c005950 tx=0x7fdd8c00b410 comp rx=0 tx=0).stop 2023-10-06T22:18:37.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.754+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd9010f640 msgr2=0x7fdd90115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:37.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.754+0000 7fdd97dff700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd9010f640 0x7fdd90115f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000c920 tx=0x7fdd8000cc30 comp rx=0 tx=0).stop 2023-10-06T22:18:37.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.755+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 shutdown_connections 2023-10-06T22:18:37.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.755+0000 7fdd97dff700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fdd7c0a1800 0x7fdd7c0a3cc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.755+0000 7fdd97dff700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9011a690 0x7fdd901a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.755+0000 7fdd97dff700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd901083f0 0x7fdd90115a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.755+0000 7fdd97dff700 1 --2- 172.21.15.57:0/4039261123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd9010f640 0x7fdd90115f80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:37.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.755+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 >> 172.21.15.57:0/4039261123 conn(0x7fdd9006d1a0 msgr2=0x7fdd9011aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:37.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.757+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 shutdown_connections 2023-10-06T22:18:37.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:18:37.757+0000 7fdd97dff700 1 -- 172.21.15.57:0/4039261123 wait complete. 2023-10-06T22:18:37.808 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:18:39.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:39 smithi057 ceph-mon[98492]: pgmap v120: 97 pgs: 97 active+clean; 581 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 470 op/s 2023-10-06T22:18:39.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:39 smithi156 ceph-mon[134358]: pgmap v120: 97 pgs: 97 active+clean; 581 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 470 op/s 2023-10-06T22:18:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:39 smithi190 ceph-mon[134891]: pgmap v120: 97 pgs: 97 active+clean; 581 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 470 op/s 2023-10-06T22:18:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:41 smithi057 ceph-mon[98492]: pgmap v121: 97 pgs: 97 active+clean; 581 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.0 MiB/s wr, 438 op/s 2023-10-06T22:18:41.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:41 smithi156 ceph-mon[134358]: pgmap v121: 97 pgs: 97 active+clean; 581 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.0 MiB/s wr, 438 op/s 2023-10-06T22:18:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:41 smithi190 ceph-mon[134891]: pgmap v121: 97 pgs: 97 active+clean; 581 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.0 MiB/s wr, 438 op/s 2023-10-06T22:18:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 -- 172.21.15.57:0/339227569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01010d0 msgr2=0x7f2bf01014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/339227569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01010d0 0x7f2bf01014b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009990 tx=0x7f2be0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 -- 172.21.15.57:0/339227569 shutdown_connections 2023-10-06T22:18:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/339227569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf01061f0 0x7f2bf010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/339227569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01010d0 0x7f2bf01014b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/339227569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf0101a80 0x7f2bf0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 -- 172.21.15.57:0/339227569 >> 172.21.15.57:0/339227569 conn(0x7f2bf00fc950 msgr2=0x7f2bf00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 -- 172.21.15.57:0/339227569 shutdown_connections 2023-10-06T22:18:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.031+0000 7f2bf4dff700 1 -- 172.21.15.57:0/339227569 wait complete. 2023-10-06T22:18:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.032+0000 7f2bf4dff700 1 Processor -- start 2023-10-06T22:18:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bf4dff700 1 -- start start 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bf4dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf01010d0 0x7f2bf0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bf4dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01061f0 0x7f2bf00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bf4dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf01a3170 0x7f2bf01a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bf4dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf01a5aa0 con 0x7f2bf01010d0 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bf4dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf01a5be0 con 0x7f2bf01061f0 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bf4dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf01a5d50 con 0x7f2bf01a3170 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bedd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01061f0 0x7f2bf00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bedd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01061f0 0x7f2bf00738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:44420/0 (socket says 172.21.15.57:44420) 2023-10-06T22:18:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3899996838 learned_addr learned my addr 172.21.15.57:0/3899996838 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2beed9d700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf01a3170 0x7f2bf01a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.033+0000 7f2bee59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf01010d0 0x7f2bf0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3899996838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf01a3170 msgr2=0x7f2bf01a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf01a3170 0x7f2bf01a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf01010d0 msgr2=0x7f2bf0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf01010d0 0x7f2bf0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3899996838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be4009520 con 0x7f2bf01061f0 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf01010d0 0x7f2bf0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.034+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01061f0 0x7f2bf00738d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2be400ea30 tx=0x7f2be400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.035+0000 7f2bdf7fe700 1 -- 172.21.15.57:0/3899996838 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4009820 con 0x7f2bf01061f0 2023-10-06T22:18:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.035+0000 7f2bdf7fe700 1 -- 172.21.15.57:0/3899996838 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be40044e0 con 0x7f2bf01061f0 2023-10-06T22:18:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.035+0000 7f2bdf7fe700 1 -- 172.21.15.57:0/3899996838 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4003dc0 con 0x7f2bf01061f0 2023-10-06T22:18:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.035+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be00095f0 con 0x7f2bf01061f0 2023-10-06T22:18:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.035+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf01a62d0 con 0x7f2bf01061f0 2023-10-06T22:18:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.040+0000 7f2bdf7fe700 1 -- 172.21.15.57:0/3899996838 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2be4003f80 con 0x7f2bf01061f0 2023-10-06T22:18:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.041+0000 7f2c08729b80 1 Processor -- start 2023-10-06T22:18:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.041+0000 7f2c08729b80 1 -- start start 2023-10-06T22:18:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2c08729b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f417400 con 0x55df0f4adc00 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2c08729b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f408bc0 con 0x55df0f4b93d0 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2c08729b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f4064e0 con 0x55df0f4ae520 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2bee59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2beed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2bee59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56396/0 (socket says 172.21.15.57:56396) 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2bee59c700 1 -- 172.21.15.57:0/3611765553 learned_addr learned my addr 172.21.15.57:0/3611765553 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.042+0000 7f2bdf7fe700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2bd80a1670 0x7f2bd80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2bee59c700 1 -- 172.21.15.57:0/3611765553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 msgr2=0x55df0f4bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2bee59c700 1 -- 172.21.15.57:0/3611765553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 msgr2=0x55df0f4adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2bee59c700 1 -- 172.21.15.57:0/3611765553 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df0f22cd80 con 0x55df0f4ae520 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2bdf7fe700 1 -- 172.21.15.57:0/3899996838 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2be4014070 con 0x7f2bf01061f0 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2beed9d700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2bd80a1670 0x7f2bd80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.043+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b8e90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2be0005c00 tx=0x7f2be0005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc5268d73b84f09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.044+0000 7f2beed9d700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2bd80a1670 0x7f2bd80a3b30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2be80075c0 tx=0x7f2be8009040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.044+0000 7f2bdcff9700 1 -- 172.21.15.57:0/3611765553 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000bee0 con 0x55df0f4ae520 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.044+0000 7f2bdcff9700 1 -- 172.21.15.57:0/3611765553 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be001e040 con 0x55df0f4ae520 2023-10-06T22:18:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.044+0000 7f2bdcff9700 1 -- 172.21.15.57:0/3611765553 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0004c10 con 0x55df0f4ae520 2023-10-06T22:18:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 -- 172.21.15.57:0/3611765553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 msgr2=0x55df0f4b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b8e90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2be0005c00 tx=0x7f2be0005c30 comp rx=0 tx=0).stop 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 -- 172.21.15.57:0/3611765553 shutdown_connections 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3611765553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b8e90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 -- 172.21.15.57:0/3611765553 >> 172.21.15.57:0/3611765553 conn(0x55df0f423200 msgr2=0x55df0f4aa080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 -- 172.21.15.57:0/3611765553 shutdown_connections 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.045+0000 7f2c08729b80 1 -- 172.21.15.57:0/3611765553 wait complete. 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.046+0000 7f2c08729b80 1 -- start start 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.046+0000 7f2c08729b80 1 Processor -- start 2023-10-06T22:18:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.046+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.046+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2c08729b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f401dd0 con 0x55df0f4adc00 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2c08729b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f39e6b0 con 0x55df0f4b93d0 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2c08729b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f39e350 con 0x55df0f4ae520 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2c08729b80 10 client.0 _ll_register_callbacks cb 0x55df0f1fb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56400/0 (socket says 172.21.15.57:56400) 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3938174546 learned_addr learned my addr 172.21.15.57:0/3938174546 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2beed9d700 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3938174546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 msgr2=0x55df0f4b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3938174546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 msgr2=0x55df0f4b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4b8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.047+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3938174546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf010e5e0 con 0x55df0f4ae520 2023-10-06T22:18:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b60a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c5b0 tx=0x7f2be409a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc7bb6a24a47102 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2bd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:18:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3938174546 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400cbd0 con 0x55df0f4ae520 2023-10-06T22:18:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3938174546 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2be409aae0 con 0x55df0f4ae520 2023-10-06T22:18:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df0f4a1d50 con 0x55df0f4ae520 2023-10-06T22:18:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3938174546 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2be40b2ea0 con 0x55df0f4ae520 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2c08729b80 10 client.26446 fetch_fsmap learned FSMap version 36 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.048+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55df0f215c00 con 0x55df0f4ae520 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3938174546 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be40b6020 con 0x55df0f4ae520 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3938174546 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f2be40b1070 con 0x55df0f4ae520 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2c08729b80 10 client.26446 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2c08729b80 10 client.26446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2c08729b80 20 client.26446 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f4c8da0 0x55df0f4cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2c08729b80 4 client.26446 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df0f3fadb0 con 0x55df0f4c8da0 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.049+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f4c8da0 0x55df0f4cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.050+0000 7f2bd5ffb700 10 client.26446 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.050+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f4c8da0 0x55df0f4cb1c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a780d9dadabc81aa server_cookie=5dab9af1e3e813e in_seq=0 out_seq=0 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3938174546 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df0f3fadb0 con 0x55df0f4c8da0 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2bd5ffb700 10 client.26446 handle_command_reply: tid=0 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 client.26446 shutdown 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 msgr2=0x55df0f4b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b60a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c5b0 tx=0x7f2be409a230 comp rx=0 tx=0).stop 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 shutdown_connections 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f4c8da0 0x55df0f4cb1c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3938174546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4b60a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2bd5ffb700 0 client.26446 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 >> 172.21.15.57:0/3938174546 conn(0x55df0f423200 msgr2=0x55df0f4a9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 shutdown_connections 2023-10-06T22:18:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.051+0000 7f2c08729b80 1 -- 172.21.15.57:0/3938174546 wait complete. 2023-10-06T22:18:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.052+0000 7f2c08729b80 20 client.26446 trim_cache size 0 max 16384 2023-10-06T22:18:42.057 INFO:tasks.workunit.client.0.smithi057.stdout: 180 850966 522 578883 672 147586 213 2023-10-06T22:18:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 Processor -- start 2023-10-06T22:18:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 -- start start 2023-10-06T22:18:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4adc00 0x55df0f4b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4ae520 0x55df0f4c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f2828c0 con 0x55df0f4ae520 2023-10-06T22:18:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f4b6fd0 con 0x55df0f4b93d0 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.071+0000 7f2c08729b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f4af690 con 0x55df0f4adc00 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4adc00 0x55df0f4b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4adc00 0x55df0f4b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56416/0 (socket says 172.21.15.57:56416) 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 -- 172.21.15.57:0/3132972760 learned_addr learned my addr 172.21.15.57:0/3132972760 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4ae520 0x55df0f4c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2beed9d700 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 -- 172.21.15.57:0/3132972760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 msgr2=0x55df0f4b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 -- 172.21.15.57:0/3132972760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4ae520 msgr2=0x55df0f4c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4ae520 0x55df0f4c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.072+0000 7f2bee59c700 1 -- 172.21.15.57:0/3132972760 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df0f2531f0 con 0x55df0f4adc00 2023-10-06T22:18:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.073+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4adc00 0x55df0f4b0b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2be000bb30 tx=0x7f2be0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3fc93c426b3634 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.073+0000 7f2bdd7fa700 1 -- 172.21.15.57:0/3132972760 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0032070 con 0x55df0f4adc00 2023-10-06T22:18:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.073+0000 7f2bdd7fa700 1 -- 172.21.15.57:0/3132972760 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be0004640 con 0x55df0f4adc00 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.073+0000 7f2bdd7fa700 1 -- 172.21.15.57:0/3132972760 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000fe30 con 0x55df0f4adc00 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.074+0000 7f2c08729b80 1 -- 172.21.15.57:0/3132972760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4adc00 msgr2=0x55df0f4b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.074+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4adc00 0x55df0f4b0b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2be000bb30 tx=0x7f2be0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.074+0000 7f2c08729b80 1 -- 172.21.15.57:0/3132972760 shutdown_connections 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.074+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.074+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4adc00 0x55df0f4b0b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.074+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3132972760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4ae520 0x55df0f4c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.074+0000 7f2c08729b80 1 -- 172.21.15.57:0/3132972760 >> 172.21.15.57:0/3132972760 conn(0x55df0f423200 msgr2=0x55df0f4c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.075+0000 7f2c08729b80 1 -- 172.21.15.57:0/3132972760 shutdown_connections 2023-10-06T22:18:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.075+0000 7f2c08729b80 1 -- 172.21.15.57:0/3132972760 wait complete. 2023-10-06T22:18:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.075+0000 7f2c08729b80 1 -- start start 2023-10-06T22:18:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 1 Processor -- start 2023-10-06T22:18:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f524550 con 0x55df0f4adc00 2023-10-06T22:18:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f5246c0 con 0x55df0f4b93d0 2023-10-06T22:18:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0f524880 con 0x55df0f4ae520 2023-10-06T22:18:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.076+0000 7f2c08729b80 10 client.0 _ll_register_callbacks cb 0x7f2be4096720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4bdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56432/0 (socket says 172.21.15.57:56432) 2023-10-06T22:18:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3609487603 learned_addr learned my addr 172.21.15.57:0/3609487603 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2beed9d700 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3609487603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 msgr2=0x55df0f4be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3609487603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 msgr2=0x55df0f4bd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.077+0000 7f2bedd9b700 1 -- 172.21.15.57:0/3609487603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df0f22cd80 con 0x55df0f4ae520 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.078+0000 7f2bedd9b700 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4bdc50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2be409a370 tx=0x7f2be4096110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d11d3ecc69b520b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.078+0000 7f2bd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.078+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3609487603 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be40b2ea0 con 0x55df0f4ae520 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.078+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3609487603 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f2be40b7080 con 0x55df0f4ae520 2023-10-06T22:18:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.078+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2be001e040 con 0x55df0f4ae520 2023-10-06T22:18:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.078+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3609487603 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2be4004230 con 0x55df0f4ae520 2023-10-06T22:18:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 10 client.26452 fetch_fsmap learned FSMap version 36 2023-10-06T22:18:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55df0f22b250 con 0x55df0f4ae520 2023-10-06T22:18:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3609487603 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be001e040 con 0x55df0f4ae520 2023-10-06T22:18:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3609487603 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f2be40b1070 con 0x55df0f4ae520 2023-10-06T22:18:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 10 client.26452 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T22:18:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 10 client.26452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 20 client.26452 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f525ac0 0x55df0f527eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 4 client.26452 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:18:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df0f3faad0 con 0x55df0f525ac0 2023-10-06T22:18:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.079+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f525ac0 0x55df0f527eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.080+0000 7f2bee59c700 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f525ac0 0x55df0f527eb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b95d5882598c9b server_cookie=3c0a41a29a8e2301 in_seq=0 out_seq=0 2023-10-06T22:18:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.080+0000 7f2bd5ffb700 10 client.26452 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.699+0000 7f2bd5ffb700 1 -- 172.21.15.57:0/3609487603 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df0f3faad0 con 0x55df0f525ac0 2023-10-06T22:18:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.699+0000 7f2bd5ffb700 10 client.26452 handle_command_reply: tid=0 2023-10-06T22:18:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.700+0000 7f2c08729b80 1 client.26452 shutdown 2023-10-06T22:18:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.701+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 msgr2=0x55df0f4bdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.701+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4bdc50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2be409a370 tx=0x7f2be4096110 comp rx=0 tx=0).stop 2023-10-06T22:18:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.710+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 shutdown_connections 2023-10-06T22:18:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.710+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55df0f525ac0 0x55df0f527eb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.710+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55df0f4b93d0 0x55df0f4be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.710+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df0f4adc00 0x55df0f4bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.710+0000 7f2c08729b80 1 --2- 172.21.15.57:0/3609487603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55df0f4ae520 0x55df0f4bdc50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.710+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 >> 172.21.15.57:0/3609487603 conn(0x55df0f423200 msgr2=0x55df0f4c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.718+0000 7f2bd5ffb700 0 client.26452 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.722+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 shutdown_connections 2023-10-06T22:18:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.723+0000 7f2c08729b80 1 -- 172.21.15.57:0/3609487603 wait complete. 2023-10-06T22:18:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.723+0000 7f2c08729b80 20 client.26452 trim_cache size 0 max 16384 2023-10-06T22:18:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:18:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:18:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:18:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:18:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.775+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2bd80a1670 msgr2=0x7f2bd80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.775+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2bd80a1670 0x7f2bd80a3b30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2be80075c0 tx=0x7f2be8009040 comp rx=0 tx=0).stop 2023-10-06T22:18:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.775+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01061f0 msgr2=0x7f2bf00738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.775+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01061f0 0x7f2bf00738d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2be400ea30 tx=0x7f2be400ed40 comp rx=0 tx=0).stop 2023-10-06T22:18:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.788+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 shutdown_connections 2023-10-06T22:18:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.788+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2bd80a1670 0x7f2bd80a3b30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.788+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf01a3170 0x7f2bf01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.788+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf01010d0 0x7f2bf0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.788+0000 7f2bf4dff700 1 --2- 172.21.15.57:0/3899996838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bf01061f0 0x7f2bf00738d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.788+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 >> 172.21.15.57:0/3899996838 conn(0x7f2bf00fc950 msgr2=0x7f2bf0104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.790+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 shutdown_connections 2023-10-06T22:18:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:42.790+0000 7f2bf4dff700 1 -- 172.21.15.57:0/3899996838 wait complete. 2023-10-06T22:18:43.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:43 smithi156 ceph-mon[134358]: pgmap v122: 97 pgs: 97 active+clean; 612 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 587 op/s 2023-10-06T22:18:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:43 smithi190 ceph-mon[134891]: pgmap v122: 97 pgs: 97 active+clean; 612 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 587 op/s 2023-10-06T22:18:43.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:43 smithi057 ceph-mon[98492]: pgmap v122: 97 pgs: 97 active+clean; 612 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 587 op/s 2023-10-06T22:18:45.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:45 smithi156 ceph-mon[134358]: pgmap v123: 97 pgs: 97 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-10-06T22:18:45.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:45 smithi190 ceph-mon[134891]: pgmap v123: 97 pgs: 97 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-10-06T22:18:45.719 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:45 smithi057 ceph-mon[98492]: pgmap v123: 97 pgs: 97 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2023-10-06T22:18:46.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:46 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:46.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:46 smithi190 ceph-mon[134891]: pgmap v124: 97 pgs: 97 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.5 MiB/s wr, 201 op/s 2023-10-06T22:18:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:46 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:46 smithi057 ceph-mon[98492]: pgmap v124: 97 pgs: 97 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.5 MiB/s wr, 201 op/s 2023-10-06T22:18:46.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:46 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:46.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:46 smithi156 ceph-mon[134358]: pgmap v124: 97 pgs: 97 active+clean; 617 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.5 MiB/s wr, 201 op/s 2023-10-06T22:18:49.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:48 smithi156 ceph-mon[134358]: pgmap v125: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 638 op/s 2023-10-06T22:18:49.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:48 smithi190 ceph-mon[134891]: pgmap v125: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 638 op/s 2023-10-06T22:18:49.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:48 smithi057 ceph-mon[98492]: pgmap v125: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 638 op/s 2023-10-06T22:18:51.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:51 smithi190 ceph-mon[134891]: pgmap v126: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 601 op/s 2023-10-06T22:18:51.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:51 smithi057 ceph-mon[98492]: pgmap v126: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 601 op/s 2023-10-06T22:18:51.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:51 smithi156 ceph-mon[134358]: pgmap v126: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 601 op/s 2023-10-06T22:18:52.061 INFO:tasks.workunit.client.0.smithi057.stdout: 192 801560 576 548027 552 144847 655 2023-10-06T22:18:52.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:52 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:52.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:52 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:52.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:52 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:53.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:53 smithi057 ceph-mon[98492]: pgmap v127: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 670 op/s 2023-10-06T22:18:53.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:53 smithi156 ceph-mon[134358]: pgmap v127: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 670 op/s 2023-10-06T22:18:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:53 smithi190 ceph-mon[134891]: pgmap v127: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 670 op/s 2023-10-06T22:18:55.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:55 smithi057 ceph-mon[98492]: pgmap v128: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 524 op/s 2023-10-06T22:18:55.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:55 smithi156 ceph-mon[134358]: pgmap v128: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 524 op/s 2023-10-06T22:18:55.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:55 smithi190 ceph-mon[134891]: pgmap v128: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 524 op/s 2023-10-06T22:18:56.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:56 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:18:56.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:56 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:18:56.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:56 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:18:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.875+0000 7f4d8fdff700 1 -- 172.21.15.57:0/4190337584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 msgr2=0x7f4d8810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.875+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/4190337584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 0x7f4d8810dd70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000b0a0 tx=0x7f4d8000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.875+0000 7f4d8fdff700 1 -- 172.21.15.57:0/4190337584 shutdown_connections 2023-10-06T22:18:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.875+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/4190337584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 0x7f4d8810dd70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.875+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/4190337584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d881010d0 0x7f4d881014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.875+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/4190337584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d88101a80 0x7f4d88105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.875+0000 7f4d8fdff700 1 -- 172.21.15.57:0/4190337584 >> 172.21.15.57:0/4190337584 conn(0x7f4d880fc950 msgr2=0x7f4d880fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.876+0000 7f4d8fdff700 1 -- 172.21.15.57:0/4190337584 shutdown_connections 2023-10-06T22:18:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.876+0000 7f4d8fdff700 1 -- 172.21.15.57:0/4190337584 wait complete. 2023-10-06T22:18:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.877+0000 7f4d8fdff700 1 Processor -- start 2023-10-06T22:18:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.877+0000 7f4d8fdff700 1 -- start start 2023-10-06T22:18:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8fdff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d881010d0 0x7f4d88106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8fdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 0x7f4d88107230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d88107770 0x7f4d881a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8fdff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d881a9f50 con 0x7f4d881010d0 2023-10-06T22:18:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88199070 con 0x7f4d88107770 2023-10-06T22:18:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8fdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88199230 con 0x7f4d881061f0 2023-10-06T22:18:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8e39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d88107770 0x7f4d881a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8e39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d88107770 0x7f4d881a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46962/0 (socket says 172.21.15.57:46962) 2023-10-06T22:18:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8e39c700 1 -- 172.21.15.57:0/2618715719 learned_addr learned my addr 172.21.15.57:0/2618715719 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.878+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 0x7f4d88107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.879+0000 7f4d8e39c700 1 -- 172.21.15.57:0/2618715719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 msgr2=0x7f4d88107230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.879+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 0x7f4d88107230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.879+0000 7f4d8e39c700 1 -- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d881010d0 msgr2=0x7f4d88106cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:18:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.879+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d881010d0 0x7f4d88106cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.879+0000 7f4d8e39c700 1 -- 172.21.15.57:0/2618715719 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d84009520 con 0x7f4d88107770 2023-10-06T22:18:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.879+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d88107770 0x7f4d881a98a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4d800062a0 tx=0x7f4d8000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.879+0000 7f4d7effd700 1 -- 172.21.15.57:0/2618715719 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000e040 con 0x7f4d88107770 2023-10-06T22:18:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.880+0000 7f4d7effd700 1 -- 172.21.15.57:0/2618715719 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d80003e10 con 0x7f4d88107770 2023-10-06T22:18:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.880+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d78009520 con 0x7f4d88107770 2023-10-06T22:18:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.880+0000 7f4d7effd700 1 -- 172.21.15.57:0/2618715719 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d800052a0 con 0x7f4d88107770 2023-10-06T22:18:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.880+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d80009d00 con 0x7f4d88107770 2023-10-06T22:18:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.885+0000 7f4d7effd700 1 -- 172.21.15.57:0/2618715719 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4d8001d480 con 0x7f4d88107770 2023-10-06T22:18:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.886+0000 7f4da3759b80 1 Processor -- start 2023-10-06T22:18:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4da3759b80 1 -- start start 2023-10-06T22:18:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411dc00 0x56408411dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411e520 0x564084128e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x56408412b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4d7effd700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4d740a1630 0x7f4d740a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4da3759b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564084087400 con 0x56408411e520 2023-10-06T22:18:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4da3759b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564084078bc0 con 0x56408411dc00 2023-10-06T22:18:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.887+0000 7f4da3759b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640840764e0 con 0x5640841293d0 2023-10-06T22:18:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8e39c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x56408412b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8e39c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x56408412b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44486/0 (socket says 172.21.15.57:44486) 2023-10-06T22:18:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8e39c700 1 -- 172.21.15.57:0/232815712 learned_addr learned my addr 172.21.15.57:0/232815712 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411dc00 0x56408411dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d7effd700 1 -- 172.21.15.57:0/2618715719 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4d800ce450 con 0x7f4d88107770 2023-10-06T22:18:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4d740a1630 0x7f4d740a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8e39c700 1 -- 172.21.15.57:0/232815712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411dc00 msgr2=0x56408411dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411dc00 0x56408411dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411e520 0x564084128e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.888+0000 7f4d8e39c700 1 -- 172.21.15.57:0/232815712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411e520 msgr2=0x564084128e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.889+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411e520 0x564084128e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.889+0000 7f4d8e39c700 1 -- 172.21.15.57:0/232815712 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564083e9cd80 con 0x5640841293d0 2023-10-06T22:18:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.889+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411e520 0x564084128e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:18:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.889+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x56408412b7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4d80097a10 tx=0x7f4d8001db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5cf112963a5a81e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.889+0000 7f4d73fff700 1 -- 172.21.15.57:0/232815712 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d80011390 con 0x5640841293d0 2023-10-06T22:18:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.889+0000 7f4d73fff700 1 -- 172.21.15.57:0/232815712 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d8009bc10 con 0x5640841293d0 2023-10-06T22:18:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.889+0000 7f4d73fff700 1 -- 172.21.15.57:0/232815712 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8008a3a0 con 0x5640841293d0 2023-10-06T22:18:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4da3759b80 1 -- 172.21.15.57:0/232815712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 msgr2=0x56408412b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4da3759b80 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x56408412b7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4d80097a10 tx=0x7f4d8001db90 comp rx=0 tx=0).stop 2023-10-06T22:18:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4da3759b80 1 -- 172.21.15.57:0/232815712 shutdown_connections 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4da3759b80 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x56408412b7c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4da3759b80 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411dc00 0x56408411dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4d740a1630 0x7f4d740a3af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009ca0 tx=0x7f4d84009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4da3759b80 1 --2- 172.21.15.57:0/232815712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411e520 0x564084128e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.890+0000 7f4da3759b80 1 -- 172.21.15.57:0/232815712 >> 172.21.15.57:0/232815712 conn(0x564084093200 msgr2=0x56408411a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.891+0000 7f4da3759b80 1 -- 172.21.15.57:0/232815712 shutdown_connections 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.891+0000 7f4da3759b80 1 -- 172.21.15.57:0/232815712 wait complete. 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.891+0000 7f4da3759b80 1 -- start start 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.892+0000 7f4da3759b80 1 Processor -- start 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x564084128d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x564084126070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x5640841266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4da3759b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564084071dd0 con 0x56408411dc00 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4da3759b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408400e6b0 con 0x56408411e520 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4da3759b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408400e350 con 0x5640841293d0 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4da3759b80 10 client.0 _ll_register_callbacks cb 0x564083e6b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4d8d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x564084126070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4d8d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x564084126070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46986/0 (socket says 172.21.15.57:46986) 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.893+0000 7f4d8d39a700 1 -- 172.21.15.57:0/3219832943 learned_addr learned my addr 172.21.15.57:0/3219832943 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.894+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x564084128d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.894+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x5640841266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.894+0000 7f4d8db9b700 1 -- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 msgr2=0x5640841266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.894+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x5640841266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.894+0000 7f4d8db9b700 1 -- 172.21.15.57:0/3219832943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 msgr2=0x564084126070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.894+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x564084126070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.894+0000 7f4d8db9b700 1 -- 172.21.15.57:0/3219832943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d80009d00 con 0x56408411dc00 2023-10-06T22:18:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.895+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x564084128d50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4d840106b0 tx=0x7f4d840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca10a5876d878f6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.897+0000 7f4d717fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:18:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.897+0000 7f4d717fa700 1 -- 172.21.15.57:0/3219832943 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84004a30 con 0x56408411dc00 2023-10-06T22:18:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.897+0000 7f4d717fa700 1 -- 172.21.15.57:0/3219832943 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4d840098d0 con 0x56408411dc00 2023-10-06T22:18:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.897+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564084111d50 con 0x56408411dc00 2023-10-06T22:18:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.898+0000 7f4d717fa700 1 -- 172.21.15.57:0/3219832943 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d84010e80 con 0x56408411dc00 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.898+0000 7f4da3759b80 10 client.16458 fetch_fsmap learned FSMap version 39 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.898+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564083e85c00 con 0x56408411dc00 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.898+0000 7f4d717fa700 1 -- 172.21.15.57:0/3219832943 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564084111d50 con 0x56408411dc00 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4d717fa700 1 -- 172.21.15.57:0/3219832943 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 39) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4d84018070 con 0x56408411dc00 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4da3759b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4da3759b80 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4da3759b80 20 client.16458 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4da3759b80 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564084138b80 0x56408413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4da3759b80 4 client.16458 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:18:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56408406adb0 con 0x564084138b80 2023-10-06T22:18:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.899+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564084138b80 0x56408413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.900+0000 7f4d717fa700 10 client.16458 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.901+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564084138b80 0x56408413afa0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48be13bc4b49c53 server_cookie=128256278bc3728e in_seq=0 out_seq=0 2023-10-06T22:18:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4d717fa700 1 -- 172.21.15.57:0/3219832943 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56408406adb0 con 0x564084138b80 2023-10-06T22:18:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4d717fa700 10 client.16458 handle_command_reply: tid=0 2023-10-06T22:18:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 client.16458 shutdown 2023-10-06T22:18:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 msgr2=0x564084128d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x564084128d50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4d840106b0 tx=0x7f4d840047c0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 shutdown_connections 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x564084138b80 0x56408413afa0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640841293d0 0x5640841266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x564084128d50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 --2- 172.21.15.57:0/3219832943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x564084126070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 >> 172.21.15.57:0/3219832943 conn(0x564084093200 msgr2=0x564084119c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.903+0000 7f4d717fa700 0 client.16458 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.904+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 shutdown_connections 2023-10-06T22:18:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.904+0000 7f4da3759b80 1 -- 172.21.15.57:0/3219832943 wait complete. 2023-10-06T22:18:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.904+0000 7f4da3759b80 20 client.16458 trim_cache size 0 max 16384 2023-10-06T22:18:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 Processor -- start 2023-10-06T22:18:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 -- start start 2023-10-06T22:18:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411dc00 0x564084128880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x5640841213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5640841293d0 0x564084121930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564083ef28c0 con 0x5640841293d0 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564084111c40 con 0x56408411e520 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.925+0000 7f4da3759b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564084114200 con 0x56408411dc00 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8db9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411dc00 0x564084128880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x5640841213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8db9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411dc00 0x564084128880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44508/0 (socket says 172.21.15.57:44508) 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8db9b700 1 -- 172.21.15.57:0/4091536280 learned_addr learned my addr 172.21.15.57:0/4091536280 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5640841293d0 0x564084121930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8d39a700 1 -- 172.21.15.57:0/4091536280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411dc00 msgr2=0x564084128880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411dc00 0x564084128880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8d39a700 1 -- 172.21.15.57:0/4091536280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5640841293d0 msgr2=0x564084121930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5640841293d0 0x564084121930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8d39a700 1 -- 172.21.15.57:0/4091536280 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564083ec31f0 con 0x56408411e520 2023-10-06T22:18:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.926+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x5640841213f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56408413b870 tx=0x7f4d780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22e73f706c8527b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4d7cbff700 1 -- 172.21.15.57:0/4091536280 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78029040 con 0x56408411e520 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4d7cbff700 1 -- 172.21.15.57:0/4091536280 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d7802b040 con 0x56408411e520 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4d7cbff700 1 -- 172.21.15.57:0/4091536280 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78024570 con 0x56408411e520 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4da3759b80 1 -- 172.21.15.57:0/4091536280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 msgr2=0x5640841213f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4da3759b80 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x5640841213f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56408413b870 tx=0x7f4d780047c0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4da3759b80 1 -- 172.21.15.57:0/4091536280 shutdown_connections 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4da3759b80 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5640841293d0 0x564084121930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4da3759b80 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411dc00 0x564084128880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4da3759b80 1 --2- 172.21.15.57:0/4091536280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56408411e520 0x5640841213f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.927+0000 7f4da3759b80 1 -- 172.21.15.57:0/4091536280 >> 172.21.15.57:0/4091536280 conn(0x564084093200 msgr2=0x56408413ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.928+0000 7f4da3759b80 1 -- 172.21.15.57:0/4091536280 shutdown_connections 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.928+0000 7f4da3759b80 1 -- 172.21.15.57:0/4091536280 wait complete. 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.928+0000 7f4da3759b80 1 -- start start 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.928+0000 7f4da3759b80 1 Processor -- start 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x56408412dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411e520 0x56408412e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4da3759b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640841293d0 0x56408412e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4da3759b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408418ba90 con 0x56408411dc00 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4da3759b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408418bc00 con 0x5640841293d0 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4da3759b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408418bdc0 con 0x56408411e520 2023-10-06T22:18:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4da3759b80 10 client.0 _ll_register_callbacks cb 0x7f4d8400eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4d8db9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x56408412dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4d8d39a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411e520 0x56408412e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4d8d39a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411e520 0x56408412e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44512/0 (socket says 172.21.15.57:44512) 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4d8db9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x56408412dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57998/0 (socket says 172.21.15.57:57998) 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4d8d39a700 1 -- 172.21.15.57:0/1836639089 learned_addr learned my addr 172.21.15.57:0/1836639089 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.929+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640841293d0 0x56408412e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8d39a700 1 -- 172.21.15.57:0/1836639089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640841293d0 msgr2=0x56408412e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640841293d0 0x56408412e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8d39a700 1 -- 172.21.15.57:0/1836639089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 msgr2=0x56408412dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:18:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x56408412dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8d39a700 1 -- 172.21.15.57:0/1836639089 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d78009e50 con 0x56408411e520 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8e39c700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640841293d0 0x56408412e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x56408412dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d8d39a700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411e520 0x56408412e2e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d78032040 tx=0x7f4d78004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63463d7723c35779 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.930+0000 7f4d717fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4d717fa700 1 -- 172.21.15.57:0/1836639089 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78029070 con 0x56408411e520 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4d717fa700 1 -- 172.21.15.57:0/1836639089 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4d78000e40 con 0x56408411e520 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d8009bc10 con 0x56408411e520 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4d717fa700 1 -- 172.21.15.57:0/1836639089 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d78030d80 con 0x56408411e520 2023-10-06T22:18:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4da3759b80 10 client.26470 fetch_fsmap learned FSMap version 39 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564083e9b250 con 0x56408411e520 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4d717fa700 1 -- 172.21.15.57:0/1836639089 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d8009bc10 con 0x56408411e520 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4d717fa700 1 -- 172.21.15.57:0/1836639089 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4d78024b10 con 0x56408411e520 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4da3759b80 10 client.26470 fetch_fsmap finished waiting for FSMap version 39 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4da3759b80 10 client.26470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.931+0000 7f4da3759b80 20 client.26470 populate_metadata read hostname 'smithi057' 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.932+0000 7f4da3759b80 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56408418d4c0 0x56408418f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.932+0000 7f4da3759b80 4 client.26470 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.932+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56408406aad0 con 0x56408418d4c0 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.932+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56408418d4c0 0x56408418f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:18:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.932+0000 7f4d8db9b700 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56408418d4c0 0x56408418f8e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9160b4b2826df3c3 server_cookie=f94ac67bd332a5b1 in_seq=0 out_seq=0 2023-10-06T22:18:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:56.932+0000 7f4d717fa700 10 client.26470 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:57 smithi156 ceph-mon[134358]: pgmap v129: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.8 MiB/s wr, 508 op/s 2023-10-06T22:18:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:57 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:57 smithi190 ceph-mon[134891]: pgmap v129: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.8 MiB/s wr, 508 op/s 2023-10-06T22:18:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:57 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:57 smithi057 ceph-mon[98492]: pgmap v129: 97 pgs: 97 active+clean; 714 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.8 MiB/s wr, 508 op/s 2023-10-06T22:18:57.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:57 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.720+0000 7f4d717fa700 1 -- 172.21.15.57:0/1836639089 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56408406aad0 con 0x56408418d4c0 2023-10-06T22:18:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.720+0000 7f4d717fa700 10 client.26470 handle_command_reply: tid=0 2023-10-06T22:18:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.721+0000 7f4da3759b80 1 client.26470 shutdown 2023-10-06T22:18:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.778+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411e520 msgr2=0x56408412e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.778+0000 7f4da3759b80 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411e520 0x56408412e2e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d78032040 tx=0x7f4d78004600 comp rx=0 tx=0).stop 2023-10-06T22:18:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.784+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 shutdown_connections 2023-10-06T22:18:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.784+0000 7f4da3759b80 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56408418d4c0 0x56408418f8e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.784+0000 7f4da3759b80 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640841293d0 0x56408412e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.784+0000 7f4da3759b80 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56408411dc00 0x56408412dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.784+0000 7f4da3759b80 1 --2- 172.21.15.57:0/1836639089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56408411e520 0x56408412e2e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.784+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 >> 172.21.15.57:0/1836639089 conn(0x564084093200 msgr2=0x5640841213f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.784+0000 7f4d717fa700 0 client.26470 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:18:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.785+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 shutdown_connections 2023-10-06T22:18:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.785+0000 7f4da3759b80 1 -- 172.21.15.57:0/1836639089 wait complete. 2023-10-06T22:18:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.796+0000 7f4da3759b80 20 client.26470 trim_cache size 0 max 16384 2023-10-06T22:18:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:18:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:18:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:18:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:18:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.830+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4d740a1630 msgr2=0x7f4d740a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.830+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4d740a1630 0x7f4d740a3af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009ca0 tx=0x7f4d84009260 comp rx=0 tx=0).stop 2023-10-06T22:18:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.830+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d88107770 msgr2=0x7f4d881a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:18:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.830+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d88107770 0x7f4d881a98a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4d800062a0 tx=0x7f4d8000bc80 comp rx=0 tx=0).stop 2023-10-06T22:18:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.832+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 shutdown_connections 2023-10-06T22:18:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.832+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4d740a1630 0x7f4d740a3af0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.832+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d88107770 0x7f4d881a98a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.832+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d881010d0 0x7f4d88106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.832+0000 7f4d8fdff700 1 --2- 172.21.15.57:0/2618715719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d881061f0 0x7f4d88107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:18:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.832+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 >> 172.21.15.57:0/2618715719 conn(0x7f4d880fc950 msgr2=0x7f4d88102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:18:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.833+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 shutdown_connections 2023-10-06T22:18:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:18:58.833+0000 7f4d8fdff700 1 -- 172.21.15.57:0/2618715719 wait complete. 2023-10-06T22:18:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:59 smithi190 ceph-mon[134891]: pgmap v130: 97 pgs: 97 active+clean; 720 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.4 MiB/s wr, 572 op/s 2023-10-06T22:18:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:18:59 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:59.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:59 smithi057 ceph-mon[98492]: pgmap v130: 97 pgs: 97 active+clean; 720 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.4 MiB/s wr, 572 op/s 2023-10-06T22:18:59.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:18:59 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:18:59.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:59 smithi156 ceph-mon[134358]: pgmap v130: 97 pgs: 97 active+clean; 720 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.4 MiB/s wr, 572 op/s 2023-10-06T22:18:59.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:18:59 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:19:00.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:00 smithi057 ceph-mon[98492]: pgmap v131: 97 pgs: 97 active+clean; 720 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-10-06T22:19:00.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:00 smithi156 ceph-mon[134358]: pgmap v131: 97 pgs: 97 active+clean; 720 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-10-06T22:19:00.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:00 smithi190 ceph-mon[134891]: pgmap v131: 97 pgs: 97 active+clean; 720 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-10-06T22:19:02.070 INFO:tasks.workunit.client.0.smithi057.stdout: 202 812157 817 558493 486 149946 293 2023-10-06T22:19:03.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:02 smithi156 ceph-mon[134358]: pgmap v132: 97 pgs: 97 active+clean; 723 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 501 op/s 2023-10-06T22:19:03.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:02 smithi190 ceph-mon[134891]: pgmap v132: 97 pgs: 97 active+clean; 723 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 501 op/s 2023-10-06T22:19:03.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:02 smithi057 ceph-mon[98492]: pgmap v132: 97 pgs: 97 active+clean; 723 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 501 op/s 2023-10-06T22:19:05.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:04 smithi156 ceph-mon[134358]: pgmap v133: 97 pgs: 97 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 469 op/s 2023-10-06T22:19:05.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:04 smithi190 ceph-mon[134891]: pgmap v133: 97 pgs: 97 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 469 op/s 2023-10-06T22:19:05.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:04 smithi057 ceph-mon[98492]: pgmap v133: 97 pgs: 97 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 469 op/s 2023-10-06T22:19:06.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:06 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:06.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:06 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:06.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:06 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:06.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:06 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:19:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:06 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:06 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:06 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:06.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:06 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:19:06.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:06 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:06.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:06 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:06.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:06 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:06.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:06 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:19:07.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:07.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:07.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:07.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:07.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:07.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: pgmap v134: 97 pgs: 97 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2023-10-06T22:19:07.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:19:07.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:19:07.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:07 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:19:07.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:07.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: pgmap v134: 97 pgs: 97 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:19:07.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:19:07.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:07 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:19:07.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:07.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:19:07.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: pgmap v134: 97 pgs: 97 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2023-10-06T22:19:07.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:19:07.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:19:07.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:07 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:19:07.810 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:19:08.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 -- 172.21.15.57:0/4093372010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f500010d570 msgr2=0x7f500010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 --2- 172.21.15.57:0/4093372010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f500010d570 0x7f500010d950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0099c0 tx=0x7f4ffc009cd0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 -- 172.21.15.57:0/4093372010 shutdown_connections 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 --2- 172.21.15.57:0/4093372010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5000107f90 0x7f5000114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 --2- 172.21.15.57:0/4093372010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f500010d570 0x7f500010d950 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 --2- 172.21.15.57:0/4093372010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50001075d0 0x7f5000107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 -- 172.21.15.57:0/4093372010 >> 172.21.15.57:0/4093372010 conn(0x7f5000076d50 msgr2=0x7f5000105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 -- 172.21.15.57:0/4093372010 shutdown_connections 2023-10-06T22:19:08.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.057+0000 7f5007bff700 1 -- 172.21.15.57:0/4093372010 wait complete. 2023-10-06T22:19:08.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.059+0000 7f5007bff700 1 Processor -- start 2023-10-06T22:19:08.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.059+0000 7f5007bff700 1 -- start start 2023-10-06T22:19:08.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f5007bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50001075d0 0x7f500019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f5007bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5000107f90 0x7f500019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f500599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50001075d0 0x7f500019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f500599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50001075d0 0x7f500019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:48352/0 (socket says 172.21.15.57:48352) 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f5007bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500010d570 0x7f50001a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001a2550 con 0x7f5000107f90 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f500599b700 1 -- 172.21.15.57:0/2557531744 learned_addr learned my addr 172.21.15.57:0/2557531744 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001a26c0 con 0x7f50001075d0 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001a2880 con 0x7f500010d570 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f500519a700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5000107f90 0x7f500019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f500619c700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500010d570 0x7f50001a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f500599b700 1 -- 172.21.15.57:0/2557531744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500010d570 msgr2=0x7f50001a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.060+0000 7f500599b700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500010d570 0x7f50001a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f500599b700 1 -- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5000107f90 msgr2=0x7f500019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f500599b700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5000107f90 0x7f500019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f500599b700 1 -- 172.21.15.57:0/2557531744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0009520 con 0x7f50001075d0 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f500619c700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500010d570 0x7f50001a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f500599b700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50001075d0 0x7f500019d570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098e0 tx=0x7f4ffc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f4ff6ffd700 1 -- 172.21.15.57:0/2557531744 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc01d070 con 0x7f50001075d0 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc0095f0 con 0x7f50001075d0 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.061+0000 7f4ff6ffd700 1 -- 172.21.15.57:0/2557531744 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ffc00fb80 con 0x7f50001075d0 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.062+0000 7f4ff6ffd700 1 -- 172.21.15.57:0/2557531744 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0173f0 con 0x7f50001075d0 2023-10-06T22:19:08.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.062+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50001a2d90 con 0x7f50001075d0 2023-10-06T22:19:08.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.065+0000 7f4ff6ffd700 1 -- 172.21.15.57:0/2557531744 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4ffc00fcf0 con 0x7f50001075d0 2023-10-06T22:19:08.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.066+0000 7f501b4c1b80 1 Processor -- start 2023-10-06T22:19:08.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.067+0000 7f501b4c1b80 1 -- start start 2023-10-06T22:19:08.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.067+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d0c0 0x55e1b432d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.067+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4338350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.067+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b4338890 0x55e1b433ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.067+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4296460 con 0x55e1b432d0c0 2023-10-06T22:19:08.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.067+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4287c20 con 0x55e1b432d9e0 2023-10-06T22:19:08.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.067+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4285540 con 0x55e1b4338890 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500619c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d0c0 0x55e1b432d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500619c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d0c0 0x55e1b432d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53960/0 (socket says 172.21.15.57:53960) 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500619c700 1 -- 172.21.15.57:0/3749203756 learned_addr learned my addr 172.21.15.57:0/3749203756 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500599b700 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4338350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500519a700 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b4338890 0x55e1b433ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f4ff6ffd700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4fec0a15f0 0x7f4fec0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500519a700 1 -- 172.21.15.57:0/3749203756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 msgr2=0x55e1b4338350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500519a700 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4338350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500519a700 1 -- 172.21.15.57:0/3749203756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d0c0 msgr2=0x55e1b432d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500519a700 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d0c0 0x55e1b432d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f4ff6ffd700 1 -- 172.21.15.57:0/2557531744 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4ffc0cdd30 con 0x7f50001075d0 2023-10-06T22:19:08.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.068+0000 7f500519a700 1 -- 172.21.15.57:0/3749203756 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b40abde0 con 0x55e1b4338890 2023-10-06T22:19:08.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.069+0000 7f500619c700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4fec0a15f0 0x7f4fec0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.069+0000 7f500519a700 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b4338890 0x55e1b433ac80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009e10 tx=0x7f4ff0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf5750d59db13f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:08.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.069+0000 7f4febfff700 1 -- 172.21.15.57:0/3749203756 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0019630 con 0x55e1b4338890 2023-10-06T22:19:08.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.069+0000 7f4febfff700 1 -- 172.21.15.57:0/3749203756 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff0019c50 con 0x55e1b4338890 2023-10-06T22:19:08.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.069+0000 7f4febfff700 1 -- 172.21.15.57:0/3749203756 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000ed10 con 0x55e1b4338890 2023-10-06T22:19:08.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3749203756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b4338890 msgr2=0x55e1b433ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b4338890 0x55e1b433ac80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009e10 tx=0x7f4ff0011720 comp rx=0 tx=0).stop 2023-10-06T22:19:08.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f500619c700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4fec0a15f0 0x7f4fec0a3ab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80076a0 tx=0x7f4ff8009040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:08.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3749203756 shutdown_connections 2023-10-06T22:19:08.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b4338890 0x55e1b433ac80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d0c0 0x55e1b432d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3749203756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4338350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.070+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3749203756 >> 172.21.15.57:0/3749203756 conn(0x55e1b42a2260 msgr2=0x55e1b4329450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:08.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.071+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3749203756 shutdown_connections 2023-10-06T22:19:08.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.071+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3749203756 wait complete. 2023-10-06T22:19:08.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.071+0000 7f501b4c1b80 1 -- start start 2023-10-06T22:19:08.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 1 Processor -- start 2023-10-06T22:19:08.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b43355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d9e0 0x55e1b4335b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b4338890 0x55e1b4336050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4280e30 con 0x55e1b432d9e0 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b421d710 con 0x55e1b4338890 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b421d3b0 con 0x55e1b432d0c0 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f501b4c1b80 10 client.0 _ll_register_callbacks cb 0x55e1b407a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.072+0000 7f500599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b43355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500619c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b4338890 0x55e1b4336050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b43355d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44034/0 (socket says 172.21.15.57:44034) 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500599b700 1 -- 172.21.15.57:0/3732265358 learned_addr learned my addr 172.21.15.57:0/3732265358 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500519a700 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d9e0 0x55e1b4335b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500619c700 1 -- 172.21.15.57:0/3732265358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 msgr2=0x55e1b43355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500619c700 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b43355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500619c700 1 -- 172.21.15.57:0/3732265358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d9e0 msgr2=0x55e1b4335b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500619c700 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d9e0 0x55e1b4335b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500619c700 1 -- 172.21.15.57:0/3732265358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff0009a70 con 0x55e1b4338890 2023-10-06T22:19:08.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500519a700 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d9e0 0x55e1b4335b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:19:08.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.073+0000 7f500619c700 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b4338890 0x55e1b4336050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8017d90 tx=0x7f4ff8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47e1733125483a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f4fe97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3732265358 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8016df0 con 0x55e1b4338890 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3732265358 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4ff801c030 con 0x55e1b4338890 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1b4320f10 con 0x55e1b4338890 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3732265358 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff802d460 con 0x55e1b4338890 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f501b4c1b80 10 client.26504 fetch_fsmap learned FSMap version 40 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e1b4094c60 con 0x55e1b4338890 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.074+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3732265358 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff8031020 con 0x55e1b4338890 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3732265358 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 40) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f4ff8022650 con 0x55e1b4338890 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f501b4c1b80 10 client.26504 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T22:19:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f501b4c1b80 10 client.26504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f501b4c1b80 20 client.26504 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b4348430 0x55e1b434a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f501b4c1b80 4 client.26504 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1b4279e10 con 0x55e1b4348430 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f500599b700 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b4348430 0x55e1b434a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.076+0000 7f4fe97fa700 10 client.26504 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.075+0000 7f500599b700 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b4348430 0x55e1b434a850 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9b6334f2b02286 server_cookie=b19432922af076 in_seq=0 out_seq=0 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3732265358 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e1b4279e10 con 0x55e1b4348430 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f4fe97fa700 10 client.26504 handle_command_reply: tid=0 2023-10-06T22:19:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 client.26504 shutdown 2023-10-06T22:19:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b4338890 msgr2=0x55e1b4336050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b4338890 0x55e1b4336050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8017d90 tx=0x7f4ff8004ad0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 shutdown_connections 2023-10-06T22:19:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b4348430 0x55e1b434a850 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b4338890 0x55e1b4336050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b43355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3732265358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b432d9e0 0x55e1b4335b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 >> 172.21.15.57:0/3732265358 conn(0x55e1b42a2260 msgr2=0x55e1b4328fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f4fe97fa700 0 client.26504 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.077+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 shutdown_connections 2023-10-06T22:19:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.078+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3732265358 wait complete. 2023-10-06T22:19:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.078+0000 7f501b4c1b80 20 client.26504 trim_cache size 0 max 16384 2023-10-06T22:19:08.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 Processor -- start 2023-10-06T22:19:08.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 -- start start 2023-10-06T22:19:08.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4330030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b433f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4101920 con 0x55e1b4338890 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b433f910 con 0x55e1b432d9e0 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.097+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b432a020 con 0x55e1b432d0c0 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4330030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b433f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4330030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44044/0 (socket says 172.21.15.57:44044) 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 -- 172.21.15.57:0/556769237 learned_addr learned my addr 172.21.15.57:0/556769237 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500619c700 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 -- 172.21.15.57:0/556769237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 msgr2=0x55e1b433f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b433f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 -- 172.21.15.57:0/556769237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 msgr2=0x55e1b43308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 -- 172.21.15.57:0/556769237 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b4325140 con 0x55e1b432d0c0 2023-10-06T22:19:08.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.098+0000 7f500599b700 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4330030 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0972b0 tx=0x7f4ffc0d01b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58ba442cf6f4050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.099+0000 7f4ff4bff700 1 -- 172.21.15.57:0/556769237 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0970f0 con 0x55e1b432d0c0 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.099+0000 7f4ff4bff700 1 -- 172.21.15.57:0/556769237 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ffc09ab80 con 0x55e1b432d0c0 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.099+0000 7f4ff4bff700 1 -- 172.21.15.57:0/556769237 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0a30b0 con 0x55e1b432d0c0 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.099+0000 7f501b4c1b80 1 -- 172.21.15.57:0/556769237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 msgr2=0x55e1b4330030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.099+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4330030 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0972b0 tx=0x7f4ffc0d01b0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 -- 172.21.15.57:0/556769237 shutdown_connections 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4330030 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/556769237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b433f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 -- 172.21.15.57:0/556769237 >> 172.21.15.57:0/556769237 conn(0x55e1b42a2260 msgr2=0x55e1b434a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:08.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 -- 172.21.15.57:0/556769237 shutdown_connections 2023-10-06T22:19:08.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 -- 172.21.15.57:0/556769237 wait complete. 2023-10-06T22:19:08.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.100+0000 7f501b4c1b80 1 -- start start 2023-10-06T22:19:08.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 1 Processor -- start 2023-10-06T22:19:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4395330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4395870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b43960a0 con 0x55e1b4338890 2023-10-06T22:19:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4394740 con 0x55e1b432d9e0 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b4394900 con 0x55e1b432d0c0 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.101+0000 7f501b4c1b80 10 client.0 _ll_register_callbacks cb 0x7f4ff8017760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4395870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4395870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:48384/0 (socket says 172.21.15.57:48384) 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 -- 172.21.15.57:0/3102795733 learned_addr learned my addr 172.21.15.57:0/3102795733 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500619c700 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 -- 172.21.15.57:0/3102795733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 msgr2=0x55e1b4395330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4395330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 -- 172.21.15.57:0/3102795733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 msgr2=0x55e1b43941d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43941d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500519a700 1 -- 172.21.15.57:0/3102795733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50001a2d90 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.102+0000 7f500619c700 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43941d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f500519a700 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4395870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009230 tx=0x7f4ff00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b3757d7b007cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f4fe97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3102795733 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00196f0 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff0019c50 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3102795733 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4ff0019850 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3102795733 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff0019a10 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f501b4c1b80 10 client.26510 fetch_fsmap learned FSMap version 40 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e1b40d2250 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.103+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3102795733 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff0019c50 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3102795733 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 40) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f4ff0024720 con 0x55e1b432d9e0 2023-10-06T22:19:08.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f501b4c1b80 10 client.26510 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T22:19:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f501b4c1b80 10 client.26510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f501b4c1b80 20 client.26510 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b43ae090 0x55e1b43b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f501b4c1b80 4 client.26510 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:19:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e1b4279b30 con 0x55e1b43ae090 2023-10-06T22:19:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.104+0000 7f500599b700 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b43ae090 0x55e1b43b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f500599b700 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b43ae090 0x55e1b43b04b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a8bfd69ef58faa server_cookie=353f5298cb92e525 in_seq=0 out_seq=0 2023-10-06T22:19:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f4fe97fa700 10 client.26510 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f4fe97fa700 1 -- 172.21.15.57:0/3102795733 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e1b4279b30 con 0x55e1b43ae090 2023-10-06T22:19:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f4fe97fa700 10 client.26510 handle_command_reply: tid=0 2023-10-06T22:19:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 client.26510 shutdown 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 msgr2=0x55e1b4395870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4395870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009230 tx=0x7f4ff00057c0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 shutdown_connections 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1b43ae090 0x55e1b43b04b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1b4338890 0x55e1b43941d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1b432d0c0 0x55e1b4395330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f4fe97fa700 0 client.26510 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 --2- 172.21.15.57:0/3102795733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1b432d9e0 0x55e1b4395870 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 >> 172.21.15.57:0/3102795733 conn(0x55e1b42a2260 msgr2=0x55e1b43308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 shutdown_connections 2023-10-06T22:19:08.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.106+0000 7f501b4c1b80 1 -- 172.21.15.57:0/3102795733 wait complete. 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f501b4c1b80 20 client.26510 trim_cache size 0 max 16384 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:19:08.113 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:19:08.113 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:19:08.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4fec0a15f0 msgr2=0x7f4fec0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4fec0a15f0 0x7f4fec0a3ab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80076a0 tx=0x7f4ff8009040 comp rx=0 tx=0).stop 2023-10-06T22:19:08.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50001075d0 msgr2=0x7f500019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50001075d0 0x7f500019d570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098e0 tx=0x7f4ffc00f690 comp rx=0 tx=0).stop 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 shutdown_connections 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4fec0a15f0 0x7f4fec0a3ab0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500010d570 0x7f50001a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50001075d0 0x7f500019d570 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 --2- 172.21.15.57:0/2557531744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5000107f90 0x7f500019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 >> 172.21.15.57:0/2557531744 conn(0x7f5000076d50 msgr2=0x7f500010b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.107+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 shutdown_connections 2023-10-06T22:19:08.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:08.108+0000 7f5007bff700 1 -- 172.21.15.57:0/2557531744 wait complete. 2023-10-06T22:19:08.130 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:19:09.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:09 smithi190 ceph-mon[134891]: pgmap v135: 97 pgs: 97 active+clean; 728 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 518 op/s 2023-10-06T22:19:09.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:09 smithi057 ceph-mon[98492]: pgmap v135: 97 pgs: 97 active+clean; 728 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 518 op/s 2023-10-06T22:19:09.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:09 smithi156 ceph-mon[134358]: pgmap v135: 97 pgs: 97 active+clean; 728 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 518 op/s 2023-10-06T22:19:11.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:11 smithi190 ceph-mon[134891]: pgmap v136: 97 pgs: 97 active+clean; 728 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 455 op/s 2023-10-06T22:19:11.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:11 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:19:11.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:11 smithi057 ceph-mon[98492]: pgmap v136: 97 pgs: 97 active+clean; 728 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 455 op/s 2023-10-06T22:19:11.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:11 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:19:11.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:11 smithi156 ceph-mon[134358]: pgmap v136: 97 pgs: 97 active+clean; 728 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 455 op/s 2023-10-06T22:19:11.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:11 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:19:12.070 INFO:tasks.workunit.client.0.smithi057.stdout: 226 439520 1196 298264 1237 78863 605 2023-10-06T22:19:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 -- 172.21.15.57:0/3237332117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 msgr2=0x7f4494102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 --2- 172.21.15.57:0/3237332117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494102500 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4484009990 tx=0x7f4484009ca0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 -- 172.21.15.57:0/3237332117 shutdown_connections 2023-10-06T22:19:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 --2- 172.21.15.57:0/3237332117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f449410b4f0 0x7f449410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 --2- 172.21.15.57:0/3237332117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494102500 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 --2- 172.21.15.57:0/3237332117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4494102a40 0x7f449410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 -- 172.21.15.57:0/3237332117 >> 172.21.15.57:0/3237332117 conn(0x7f44940fb860 msgr2=0x7f44940fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 -- 172.21.15.57:0/3237332117 shutdown_connections 2023-10-06T22:19:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.179+0000 7f44989ff700 1 -- 172.21.15.57:0/3237332117 wait complete. 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.180+0000 7f44989ff700 1 Processor -- start 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.180+0000 7f44989ff700 1 -- start start 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f44989ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f44989ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4494102a40 0x7f4494197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f44989ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449410b4f0 0x7f449419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f44989ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449419c1c0 con 0x7f4494102120 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f44989ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449419c330 con 0x7f4494102a40 2023-10-06T22:19:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f44989ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449419c4f0 con 0x7f449410b4f0 2023-10-06T22:19:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4494102a40 0x7f4494197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f449259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4492d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449410b4f0 0x7f449419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f449259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37008/0 (socket says 172.21.15.57:37008) 2023-10-06T22:19:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f449259c700 1 -- 172.21.15.57:0/4260680946 learned_addr learned my addr 172.21.15.57:0/4260680946 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4494102a40 0x7f4494197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59348/0 (socket says 172.21.15.57:59348) 2023-10-06T22:19:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4491d9b700 1 -- 172.21.15.57:0/4260680946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449410b4f0 msgr2=0x7f449419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4491d9b700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449410b4f0 0x7f449419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4491d9b700 1 -- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 msgr2=0x7f4494196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4491d9b700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f4491d9b700 1 -- 172.21.15.57:0/4260680946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840095f0 con 0x7f4494102a40 2023-10-06T22:19:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.181+0000 7f449259c700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:19:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.182+0000 7f4491d9b700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4494102a40 0x7f4494197510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f448800e9f0 tx=0x7f448800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.182+0000 7f44837fe700 1 -- 172.21.15.57:0/4260680946 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800c9c0 con 0x7f4494102a40 2023-10-06T22:19:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.182+0000 7f44837fe700 1 -- 172.21.15.57:0/4260680946 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4488010430 con 0x7f4494102a40 2023-10-06T22:19:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.182+0000 7f44837fe700 1 -- 172.21.15.57:0/4260680946 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488018680 con 0x7f4494102a40 2023-10-06T22:19:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.182+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449419c7d0 con 0x7f4494102a40 2023-10-06T22:19:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.182+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44941a4010 con 0x7f4494102a40 2023-10-06T22:19:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.186+0000 7f44837fe700 1 -- 172.21.15.57:0/4260680946 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4488010860 con 0x7f4494102a40 2023-10-06T22:19:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.187+0000 7f44837fe700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f447c0a1670 0x7f447c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.187+0000 7f44837fe700 1 -- 172.21.15.57:0/4260680946 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4488014070 con 0x7f4494102a40 2023-10-06T22:19:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.187+0000 7f44ac31db80 1 Processor -- start 2023-10-06T22:19:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.187+0000 7f449259c700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f447c0a1670 0x7f447c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.187+0000 7f44ac31db80 1 -- start start 2023-10-06T22:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352acc00 0x5593352acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352ad520 0x5593352bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352bc7e0 0x5593352bebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f44ac31db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559335216400 con 0x5593352acc00 2023-10-06T22:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f44ac31db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559335207bc0 con 0x5593352bc7e0 2023-10-06T22:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f44ac31db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593352054e0 con 0x5593352ad520 2023-10-06T22:19:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352ad520 0x5593352bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352ad520 0x5593352bc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:50620/0 (socket says 172.21.15.57:50620) 2023-10-06T22:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f4491d9b700 1 -- 172.21.15.57:0/143421315 learned_addr learned my addr 172.21.15.57:0/143421315 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f449259c700 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352acc00 0x5593352acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.188+0000 7f4492d9d700 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352bc7e0 0x5593352bebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f449259c700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f447c0a1670 0x7f447c0a3b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4484006010 tx=0x7f448400b260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4491d9b700 1 -- 172.21.15.57:0/143421315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352bc7e0 msgr2=0x5593352bebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4491d9b700 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352bc7e0 0x5593352bebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4491d9b700 1 -- 172.21.15.57:0/143421315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352acc00 msgr2=0x5593352acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4491d9b700 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352acc00 0x5593352acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4491d9b700 1 -- 172.21.15.57:0/143421315 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55933502bd80 con 0x5593352ad520 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4491d9b700 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352ad520 0x5593352bc2a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4488096c00 tx=0x7f4488099bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d2da072c874e34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4480ff9700 1 -- 172.21.15.57:0/143421315 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448809e120 con 0x5593352ad520 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4480ff9700 1 -- 172.21.15.57:0/143421315 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448809e2e0 con 0x5593352ad520 2023-10-06T22:19:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.189+0000 7f4480ff9700 1 -- 172.21.15.57:0/143421315 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488099680 con 0x5593352ad520 2023-10-06T22:19:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 -- 172.21.15.57:0/143421315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352ad520 msgr2=0x5593352bc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352ad520 0x5593352bc2a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4488096c00 tx=0x7f4488099bf0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 -- 172.21.15.57:0/143421315 shutdown_connections 2023-10-06T22:19:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352bc7e0 0x5593352bebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352acc00 0x5593352acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 --2- 172.21.15.57:0/143421315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352ad520 0x5593352bc2a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 -- 172.21.15.57:0/143421315 >> 172.21.15.57:0/143421315 conn(0x559335222200 msgr2=0x5593352a9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 -- 172.21.15.57:0/143421315 shutdown_connections 2023-10-06T22:19:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.190+0000 7f44ac31db80 1 -- 172.21.15.57:0/143421315 wait complete. 2023-10-06T22:19:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.191+0000 7f44ac31db80 1 -- start start 2023-10-06T22:19:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.191+0000 7f44ac31db80 1 Processor -- start 2023-10-06T22:19:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.191+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f44ac31db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559335200dd0 con 0x5593352bc7e0 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f44ac31db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933519d6b0 con 0x5593352ad520 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f44ac31db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933519d350 con 0x5593352acc00 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f44ac31db80 10 client.0 _ll_register_callbacks cb 0x559334ffa7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59356/0 (socket says 172.21.15.57:59356) 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 -- 172.21.15.57:0/1830936880 learned_addr learned my addr 172.21.15.57:0/1830936880 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f449259c700 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4492d9d700 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 -- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 msgr2=0x5593352bbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352bbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 -- 172.21.15.57:0/1830936880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 msgr2=0x5593352b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 -- 172.21.15.57:0/1830936880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44941a4010 con 0x5593352ad520 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4491d9b700 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b5160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44880a1ff0 tx=0x7f448809a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1880bf5cf9f5a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.192+0000 7f4479ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f4479ffb700 1 -- 172.21.15.57:0/1830936880 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448809e120 con 0x5593352ad520 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f4479ffb700 1 -- 172.21.15.57:0/1830936880 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f448809a600 con 0x5593352ad520 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593352a0d50 con 0x5593352ad520 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f4479ffb700 1 -- 172.21.15.57:0/1830936880 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44880a2090 con 0x5593352ad520 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 10 client.26522 fetch_fsmap learned FSMap version 41 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559335014c00 con 0x5593352ad520 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f4479ffb700 1 -- 172.21.15.57:0/1830936880 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448807c070 con 0x5593352ad520 2023-10-06T22:19:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f4479ffb700 1 -- 172.21.15.57:0/1830936880 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 41) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4488095f80 con 0x5593352ad520 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 10 client.26522 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 10 client.26522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 20 client.26522 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x5593352d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.193+0000 7f44ac31db80 4 client.26522 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.194+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593351f9db0 con 0x5593352d0360 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.194+0000 7f449259c700 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x5593352d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.194+0000 7f449259c700 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x5593352d2750 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8796f45670ad3b21 server_cookie=e27e62e6252802f2 in_seq=0 out_seq=0 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.194+0000 7f4479ffb700 10 client.26522 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f4479ffb700 1 -- 172.21.15.57:0/1830936880 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5593351f9db0 con 0x5593352d0360 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f4479ffb700 10 client.26522 handle_command_reply: tid=0 2023-10-06T22:19:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 client.26522 shutdown 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 msgr2=0x5593352b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b5160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44880a1ff0 tx=0x7f448809a310 comp rx=0 tx=0).stop 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 shutdown_connections 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x5593352d2750 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f4479ffb700 0 client.26522 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352bbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1830936880 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b5160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.197+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 >> 172.21.15.57:0/1830936880 conn(0x559335222200 msgr2=0x5593352a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.198+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 shutdown_connections 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.198+0000 7f44ac31db80 1 -- 172.21.15.57:0/1830936880 wait complete. 2023-10-06T22:19:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.198+0000 7f44ac31db80 20 client.26522 trim_cache size 0 max 16384 2023-10-06T22:19:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.216+0000 7f44ac31db80 1 Processor -- start 2023-10-06T22:19:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.216+0000 7f44ac31db80 1 -- start start 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.216+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f44ac31db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593350818c0 con 0x5593352bc7e0 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f44ac31db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593352b10b0 con 0x5593352ad520 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f44ac31db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593352a0910 con 0x5593352acc00 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f449259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59362/0 (socket says 172.21.15.57:59362) 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 -- 172.21.15.57:0/2375082591 learned_addr learned my addr 172.21.15.57:0/2375082591 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4492d9d700 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 -- 172.21.15.57:0/2375082591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 msgr2=0x5593352b9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352b9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 -- 172.21.15.57:0/2375082591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 msgr2=0x5593352bba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352bba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.217+0000 7f4491d9b700 1 -- 172.21.15.57:0/2375082591 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593350521f0 con 0x5593352ad520 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.218+0000 7f4491d9b700 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b4060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f448800c5b0 tx=0x7f44880caa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7ec299843b8c06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.218+0000 7f44817fa700 1 -- 172.21.15.57:0/2375082591 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448809e120 con 0x5593352ad520 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.218+0000 7f44817fa700 1 -- 172.21.15.57:0/2375082591 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448807c030 con 0x5593352ad520 2023-10-06T22:19:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.218+0000 7f44817fa700 1 -- 172.21.15.57:0/2375082591 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44880b8120 con 0x5593352ad520 2023-10-06T22:19:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 -- 172.21.15.57:0/2375082591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 msgr2=0x5593352b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b4060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f448800c5b0 tx=0x7f44880caa00 comp rx=0 tx=0).stop 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 -- 172.21.15.57:0/2375082591 shutdown_connections 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352bba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 --2- 172.21.15.57:0/2375082591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352b4060 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 -- 172.21.15.57:0/2375082591 >> 172.21.15.57:0/2375082591 conn(0x559335222200 msgr2=0x5593352b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 -- 172.21.15.57:0/2375082591 shutdown_connections 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 -- 172.21.15.57:0/2375082591 wait complete. 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.219+0000 7f44ac31db80 1 -- start start 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 1 Processor -- start 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593352ca040 con 0x5593352bc7e0 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593352ca180 con 0x5593352ad520 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593352ca340 con 0x5593352acc00 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f44ac31db80 10 client.0 _ll_register_callbacks cb 0x7f4484005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f4491d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:59368/0 (socket says 172.21.15.57:59368) 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f4491d9b700 1 -- 172.21.15.57:0/1838924829 learned_addr learned my addr 172.21.15.57:0/1838924829 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.220+0000 7f4492d9d700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f449259c700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4491d9b700 1 -- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 msgr2=0x5593352ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4491d9b700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4491d9b700 1 -- 172.21.15.57:0/1838924829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 msgr2=0x5593352c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4491d9b700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4491d9b700 1 -- 172.21.15.57:0/1838924829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559335014c00 con 0x5593352ad520 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f449259c700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4492d9d700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4491d9b700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352cd260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4488010810 tx=0x7f4488010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec702f13c0885acc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4479ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:19:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4479ffb700 1 -- 172.21.15.57:0/1838924829 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448809e120 con 0x5593352ad520 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f448809e2e0 con 0x5593352ad520 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4479ffb700 1 -- 172.21.15.57:0/1838924829 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f44880a2520 con 0x5593352ad520 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f4479ffb700 1 -- 172.21.15.57:0/1838924829 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44880b8120 con 0x5593352ad520 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f44ac31db80 10 client.26528 fetch_fsmap learned FSMap version 41 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.221+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55933502a250 con 0x5593352ad520 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f4479ffb700 1 -- 172.21.15.57:0/1838924829 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448809e2e0 con 0x5593352ad520 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f4479ffb700 1 -- 172.21.15.57:0/1838924829 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 41) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f44880991c0 con 0x5593352ad520 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f44ac31db80 10 client.26528 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f44ac31db80 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f44ac31db80 20 client.26528 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x559335326b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f44ac31db80 4 client.26528 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:19:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5593351f9ad0 con 0x5593352d0360 2023-10-06T22:19:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.222+0000 7f449259c700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x559335326b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.223+0000 7f449259c700 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x559335326b10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eba61fee206840 server_cookie=99146057ad334a01 in_seq=0 out_seq=0 2023-10-06T22:19:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:12.223+0000 7f4479ffb700 10 client.26528 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:13.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:13 smithi190 ceph-mon[134891]: pgmap v137: 97 pgs: 97 active+clean; 758 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 583 op/s 2023-10-06T22:19:13.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:13 smithi057 ceph-mon[98492]: pgmap v137: 97 pgs: 97 active+clean; 758 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 583 op/s 2023-10-06T22:19:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:13 smithi156 ceph-mon[134358]: pgmap v137: 97 pgs: 97 active+clean; 758 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 583 op/s 2023-10-06T22:19:15.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:15 smithi190 ceph-mon[134891]: pgmap v138: 97 pgs: 97 active+clean; 762 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2023-10-06T22:19:15.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:15 smithi057 ceph-mon[98492]: pgmap v138: 97 pgs: 97 active+clean; 762 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2023-10-06T22:19:15.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:15 smithi156 ceph-mon[134358]: pgmap v138: 97 pgs: 97 active+clean; 762 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2023-10-06T22:19:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.978+0000 7f4479ffb700 1 -- 172.21.15.57:0/1838924829 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5593351f9ad0 con 0x5593352d0360 2023-10-06T22:19:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.978+0000 7f4479ffb700 10 client.26528 handle_command_reply: tid=0 2023-10-06T22:19:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.981+0000 7f44ac31db80 1 client.26528 shutdown 2023-10-06T22:19:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 msgr2=0x5593352cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352cd260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4488010810 tx=0x7f4488010c60 comp rx=0 tx=0).stop 2023-10-06T22:19:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 shutdown_connections 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5593352d0360 0x559335326b10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5593352bc7e0 0x5593352c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5593352acc00 0x5593352ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 --2- 172.21.15.57:0/1838924829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593352ad520 0x5593352cd260 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 >> 172.21.15.57:0/1838924829 conn(0x559335222200 msgr2=0x5593352b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f4479ffb700 0 client.26528 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 shutdown_connections 2023-10-06T22:19:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 1 -- 172.21.15.57:0/1838924829 wait complete. 2023-10-06T22:19:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.982+0000 7f44ac31db80 20 client.26528 trim_cache size 0 max 16384 2023-10-06T22:19:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:19:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:19:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:19:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:19:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f447c0a1670 msgr2=0x7f447c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f447c0a1670 0x7f447c0a3b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4484006010 tx=0x7f448400b260 comp rx=0 tx=0).stop 2023-10-06T22:19:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4494102a40 msgr2=0x7f4494197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4494102a40 0x7f4494197510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f448800e9f0 tx=0x7f448800ed00 comp rx=0 tx=0).stop 2023-10-06T22:19:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 shutdown_connections 2023-10-06T22:19:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f447c0a1670 0x7f447c0a3b30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449410b4f0 0x7f449419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4494102120 0x7f4494196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 --2- 172.21.15.57:0/4260680946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4494102a40 0x7f4494197510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.983+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 >> 172.21.15.57:0/4260680946 conn(0x7f44940fb860 msgr2=0x7f449410d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.984+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 shutdown_connections 2023-10-06T22:19:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:15.984+0000 7f44989ff700 1 -- 172.21.15.57:0/4260680946 wait complete. 2023-10-06T22:19:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:17 smithi190 ceph-mon[134891]: pgmap v139: 97 pgs: 97 active+clean; 762 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 192 op/s 2023-10-06T22:19:17.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:17 smithi057 ceph-mon[98492]: pgmap v139: 97 pgs: 97 active+clean; 762 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 192 op/s 2023-10-06T22:19:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:17 smithi156 ceph-mon[134358]: pgmap v139: 97 pgs: 97 active+clean; 762 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 192 op/s 2023-10-06T22:19:19.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:19 smithi057 ceph-mon[98492]: pgmap v140: 97 pgs: 97 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 851 op/s 2023-10-06T22:19:19.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:19 smithi156 ceph-mon[134358]: pgmap v140: 97 pgs: 97 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 851 op/s 2023-10-06T22:19:19.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:19 smithi190 ceph-mon[134891]: pgmap v140: 97 pgs: 97 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 851 op/s 2023-10-06T22:19:21.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:21 smithi057 ceph-mon[98492]: pgmap v141: 97 pgs: 97 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 799 op/s 2023-10-06T22:19:21.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:21 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:19:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:21 smithi156 ceph-mon[134358]: pgmap v141: 97 pgs: 97 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 799 op/s 2023-10-06T22:19:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:21 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:19:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:21 smithi190 ceph-mon[134891]: pgmap v141: 97 pgs: 97 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 799 op/s 2023-10-06T22:19:21.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:21 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:19:22.070 INFO:tasks.workunit.client.0.smithi057.stdout: 235 439449 760 301266 557 85946 459 2023-10-06T22:19:23.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:23 smithi156 ceph-mon[134358]: pgmap v142: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 851 op/s 2023-10-06T22:19:23.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:23 smithi190 ceph-mon[134891]: pgmap v142: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 851 op/s 2023-10-06T22:19:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:23 smithi057 ceph-mon[98492]: pgmap v142: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 851 op/s 2023-10-06T22:19:25.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:25 smithi156 ceph-mon[134358]: pgmap v143: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 725 op/s 2023-10-06T22:19:25.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:25 smithi190 ceph-mon[134891]: pgmap v143: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 725 op/s 2023-10-06T22:19:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:25 smithi057 ceph-mon[98492]: pgmap v143: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 725 op/s 2023-10-06T22:19:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:27 smithi190 ceph-mon[134891]: pgmap v144: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 712 op/s 2023-10-06T22:19:27.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:27 smithi057 ceph-mon[98492]: pgmap v144: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 712 op/s 2023-10-06T22:19:27.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:27 smithi156 ceph-mon[134358]: pgmap v144: 97 pgs: 97 active+clean; 876 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 712 op/s 2023-10-06T22:19:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:29 smithi190 ceph-mon[134891]: pgmap v145: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-06T22:19:29.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:29 smithi057 ceph-mon[98492]: pgmap v145: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-06T22:19:29.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:29 smithi156 ceph-mon[134358]: pgmap v145: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-06T22:19:30.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:30 smithi057 ceph-mon[98492]: pgmap v146: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 89 op/s 2023-10-06T22:19:30.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:30 smithi156 ceph-mon[134358]: pgmap v146: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 89 op/s 2023-10-06T22:19:30.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:30 smithi190 ceph-mon[134891]: pgmap v146: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 89 op/s 2023-10-06T22:19:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.556+0000 7fc9e05ff700 1 -- 172.21.15.57:0/3891562775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9d811a690 msgr2=0x7fc9d811ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.556+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/3891562775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9d811a690 0x7fc9d811ca80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009990 tx=0x7fc9c8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.569+0000 7fc9e05ff700 1 -- 172.21.15.57:0/3891562775 shutdown_connections 2023-10-06T22:19:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.569+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/3891562775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9d811a690 0x7fc9d811ca80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.569+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/3891562775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810f620 0x7fc9d8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.569+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/3891562775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d810fa90 0x7fc9d81083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.569+0000 7fc9e05ff700 1 -- 172.21.15.57:0/3891562775 >> 172.21.15.57:0/3891562775 conn(0x7fc9d806cfa0 msgr2=0x7fc9d806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.579+0000 7fc9e05ff700 1 -- 172.21.15.57:0/3891562775 shutdown_connections 2023-10-06T22:19:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.579+0000 7fc9e05ff700 1 -- 172.21.15.57:0/3891562775 wait complete. 2023-10-06T22:19:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.581+0000 7fc9e05ff700 1 Processor -- start 2023-10-06T22:19:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.581+0000 7fc9e05ff700 1 -- start start 2023-10-06T22:19:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.582+0000 7fc9e05ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d810f620 0x7fc9d81a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.582+0000 7fc9e05ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810fa90 0x7fc9d81a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.582+0000 7fc9e05ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9d811a690 0x7fc9d81aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.582+0000 7fc9e05ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81ab1d0 con 0x7fc9d811a690 2023-10-06T22:19:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.582+0000 7fc9e05ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81ab340 con 0x7fc9d810f620 2023-10-06T22:19:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.582+0000 7fc9e05ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81ab500 con 0x7fc9d810fa90 2023-10-06T22:19:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.583+0000 7fc9ddb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810fa90 0x7fc9d81a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.583+0000 7fc9ddb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810fa90 0x7fc9d81a6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33746/0 (socket says 172.21.15.57:33746) 2023-10-06T22:19:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.583+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/240502635 learned_addr learned my addr 172.21.15.57:0/240502635 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.584+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/240502635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d810f620 msgr2=0x7fc9d81a5fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:19:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.584+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d810f620 0x7fc9d81a5fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.584+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9d811a690 msgr2=0x7fc9d81aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.584+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9d811a690 0x7fc9d81aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.584+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/240502635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80095f0 con 0x7fc9d810fa90 2023-10-06T22:19:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.585+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810fa90 0x7fc9d81a6520 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000b840 tx=0x7fc9d000bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.709+0000 7fc9cf7fe700 1 -- 172.21.15.57:0/240502635 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000d890 con 0x7fc9d810fa90 2023-10-06T22:19:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.709+0000 7fc9cf7fe700 1 -- 172.21.15.57:0/240502635 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9d000e430 con 0x7fc9d810fa90 2023-10-06T22:19:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.709+0000 7fc9cf7fe700 1 -- 172.21.15.57:0/240502635 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0017660 con 0x7fc9d810fa90 2023-10-06T22:19:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.711+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d81ab7e0 con 0x7fc9d810fa90 2023-10-06T22:19:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.712+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d8110050 con 0x7fc9d810fa90 2023-10-06T22:19:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.720+0000 7fc9f3fa1b80 1 Processor -- start 2023-10-06T22:19:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.736+0000 7fc9f3fa1b80 1 -- start start 2023-10-06T22:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.736+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.736+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.736+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.737+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed942400 con 0x55deed9d9520 2023-10-06T22:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.737+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed933bc0 con 0x55deed9e43d0 2023-10-06T22:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.737+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed9314e0 con 0x55deed9d8c00 2023-10-06T22:19:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.748+0000 7fc9cf7fe700 1 -- 172.21.15.57:0/240502635 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc9d00177c0 con 0x7fc9d810fa90 2023-10-06T22:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.751+0000 7fc9cf7fe700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc9c40a1800 0x7fc9c40a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.751+0000 7fc9cf7fe700 1 -- 172.21.15.57:0/240502635 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc9d00cdac0 con 0x7fc9d810fa90 2023-10-06T22:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.755+0000 7fc9de39b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.755+0000 7fc9de39b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33680/0 (socket says 172.21.15.57:33680) 2023-10-06T22:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.755+0000 7fc9de39b700 1 -- 172.21.15.57:0/3869551407 learned_addr learned my addr 172.21.15.57:0/3869551407 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.761+0000 7fc9de39b700 1 -- 172.21.15.57:0/3869551407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 msgr2=0x55deed9d8fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.761+0000 7fc9de39b700 1 --2- 172.21.15.57:0/3869551407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9d8fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.761+0000 7fc9de39b700 1 -- 172.21.15.57:0/3869551407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 msgr2=0x55deed9e67c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.761+0000 7fc9de39b700 1 --2- 172.21.15.57:0/3869551407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9e67c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.761+0000 7fc9de39b700 1 -- 172.21.15.57:0/3869551407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deed757d80 con 0x55deed9d9520 2023-10-06T22:19:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.774+0000 7fc9de39b700 1 --2- 172.21.15.57:0/3869551407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e3e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009810 tx=0x7fc9d4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50733aaa3888960 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.776+0000 7fc9deb9c700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc9c40a1800 0x7fc9c40a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.786+0000 7fc9ccbfe700 1 -- 172.21.15.57:0/3869551407 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4011830 con 0x55deed9d9520 2023-10-06T22:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.787+0000 7fc9ccbfe700 1 -- 172.21.15.57:0/3869551407 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9d4011e50 con 0x55deed9d9520 2023-10-06T22:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.787+0000 7fc9ccbfe700 1 -- 172.21.15.57:0/3869551407 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4010380 con 0x55deed9d9520 2023-10-06T22:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.788+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3869551407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 msgr2=0x55deed9e3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.788+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3869551407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e3e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009810 tx=0x7fc9d4009b20 comp rx=0 tx=0).stop 2023-10-06T22:19:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.808+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3869551407 shutdown_connections 2023-10-06T22:19:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.808+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3869551407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.808+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3869551407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9d8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.808+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3869551407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e3e90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.808+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3869551407 >> 172.21.15.57:0/3869551407 conn(0x55deed94e200 msgr2=0x55deed9d5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.809+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3869551407 shutdown_connections 2023-10-06T22:19:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.809+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3869551407 wait complete. 2023-10-06T22:19:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.810+0000 7fc9f3fa1b80 1 -- start start 2023-10-06T22:19:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.810+0000 7fc9f3fa1b80 1 Processor -- start 2023-10-06T22:19:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.812+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.812+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.812+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.812+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed92cdd0 con 0x55deed9d9520 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.812+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed8c96b0 con 0x55deed9e43d0 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.812+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed8c9350 con 0x55deed9d8c00 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.813+0000 7fc9f3fa1b80 10 client.0 _ll_register_callbacks cb 0x55deed7267a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.815+0000 7fc9de39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.815+0000 7fc9de39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9e3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33760/0 (socket says 172.21.15.57:33760) 2023-10-06T22:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.815+0000 7fc9de39b700 1 -- 172.21.15.57:0/3466162884 learned_addr learned my addr 172.21.15.57:0/3466162884 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.816+0000 7fc9de39b700 1 -- 172.21.15.57:0/3466162884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 msgr2=0x55deed9e1820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.816+0000 7fc9de39b700 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9e1820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.816+0000 7fc9de39b700 1 -- 172.21.15.57:0/3466162884 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 msgr2=0x55deed9e10e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.816+0000 7fc9de39b700 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e10e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.816+0000 7fc9de39b700 1 -- 172.21.15.57:0/3466162884 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4009440 con 0x55deed9d8c00 2023-10-06T22:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.816+0000 7fc9de39b700 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9e3e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400baf0 tx=0x7fc9d400bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e4f4803f4ee147 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.820+0000 7fc9deb9c700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc9c40a1800 0x7fc9c40a3cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b5c0 tx=0x7fc9c8005fb0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.988+0000 7fc9c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:19:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.988+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/3466162884 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400b7d0 con 0x55deed9d8c00 2023-10-06T22:19:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.989+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/3466162884 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc9d4011630 con 0x55deed9d8c00 2023-10-06T22:19:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:31.989+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55deed9ccd50 con 0x55deed9d8c00 2023-10-06T22:19:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.025+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/3466162884 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9d40117f0 con 0x55deed9d8c00 2023-10-06T22:19:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.025+0000 7fc9f3fa1b80 10 client.26506 fetch_fsmap learned FSMap version 42 2023-10-06T22:19:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.025+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55deed740c00 con 0x55deed9d8c00 2023-10-06T22:19:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/3466162884 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55deed9ccd50 con 0x55deed9d8c00 2023-10-06T22:19:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/3466162884 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fc9d402a070 con 0x55deed9d8c00 2023-10-06T22:19:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9f3fa1b80 10 client.26506 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:19:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9f3fa1b80 10 client.26506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9f3fa1b80 20 client.26506 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deed9f3f00 0x55deed9f6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9f3fa1b80 4 client.26506 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:19:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55deed925db0 con 0x55deed9f3f00 2023-10-06T22:19:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.027+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deed9f3f00 0x55deed9f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.034+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deed9f3f00 0x55deed9f6320 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab31f6380d9b98c server_cookie=581699ab072c4d62 in_seq=0 out_seq=0 2023-10-06T22:19:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.035+0000 7fc9c1ffb700 10 client.26506 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.035+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/3466162884 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55deed925db0 con 0x55deed9f3f00 2023-10-06T22:19:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.035+0000 7fc9c1ffb700 10 client.26506 handle_command_reply: tid=0 2023-10-06T22:19:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.035+0000 7fc9f3fa1b80 1 client.26506 shutdown 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.044+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 msgr2=0x55deed9e3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.044+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9e3e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400baf0 tx=0x7fc9d400bf70 comp rx=0 tx=0).stop 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 shutdown_connections 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deed9f3f00 0x55deed9f6320 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9e3e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3466162884 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9e10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 >> 172.21.15.57:0/3466162884 conn(0x55deed94e200 msgr2=0x55deed9d4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9c1ffb700 0 client.26506 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 shutdown_connections 2023-10-06T22:19:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.045+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3466162884 wait complete. 2023-10-06T22:19:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.046+0000 7fc9f3fa1b80 20 client.26506 trim_cache size 0 max 16384 2023-10-06T22:19:32.074 INFO:tasks.workunit.client.0.smithi057.stdout: 245 875135 638 598542 630 180417 970 2023-10-06T22:19:32.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:32 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:19:32.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:32 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:19:32.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:32 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:19:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.505+0000 7fc9f3fa1b80 1 Processor -- start 2023-10-06T22:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.505+0000 7fc9f3fa1b80 1 -- start start 2023-10-06T22:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.506+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.506+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.506+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.506+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed7ad8c0 con 0x55deed9d9520 2023-10-06T22:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.506+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed9dbff0 con 0x55deed9e43d0 2023-10-06T22:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.506+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed9eb4c0 con 0x55deed9d8c00 2023-10-06T22:19:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.511+0000 7fc9ddb9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.511+0000 7fc9ddb9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9ef6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41282/0 (socket says 172.21.15.57:41282) 2023-10-06T22:19:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.511+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/3555374851 learned_addr learned my addr 172.21.15.57:0/3555374851 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.512+0000 7fc9deb9c700 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.512+0000 7fc9de39b700 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.512+0000 7fc9deb9c700 1 -- 172.21.15.57:0/3555374851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 msgr2=0x55deed9eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.512+0000 7fc9deb9c700 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.512+0000 7fc9deb9c700 1 -- 172.21.15.57:0/3555374851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 msgr2=0x55deed9ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.512+0000 7fc9deb9c700 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.513+0000 7fc9deb9c700 1 -- 172.21.15.57:0/3555374851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deed77e1f0 con 0x55deed9e43d0 2023-10-06T22:19:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.513+0000 7fc9deb9c700 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9dc4d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009990 tx=0x7fc9c801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b5ec5c7c8d797d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.677+0000 7fc9cd3ff700 1 -- 172.21.15.57:0/3555374851 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8029070 con 0x55deed9e43d0 2023-10-06T22:19:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.677+0000 7fc9cd3ff700 1 -- 172.21.15.57:0/3555374851 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9c800bab0 con 0x55deed9e43d0 2023-10-06T22:19:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.677+0000 7fc9cd3ff700 1 -- 172.21.15.57:0/3555374851 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c801d070 con 0x55deed9e43d0 2023-10-06T22:19:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.678+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3555374851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 msgr2=0x55deed9dc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.678+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9dc4d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009990 tx=0x7fc9c801d000 comp rx=0 tx=0).stop 2023-10-06T22:19:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.679+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3555374851 shutdown_connections 2023-10-06T22:19:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.679+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deed9dc4d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.679+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d8c00 0x55deed9eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.679+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/3555374851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d9520 0x55deed9ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.679+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3555374851 >> 172.21.15.57:0/3555374851 conn(0x55deed94e200 msgr2=0x55deed9f5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.680+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3555374851 shutdown_connections 2023-10-06T22:19:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.680+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/3555374851 wait complete. 2023-10-06T22:19:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.680+0000 7fc9f3fa1b80 1 -- start start 2023-10-06T22:19:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.681+0000 7fc9f3fa1b80 1 Processor -- start 2023-10-06T22:19:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.682+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d8c00 0x55deed9ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.682+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d9520 0x55deed9ea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.682+0000 7fc9f3fa1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deeda46cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.682+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deeda47230 con 0x55deed9d8c00 2023-10-06T22:19:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.682+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deeda47370 con 0x55deed9e43d0 2023-10-06T22:19:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.682+0000 7fc9f3fa1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deeda474b0 con 0x55deed9d9520 2023-10-06T22:19:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.682+0000 7fc9f3fa1b80 10 client.0 _ll_register_callbacks cb 0x7fc9d4009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.683+0000 7fc9ddb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d9520 0x55deed9ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.683+0000 7fc9ddb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d9520 0x55deed9ea860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:32944/0 (socket says 172.21.15.57:32944) 2023-10-06T22:19:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.683+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/2701362702 learned_addr learned my addr 172.21.15.57:0/2701362702 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.683+0000 7fc9deb9c700 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deeda46cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.684+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/2701362702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 msgr2=0x55deeda46cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.684+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deeda46cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.684+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/2701362702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d8c00 msgr2=0x55deed9ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.684+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d8c00 0x55deed9ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.684+0000 7fc9ddb9a700 1 -- 172.21.15.57:0/2701362702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d811d3b0 con 0x55deed9d9520 2023-10-06T22:19:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.684+0000 7fc9ddb9a700 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d9520 0x55deed9ea860 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0096fb0 tx=0x7fc9d009b1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=587bfee18746730b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.877+0000 7fc9c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:19:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.877+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/2701362702 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00cb6a0 con 0x55deed9d9520 2023-10-06T22:19:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.877+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/2701362702 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fc9d00a2e40 con 0x55deed9d9520 2023-10-06T22:19:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.880+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9d4011e50 con 0x55deed9d9520 2023-10-06T22:19:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.892+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/2701362702 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9d009ad10 con 0x55deed9d9520 2023-10-06T22:19:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.893+0000 7fc9f3fa1b80 10 client.26512 fetch_fsmap learned FSMap version 42 2023-10-06T22:19:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.893+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55deed756250 con 0x55deed9d9520 2023-10-06T22:19:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/2701362702 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9d4011e50 con 0x55deed9d9520 2023-10-06T22:19:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/2701362702 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fc9d00a29b0 con 0x55deed9d9520 2023-10-06T22:19:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9f3fa1b80 10 client.26512 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:19:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9f3fa1b80 10 client.26512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9f3fa1b80 20 client.26512 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deeda48ca0 0x55deeda4b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9f3fa1b80 4 client.26512 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:19:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55deed925ad0 con 0x55deeda48ca0 2023-10-06T22:19:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.903+0000 7fc9de39b700 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deeda48ca0 0x55deeda4b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.904+0000 7fc9de39b700 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deeda48ca0 0x55deeda4b0c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c6c8d92e3d4dcf4 server_cookie=97a2da25aab6e812 in_seq=0 out_seq=0 2023-10-06T22:19:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:32.908+0000 7fc9c1ffb700 10 client.26512 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:33.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:33 smithi190 ceph-mon[134891]: pgmap v147: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 131 op/s 2023-10-06T22:19:33.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:33 smithi057 ceph-mon[98492]: pgmap v147: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 131 op/s 2023-10-06T22:19:33.509 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:33 smithi156 ceph-mon[134358]: pgmap v147: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 131 op/s 2023-10-06T22:19:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.970+0000 7fc9c1ffb700 1 -- 172.21.15.57:0/2701362702 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55deed925ad0 con 0x55deeda48ca0 2023-10-06T22:19:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.970+0000 7fc9c1ffb700 10 client.26512 handle_command_reply: tid=0 2023-10-06T22:19:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.971+0000 7fc9f3fa1b80 1 client.26512 shutdown 2023-10-06T22:19:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.971+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d9520 msgr2=0x55deed9ea860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.971+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d9520 0x55deed9ea860 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0096fb0 tx=0x7fc9d009b1b0 comp rx=0 tx=0).stop 2023-10-06T22:19:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.972+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 shutdown_connections 2023-10-06T22:19:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.972+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55deeda48ca0 0x55deeda4b0c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.972+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deed9e43d0 0x55deeda46cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.972+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deed9d8c00 0x55deed9ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.972+0000 7fc9f3fa1b80 1 --2- 172.21.15.57:0/2701362702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deed9d9520 0x55deed9ea860 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.972+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 >> 172.21.15.57:0/2701362702 conn(0x55deed94e200 msgr2=0x55deed9dc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.972+0000 7fc9c1ffb700 0 client.26512 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.973+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 shutdown_connections 2023-10-06T22:19:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.973+0000 7fc9f3fa1b80 1 -- 172.21.15.57:0/2701362702 wait complete. 2023-10-06T22:19:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:33.973+0000 7fc9f3fa1b80 20 client.26512 trim_cache size 0 max 16384 2023-10-06T22:19:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:19:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:19:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:19:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:19:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.064+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc9c40a1800 msgr2=0x7fc9c40a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.064+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc9c40a1800 0x7fc9c40a3cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b5c0 tx=0x7fc9c8005fb0 comp rx=0 tx=0).stop 2023-10-06T22:19:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.064+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810fa90 msgr2=0x7fc9d81a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.064+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810fa90 0x7fc9d81a6520 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000b840 tx=0x7fc9d000bc00 comp rx=0 tx=0).stop 2023-10-06T22:19:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.065+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 shutdown_connections 2023-10-06T22:19:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.065+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc9c40a1800 0x7fc9c40a3cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.065+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9d811a690 0x7fc9d81aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.065+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d810f620 0x7fc9d81a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.065+0000 7fc9e05ff700 1 --2- 172.21.15.57:0/240502635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d810fa90 0x7fc9d81a6520 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.065+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 >> 172.21.15.57:0/240502635 conn(0x7fc9d806cfa0 msgr2=0x7fc9d8118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.073+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 shutdown_connections 2023-10-06T22:19:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:34.074+0000 7fc9e05ff700 1 -- 172.21.15.57:0/240502635 wait complete. 2023-10-06T22:19:35.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:35 smithi057 ceph-mon[98492]: pgmap v148: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 81 op/s 2023-10-06T22:19:35.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:35 smithi156 ceph-mon[134358]: pgmap v148: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 81 op/s 2023-10-06T22:19:35.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:35 smithi190 ceph-mon[134891]: pgmap v148: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 81 op/s 2023-10-06T22:19:37.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:36 smithi156 ceph-mon[134358]: pgmap v149: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2023-10-06T22:19:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:36 smithi190 ceph-mon[134891]: pgmap v149: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2023-10-06T22:19:37.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:36 smithi057 ceph-mon[98492]: pgmap v149: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2023-10-06T22:19:38.131 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:19:39.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:39 smithi190 ceph-mon[134891]: pgmap v150: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2023-10-06T22:19:39.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:39 smithi057 ceph-mon[98492]: pgmap v150: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2023-10-06T22:19:39.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:39 smithi156 ceph-mon[134358]: pgmap v150: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2023-10-06T22:19:41.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:41 smithi190 ceph-mon[134891]: pgmap v151: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 296 op/s 2023-10-06T22:19:41.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:41 smithi057 ceph-mon[98492]: pgmap v151: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 296 op/s 2023-10-06T22:19:41.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:41 smithi156 ceph-mon[134358]: pgmap v151: 97 pgs: 97 active+clean; 895 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 296 op/s 2023-10-06T22:19:42.077 INFO:tasks.workunit.client.0.smithi057.stdout: 254 852495 603 589480 567 187561 939 2023-10-06T22:19:43.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:43 smithi057 ceph-mon[98492]: pgmap v152: 97 pgs: 97 active+clean; 912 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2023-10-06T22:19:43.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:43 smithi156 ceph-mon[134358]: pgmap v152: 97 pgs: 97 active+clean; 912 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2023-10-06T22:19:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:43 smithi190 ceph-mon[134891]: pgmap v152: 97 pgs: 97 active+clean; 912 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2023-10-06T22:19:43.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:43.992+0000 7fde88b9d700 1 -- 172.21.15.57:0/3936084458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410dab0 msgr2=0x7fde8411c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:43.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:43.992+0000 7fde88b9d700 1 --2- 172.21.15.57:0/3936084458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410dab0 0x7fde8411c6a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009f90 tx=0x7fde7c00b330 comp rx=0 tx=0).stop 2023-10-06T22:19:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.015+0000 7fde88b9d700 1 -- 172.21.15.57:0/3936084458 shutdown_connections 2023-10-06T22:19:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.015+0000 7fde88b9d700 1 --2- 172.21.15.57:0/3936084458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410dab0 0x7fde8411c6a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.015+0000 7fde88b9d700 1 --2- 172.21.15.57:0/3936084458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde8410f430 0x7fde8410f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.015+0000 7fde88b9d700 1 --2- 172.21.15.57:0/3936084458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde8410cf90 0x7fde8410d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.015+0000 7fde88b9d700 1 -- 172.21.15.57:0/3936084458 >> 172.21.15.57:0/3936084458 conn(0x7fde8406d0c0 msgr2=0x7fde8406d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:44.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.035+0000 7fde88b9d700 1 -- 172.21.15.57:0/3936084458 shutdown_connections 2023-10-06T22:19:44.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.035+0000 7fde88b9d700 1 -- 172.21.15.57:0/3936084458 wait complete. 2023-10-06T22:19:44.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.037+0000 7fde88b9d700 1 Processor -- start 2023-10-06T22:19:44.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.038+0000 7fde88b9d700 1 -- start start 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.038+0000 7fde88b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410cf90 0x7fde841ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.038+0000 7fde88b9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde8410dab0 0x7fde841abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.038+0000 7fde88b9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde8410f430 0x7fde841a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.038+0000 7fde88b9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde841a60d0 con 0x7fde8410dab0 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.038+0000 7fde88b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde841a6240 con 0x7fde8410cf90 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.038+0000 7fde88b9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde841a6400 con 0x7fde8410f430 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.039+0000 7fde837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410cf90 0x7fde841ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.039+0000 7fde837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410cf90 0x7fde841ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:42998/0 (socket says 172.21.15.57:42998) 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.039+0000 7fde837fe700 1 -- 172.21.15.57:0/1733313003 learned_addr learned my addr 172.21.15.57:0/1733313003 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.039+0000 7fde83fff700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde8410f430 0x7fde841a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.040+0000 7fde837fe700 1 -- 172.21.15.57:0/1733313003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde8410f430 msgr2=0x7fde841a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.040+0000 7fde837fe700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde8410f430 0x7fde841a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.040+0000 7fde837fe700 1 -- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde8410dab0 msgr2=0x7fde841abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.040+0000 7fde837fe700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde8410dab0 0x7fde841abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.040+0000 7fde837fe700 1 -- 172.21.15.57:0/1733313003 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c009c40 con 0x7fde8410cf90 2023-10-06T22:19:44.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.040+0000 7fde837fe700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410cf90 0x7fde841ab9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fde7400e9a0 tx=0x7fde7400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:44.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.160+0000 7fde80ff9700 1 -- 172.21.15.57:0/1733313003 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7400cbd0 con 0x7fde8410cf90 2023-10-06T22:19:44.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.161+0000 7fde80ff9700 1 -- 172.21.15.57:0/1733313003 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde7400cd30 con 0x7fde8410cf90 2023-10-06T22:19:44.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.161+0000 7fde80ff9700 1 -- 172.21.15.57:0/1733313003 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74018680 con 0x7fde8410cf90 2023-10-06T22:19:44.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.171+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde841a66e0 con 0x7fde8410cf90 2023-10-06T22:19:44.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.171+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde84110050 con 0x7fde8410cf90 2023-10-06T22:19:44.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.188+0000 7fde9d674b80 1 Processor -- start 2023-10-06T22:19:44.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.190+0000 7fde9d674b80 1 -- start start 2023-10-06T22:19:44.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.191+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccd2a0 0x561fd0ccd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.191+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.191+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0cdae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.191+0000 7fde9d674b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0c36460 con 0x561fd0cd8a70 2023-10-06T22:19:44.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.191+0000 7fde9d674b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0c27c20 con 0x561fd0ccdbc0 2023-10-06T22:19:44.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.191+0000 7fde9d674b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0c25540 con 0x561fd0ccd2a0 2023-10-06T22:19:44.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.193+0000 7fde82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0cdae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.193+0000 7fde82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0cdae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37914/0 (socket says 172.21.15.57:37914) 2023-10-06T22:19:44.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.193+0000 7fde82ffd700 1 -- 172.21.15.57:0/1539479824 learned_addr learned my addr 172.21.15.57:0/1539479824 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:44.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.194+0000 7fde80ff9700 1 -- 172.21.15.57:0/1733313003 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fde740187e0 con 0x7fde8410cf90 2023-10-06T22:19:44.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.196+0000 7fde83fff700 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccd2a0 0x561fd0ccd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.196+0000 7fde837fe700 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.197+0000 7fde80ff9700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fde6c0a18c0 0x7fde6c0a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.197+0000 7fde80ff9700 1 -- 172.21.15.57:0/1733313003 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fde74014070 con 0x7fde8410cf90 2023-10-06T22:19:44.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.205+0000 7fde83fff700 1 -- 172.21.15.57:0/1539479824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccdbc0 msgr2=0x561fd0cd8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.205+0000 7fde83fff700 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.205+0000 7fde83fff700 1 -- 172.21.15.57:0/1539479824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 msgr2=0x561fd0cdae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.205+0000 7fde83fff700 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0cdae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.205+0000 7fde83fff700 1 -- 172.21.15.57:0/1539479824 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fd0a4bde0 con 0x561fd0ccd2a0 2023-10-06T22:19:44.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.206+0000 7fde83fff700 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccd2a0 0x561fd0ccd680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00b6c0 tx=0x7fde7c0093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f1bed48087f747e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:44.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.209+0000 7fde83fff700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fde6c0a18c0 0x7fde6c0a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.235+0000 7fde83fff700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fde6c0a18c0 0x7fde6c0a3d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fde7c007fd0 tx=0x7fde7c0050a0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:44.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.239+0000 7fde71ffb700 1 -- 172.21.15.57:0/1539479824 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00e070 con 0x561fd0ccd2a0 2023-10-06T22:19:44.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.239+0000 7fde71ffb700 1 -- 172.21.15.57:0/1539479824 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde7c007cd0 con 0x561fd0ccd2a0 2023-10-06T22:19:44.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.239+0000 7fde71ffb700 1 -- 172.21.15.57:0/1539479824 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00bd60 con 0x561fd0ccd2a0 2023-10-06T22:19:44.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.240+0000 7fde9d674b80 1 -- 172.21.15.57:0/1539479824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccd2a0 msgr2=0x561fd0ccd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:44.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.240+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccd2a0 0x561fd0ccd680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00b6c0 tx=0x7fde7c0093a0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.244+0000 7fde9d674b80 1 -- 172.21.15.57:0/1539479824 shutdown_connections 2023-10-06T22:19:44.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.244+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0cdae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.244+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccd2a0 0x561fd0ccd680 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.244+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1539479824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.244+0000 7fde9d674b80 1 -- 172.21.15.57:0/1539479824 >> 172.21.15.57:0/1539479824 conn(0x561fd0c42260 msgr2=0x561fd0cc9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:44.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.245+0000 7fde9d674b80 1 -- 172.21.15.57:0/1539479824 shutdown_connections 2023-10-06T22:19:44.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.247+0000 7fde9d674b80 1 -- 172.21.15.57:0/1539479824 wait complete. 2023-10-06T22:19:44.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.248+0000 7fde9d674b80 1 -- start start 2023-10-06T22:19:44.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.248+0000 7fde9d674b80 1 Processor -- start 2023-10-06T22:19:44.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.254+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.254+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.254+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0cd8a70 0x561fd0cd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.255+0000 7fde9d674b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0c20e30 con 0x561fd0ccdbc0 2023-10-06T22:19:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.255+0000 7fde9d674b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0bbd710 con 0x561fd0ccd2a0 2023-10-06T22:19:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.255+0000 7fde9d674b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0bbd3b0 con 0x561fd0cd8a70 2023-10-06T22:19:44.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.255+0000 7fde9d674b80 10 client.0 _ll_register_callbacks cb 0x561fd0a1a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.266+0000 7fde837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.266+0000 7fde837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:43022/0 (socket says 172.21.15.57:43022) 2023-10-06T22:19:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.266+0000 7fde837fe700 1 -- 172.21.15.57:0/3968117141 learned_addr learned my addr 172.21.15.57:0/3968117141 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.267+0000 7fde83fff700 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0cd8a70 0x561fd0cd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.267+0000 7fde82ffd700 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.268+0000 7fde837fe700 1 -- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0cd8a70 msgr2=0x561fd0cd6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.268+0000 7fde837fe700 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0cd8a70 0x561fd0cd6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.268+0000 7fde837fe700 1 -- 172.21.15.57:0/3968117141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccdbc0 msgr2=0x561fd0cd5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.268+0000 7fde837fe700 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.268+0000 7fde837fe700 1 -- 172.21.15.57:0/3968117141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8411d210 con 0x561fd0ccd2a0 2023-10-06T22:19:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.269+0000 7fde837fe700 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd57d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde7400c5b0 tx=0x7fde74099e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e87a014da31284 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:44.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.402+0000 7fde66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:19:44.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.402+0000 7fde66ffd700 1 -- 172.21.15.57:0/3968117141 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74099910 con 0x561fd0ccd2a0 2023-10-06T22:19:44.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.402+0000 7fde66ffd700 1 -- 172.21.15.57:0/3968117141 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fde7409a220 con 0x561fd0ccd2a0 2023-10-06T22:19:44.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.402+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fd0cc1160 con 0x561fd0ccd2a0 2023-10-06T22:19:44.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.409+0000 7fde66ffd700 1 -- 172.21.15.57:0/3968117141 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde740cf300 con 0x561fd0ccd2a0 2023-10-06T22:19:44.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.409+0000 7fde9d674b80 10 client.26546 fetch_fsmap learned FSMap version 42 2023-10-06T22:19:44.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.409+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x561fd0a34c60 con 0x561fd0ccd2a0 2023-10-06T22:19:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.421+0000 7fde66ffd700 1 -- 172.21.15.57:0/3968117141 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561fd0cc1160 con 0x561fd0ccd2a0 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.421+0000 7fde66ffd700 1 -- 172.21.15.57:0/3968117141 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fde74020bd0 con 0x561fd0ccd2a0 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.422+0000 7fde9d674b80 10 client.26546 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.422+0000 7fde9d674b80 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.422+0000 7fde9d674b80 20 client.26546 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.422+0000 7fde9d674b80 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0ce8560 0x561fd0cea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.422+0000 7fde9d674b80 4 client.26546 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.422+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fd0c19e10 con 0x561fd0ce8560 2023-10-06T22:19:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.423+0000 7fde82ffd700 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0ce8560 0x561fd0cea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.439+0000 7fde82ffd700 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0ce8560 0x561fd0cea980 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfabb61a85d5eaba server_cookie=e9dd0a5e3636db31 in_seq=0 out_seq=0 2023-10-06T22:19:44.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.439+0000 7fde66ffd700 10 client.26546 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:44.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.442+0000 7fde66ffd700 1 -- 172.21.15.57:0/3968117141 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561fd0c19e10 con 0x561fd0ce8560 2023-10-06T22:19:44.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.442+0000 7fde66ffd700 10 client.26546 handle_command_reply: tid=0 2023-10-06T22:19:44.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.442+0000 7fde9d674b80 1 client.26546 shutdown 2023-10-06T22:19:44.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 msgr2=0x561fd0cd57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:44.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd57d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde7400c5b0 tx=0x7fde74099e10 comp rx=0 tx=0).stop 2023-10-06T22:19:44.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 shutdown_connections 2023-10-06T22:19:44.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0ce8560 0x561fd0cea980 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0cd8a70 0x561fd0cd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd57d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 --2- 172.21.15.57:0/3968117141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccdbc0 0x561fd0cd5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.443+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 >> 172.21.15.57:0/3968117141 conn(0x561fd0c42260 msgr2=0x561fd0cc91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:44.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.444+0000 7fde66ffd700 0 client.26546 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:44.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.444+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 shutdown_connections 2023-10-06T22:19:44.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.444+0000 7fde9d674b80 1 -- 172.21.15.57:0/3968117141 wait complete. 2023-10-06T22:19:44.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.445+0000 7fde9d674b80 20 client.26546 trim_cache size 0 max 16384 2023-10-06T22:19:44.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.562+0000 7fde9d674b80 1 Processor -- start 2023-10-06T22:19:44.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.562+0000 7fde9d674b80 1 -- start start 2023-10-06T22:19:44.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.563+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.563+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.563+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0cd8a70 0x561fd0cd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.563+0000 7fde9d674b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0ce4f70 con 0x561fd0ccd2a0 2023-10-06T22:19:44.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.563+0000 7fde9d674b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0aa1920 con 0x561fd0cd8a70 2023-10-06T22:19:44.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.563+0000 7fde9d674b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0ccf130 con 0x561fd0ccdbc0 2023-10-06T22:19:44.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.565+0000 7fde83fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0cd8a70 0x561fd0cd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.565+0000 7fde83fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0cd8a70 0x561fd0cd2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:43028/0 (socket says 172.21.15.57:43028) 2023-10-06T22:19:44.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.565+0000 7fde83fff700 1 -- 172.21.15.57:0/1215740168 learned_addr learned my addr 172.21.15.57:0/1215740168 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:44.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.566+0000 7fde82ffd700 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.566+0000 7fde837fe700 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.569+0000 7fde83fff700 1 -- 172.21.15.57:0/1215740168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 msgr2=0x561fd0cdf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.569+0000 7fde83fff700 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.569+0000 7fde83fff700 1 -- 172.21.15.57:0/1215740168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccd2a0 msgr2=0x561fd0cd0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.569+0000 7fde83fff700 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.570+0000 7fde83fff700 1 -- 172.21.15.57:0/1215740168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fd0cef5b0 con 0x561fd0cd8a70 2023-10-06T22:19:44.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.576+0000 7fde83fff700 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0cd8a70 0x561fd0cd2a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00b370 tx=0x7fde7c02c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a434b91c74b209 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:44.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.589+0000 7fde727fc700 1 -- 172.21.15.57:0/1215740168 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00e040 con 0x561fd0cd8a70 2023-10-06T22:19:44.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.589+0000 7fde727fc700 1 -- 172.21.15.57:0/1215740168 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde7c035070 con 0x561fd0cd8a70 2023-10-06T22:19:44.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.589+0000 7fde727fc700 1 -- 172.21.15.57:0/1215740168 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c02a070 con 0x561fd0cd8a70 2023-10-06T22:19:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.593+0000 7fde9d674b80 1 -- 172.21.15.57:0/1215740168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0cd8a70 msgr2=0x561fd0cd2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.594+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0cd8a70 0x561fd0cd2a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00b370 tx=0x7fde7c02c000 comp rx=0 tx=0).stop 2023-10-06T22:19:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.594+0000 7fde9d674b80 1 -- 172.21.15.57:0/1215740168 shutdown_connections 2023-10-06T22:19:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.594+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0cd8a70 0x561fd0cd2a70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.594+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0ccd2a0 0x561fd0cd0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.594+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1215740168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.594+0000 7fde9d674b80 1 -- 172.21.15.57:0/1215740168 >> 172.21.15.57:0/1215740168 conn(0x561fd0c42260 msgr2=0x561fd0cd0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:44.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.718+0000 7fde9d674b80 1 -- 172.21.15.57:0/1215740168 shutdown_connections 2023-10-06T22:19:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.718+0000 7fde9d674b80 1 -- 172.21.15.57:0/1215740168 wait complete. 2023-10-06T22:19:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.719+0000 7fde9d674b80 1 -- start start 2023-10-06T22:19:44.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.719+0000 7fde9d674b80 1 Processor -- start 2023-10-06T22:19:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.721+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cdd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.721+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.721+0000 7fde9d674b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0d3b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.721+0000 7fde9d674b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0d3b8d0 con 0x561fd0cd8a70 2023-10-06T22:19:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.721+0000 7fde9d674b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0d3ba10 con 0x561fd0ccd2a0 2023-10-06T22:19:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.721+0000 7fde9d674b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd0d3bba0 con 0x561fd0ccdbc0 2023-10-06T22:19:44.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.721+0000 7fde9d674b80 10 client.0 _ll_register_callbacks cb 0x7fde74099ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:44.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.727+0000 7fde82ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.727+0000 7fde837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cdd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.727+0000 7fde837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cdd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:43030/0 (socket says 172.21.15.57:43030) 2023-10-06T22:19:44.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.727+0000 7fde837fe700 1 -- 172.21.15.57:0/1698725286 learned_addr learned my addr 172.21.15.57:0/1698725286 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:44.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.728+0000 7fde837fe700 1 -- 172.21.15.57:0/1698725286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 msgr2=0x561fd0cdd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.728+0000 7fde837fe700 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.728+0000 7fde837fe700 1 -- 172.21.15.57:0/1698725286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 msgr2=0x561fd0d3b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:44.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.728+0000 7fde837fe700 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0d3b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.728+0000 7fde837fe700 1 -- 172.21.15.57:0/1698725286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fd0a4bde0 con 0x561fd0ccd2a0 2023-10-06T22:19:44.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.728+0000 7fde837fe700 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cdd2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fde740a2080 tx=0x7fde74020de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30e1512d6dd434a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:44.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.861+0000 7fde66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:19:44.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.861+0000 7fde66ffd700 1 -- 172.21.15.57:0/1698725286 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740b8490 con 0x561fd0ccd2a0 2023-10-06T22:19:44.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.861+0000 7fde66ffd700 1 -- 172.21.15.57:0/1698725286 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fde7409a220 con 0x561fd0ccd2a0 2023-10-06T22:19:44.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.862+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde7c007cd0 con 0x561fd0ccd2a0 2023-10-06T22:19:44.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.863+0000 7fde66ffd700 1 -- 172.21.15.57:0/1698725286 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde740d0580 con 0x561fd0ccd2a0 2023-10-06T22:19:44.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.863+0000 7fde9d674b80 10 client.26552 fetch_fsmap learned FSMap version 42 2023-10-06T22:19:44.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.863+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x561fd0a72250 con 0x561fd0ccd2a0 2023-10-06T22:19:44.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.868+0000 7fde66ffd700 1 -- 172.21.15.57:0/1698725286 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde7c007cd0 con 0x561fd0ccd2a0 2023-10-06T22:19:44.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.868+0000 7fde66ffd700 1 -- 172.21.15.57:0/1698725286 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fde740992a0 con 0x561fd0ccd2a0 2023-10-06T22:19:44.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.869+0000 7fde9d674b80 10 client.26552 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:19:44.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.869+0000 7fde9d674b80 10 client.26552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:44.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.869+0000 7fde9d674b80 20 client.26552 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:44.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.869+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0d3cea0 0x561fd0d3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:44.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.869+0000 7fde9d674b80 4 client.26552 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:19:44.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.869+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561fd0c19b30 con 0x561fd0d3cea0 2023-10-06T22:19:44.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.870+0000 7fde82ffd700 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0d3cea0 0x561fd0d3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.885+0000 7fde82ffd700 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0d3cea0 0x561fd0d3f290 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf7132a6e3e0efd server_cookie=2d2e85f45dff87d2 in_seq=0 out_seq=0 2023-10-06T22:19:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.886+0000 7fde66ffd700 10 client.26552 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:44.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.886+0000 7fde66ffd700 1 -- 172.21.15.57:0/1698725286 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561fd0c19b30 con 0x561fd0d3cea0 2023-10-06T22:19:44.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.886+0000 7fde66ffd700 10 client.26552 handle_command_reply: tid=0 2023-10-06T22:19:44.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.886+0000 7fde9d674b80 1 client.26552 shutdown 2023-10-06T22:19:44.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 msgr2=0x561fd0cdd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:44.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cdd2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fde740a2080 tx=0x7fde74020de0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 shutdown_connections 2023-10-06T22:19:44.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561fd0d3cea0 0x561fd0d3f290 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561fd0cd8a70 0x561fd0d3b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561fd0ccd2a0 0x561fd0cdd2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 --2- 172.21.15.57:0/1698725286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561fd0ccdbc0 0x561fd0cdd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:44.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.887+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 >> 172.21.15.57:0/1698725286 conn(0x561fd0c42260 msgr2=0x561fd0cd5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:44.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.888+0000 7fde66ffd700 0 client.26552 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:44.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.888+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 shutdown_connections 2023-10-06T22:19:44.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.888+0000 7fde9d674b80 1 -- 172.21.15.57:0/1698725286 wait complete. 2023-10-06T22:19:44.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.889+0000 7fde9d674b80 20 client.26552 trim_cache size 0 max 16384 2023-10-06T22:19:44.891 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:19:44.892 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:19:45.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.999+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fde6c0a18c0 msgr2=0x7fde6c0a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:45.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.999+0000 7fde88b9d700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fde6c0a18c0 0x7fde6c0a3d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fde7c007fd0 tx=0x7fde7c0050a0 comp rx=0 tx=0).stop 2023-10-06T22:19:45.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.999+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410cf90 msgr2=0x7fde841ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:45.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:44.999+0000 7fde88b9d700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410cf90 0x7fde841ab9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fde7400e9a0 tx=0x7fde7400ed60 comp rx=0 tx=0).stop 2023-10-06T22:19:45.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.013+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 shutdown_connections 2023-10-06T22:19:45.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.013+0000 7fde88b9d700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fde6c0a18c0 0x7fde6c0a3d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:45.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.013+0000 7fde88b9d700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde8410f430 0x7fde841a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:45.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.013+0000 7fde88b9d700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8410cf90 0x7fde841ab9f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:45.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.013+0000 7fde88b9d700 1 --2- 172.21.15.57:0/1733313003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde8410dab0 0x7fde841abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:45.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.013+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 >> 172.21.15.57:0/1733313003 conn(0x7fde8406d0c0 msgr2=0x7fde840705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:45.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.013+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 shutdown_connections 2023-10-06T22:19:45.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:19:45.014+0000 7fde88b9d700 1 -- 172.21.15.57:0/1733313003 wait complete. 2023-10-06T22:19:45.184 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:19:45.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:45 smithi156 ceph-mon[134358]: pgmap v153: 97 pgs: 97 active+clean; 913 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2023-10-06T22:19:45.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:45 smithi190 ceph-mon[134891]: pgmap v153: 97 pgs: 97 active+clean; 913 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2023-10-06T22:19:45.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:45 smithi057 ceph-mon[98492]: pgmap v153: 97 pgs: 97 active+clean; 913 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2023-10-06T22:19:47.253 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:46 smithi156 ceph-mon[134358]: pgmap v154: 97 pgs: 97 active+clean; 913 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2023-10-06T22:19:47.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:46 smithi190 ceph-mon[134891]: pgmap v154: 97 pgs: 97 active+clean; 913 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2023-10-06T22:19:47.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:46 smithi057 ceph-mon[98492]: pgmap v154: 97 pgs: 97 active+clean; 913 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2023-10-06T22:19:49.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:49 smithi190 ceph-mon[134891]: pgmap v155: 97 pgs: 97 active+clean; 954 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 508 op/s 2023-10-06T22:19:49.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:49 smithi057 ceph-mon[98492]: pgmap v155: 97 pgs: 97 active+clean; 954 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 508 op/s 2023-10-06T22:19:49.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:49 smithi156 ceph-mon[134358]: pgmap v155: 97 pgs: 97 active+clean; 954 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 508 op/s 2023-10-06T22:19:51.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:51 smithi190 ceph-mon[134891]: pgmap v156: 97 pgs: 97 active+clean; 954 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 257 op/s 2023-10-06T22:19:51.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:51 smithi057 ceph-mon[98492]: pgmap v156: 97 pgs: 97 active+clean; 954 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 257 op/s 2023-10-06T22:19:51.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:51 smithi156 ceph-mon[134358]: pgmap v156: 97 pgs: 97 active+clean; 954 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 257 op/s 2023-10-06T22:19:52.077 INFO:tasks.workunit.client.0.smithi057.stdout: 266 810517 623 553802 685 192733 813 2023-10-06T22:19:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.185+0000 7f063d7ff700 1 -- 172.21.15.57:0/670012997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 msgr2=0x7f063811ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.185+0000 7f063d7ff700 1 --2- 172.21.15.57:0/670012997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 0x7f063811ca80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0628009990 tx=0x7f0628009ca0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.186+0000 7f063d7ff700 1 -- 172.21.15.57:0/670012997 shutdown_connections 2023-10-06T22:19:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.186+0000 7f063d7ff700 1 --2- 172.21.15.57:0/670012997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 0x7f063811ca80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.186+0000 7f063d7ff700 1 --2- 172.21.15.57:0/670012997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f063810f620 0x7f0638107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.186+0000 7f063d7ff700 1 --2- 172.21.15.57:0/670012997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063810fa90 0x7f06381083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.186+0000 7f063d7ff700 1 -- 172.21.15.57:0/670012997 >> 172.21.15.57:0/670012997 conn(0x7f063806cfa0 msgr2=0x7f063806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.186+0000 7f063d7ff700 1 -- 172.21.15.57:0/670012997 shutdown_connections 2023-10-06T22:19:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.187+0000 7f063d7ff700 1 -- 172.21.15.57:0/670012997 wait complete. 2023-10-06T22:19:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.190+0000 7f063d7ff700 1 Processor -- start 2023-10-06T22:19:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.190+0000 7f063d7ff700 1 -- start start 2023-10-06T22:19:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.191+0000 7f063d7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063810f620 0x7f0638119b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.191+0000 7f063d7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f063810fa90 0x7f063811a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.191+0000 7f063d7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 0x7f0638113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.191+0000 7f063d7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638114120 con 0x7f063810fa90 2023-10-06T22:19:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.191+0000 7f063d7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638114290 con 0x7f063810f620 2023-10-06T22:19:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.191+0000 7f063d7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638114450 con 0x7f063811a690 2023-10-06T22:19:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.192+0000 7f06367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f063810fa90 0x7f063811a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.200+0000 7f06367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f063810fa90 0x7f063811a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46896/0 (socket says 172.21.15.57:46896) 2023-10-06T22:19:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.200+0000 7f06367fc700 1 -- 172.21.15.57:0/3396241986 learned_addr learned my addr 172.21.15.57:0/3396241986 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.200+0000 7f0636ffd700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063810f620 0x7f0638119b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.201+0000 7f06377fe700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 0x7f0638113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.201+0000 7f0636ffd700 1 -- 172.21.15.57:0/3396241986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 msgr2=0x7f0638113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.201+0000 7f0636ffd700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 0x7f0638113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.201+0000 7f0636ffd700 1 -- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f063810fa90 msgr2=0x7f063811a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.201+0000 7f0636ffd700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f063810fa90 0x7f063811a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.201+0000 7f0636ffd700 1 -- 172.21.15.57:0/3396241986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280095f0 con 0x7f063810f620 2023-10-06T22:19:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.207+0000 7f0636ffd700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063810f620 0x7f0638119b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f063000ef10 tx=0x7f063000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.286+0000 7f0627fff700 1 -- 172.21.15.57:0/3396241986 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000cad0 con 0x7f063810f620 2023-10-06T22:19:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.286+0000 7f0627fff700 1 -- 172.21.15.57:0/3396241986 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063000f040 con 0x7f063810f620 2023-10-06T22:19:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.286+0000 7f0627fff700 1 -- 172.21.15.57:0/3396241986 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300146e0 con 0x7f063810f620 2023-10-06T22:19:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.286+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06381146d0 con 0x7f063810f620 2023-10-06T22:19:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.287+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06381bb590 con 0x7f063810f620 2023-10-06T22:19:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.307+0000 7f0651196b80 1 Processor -- start 2023-10-06T22:19:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.308+0000 7f0651196b80 1 -- start start 2023-10-06T22:19:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.309+0000 7f0651196b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.309+0000 7f0651196b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.309+0000 7f0651196b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.309+0000 7f0651196b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f954400 con 0x557b3f9eb520 2023-10-06T22:19:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.309+0000 7f0651196b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f945bc0 con 0x557b3f9f63d0 2023-10-06T22:19:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.309+0000 7f0651196b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f9434e0 con 0x557b3f9eac00 2023-10-06T22:19:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.309+0000 7f06377fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.311+0000 7f06367fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.311+0000 7f06367fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:33606/0 (socket says 172.21.15.57:33606) 2023-10-06T22:19:52.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.311+0000 7f06367fc700 1 -- 172.21.15.57:0/3066369216 learned_addr learned my addr 172.21.15.57:0/3066369216 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:52.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.314+0000 7f0627fff700 1 -- 172.21.15.57:0/3396241986 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f063000cc30 con 0x7f063810f620 2023-10-06T22:19:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.318+0000 7f0627fff700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06200a1800 0x7f06200a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.318+0000 7f0627fff700 1 -- 172.21.15.57:0/3396241986 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f06300cd420 con 0x7f063810f620 2023-10-06T22:19:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.319+0000 7f0636ffd700 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.320+0000 7f06377fe700 1 -- 172.21.15.57:0/3066369216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 msgr2=0x557b3f9f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.320+0000 7f06377fe700 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.320+0000 7f06377fe700 1 -- 172.21.15.57:0/3066369216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 msgr2=0x557b3f9f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.320+0000 7f06377fe700 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.320+0000 7f06377fe700 1 -- 172.21.15.57:0/3066369216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b3f769d80 con 0x557b3f9eac00 2023-10-06T22:19:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.330+0000 7f06377fe700 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9eafe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06280098e0 tx=0x7f062800b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86b56987c5a8010 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.330+0000 7f06377fe700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06200a1800 0x7f06200a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.356+0000 7f06377fe700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06200a1800 0x7f06200a3cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f062800b2e0 tx=0x7f0628004ef0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.464+0000 7f06257fa700 1 -- 172.21.15.57:0/3066369216 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628027070 con 0x557b3f9eac00 2023-10-06T22:19:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.464+0000 7f06257fa700 1 -- 172.21.15.57:0/3066369216 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f062801e040 con 0x557b3f9eac00 2023-10-06T22:19:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.464+0000 7f06257fa700 1 -- 172.21.15.57:0/3066369216 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628004c90 con 0x557b3f9eac00 2023-10-06T22:19:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.466+0000 7f0651196b80 1 -- 172.21.15.57:0/3066369216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 msgr2=0x557b3f9eafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.466+0000 7f0651196b80 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9eafe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06280098e0 tx=0x7f062800b830 comp rx=0 tx=0).stop 2023-10-06T22:19:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.467+0000 7f0651196b80 1 -- 172.21.15.57:0/3066369216 shutdown_connections 2023-10-06T22:19:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.467+0000 7f0651196b80 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.467+0000 7f0651196b80 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9eafe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.467+0000 7f0651196b80 1 --2- 172.21.15.57:0/3066369216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.467+0000 7f0651196b80 1 -- 172.21.15.57:0/3066369216 >> 172.21.15.57:0/3066369216 conn(0x557b3f960200 msgr2=0x557b3f9e7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.467+0000 7f0651196b80 1 -- 172.21.15.57:0/3066369216 shutdown_connections 2023-10-06T22:19:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.468+0000 7f0651196b80 1 -- 172.21.15.57:0/3066369216 wait complete. 2023-10-06T22:19:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.468+0000 7f0651196b80 1 -- start start 2023-10-06T22:19:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.469+0000 7f0651196b80 1 Processor -- start 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.470+0000 7f0651196b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.470+0000 7f0651196b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.470+0000 7f0651196b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.470+0000 7f0651196b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f93edd0 con 0x557b3f9eb520 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.470+0000 7f0651196b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f8db6b0 con 0x557b3f9f63d0 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.470+0000 7f0651196b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f8db350 con 0x557b3f9eac00 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.470+0000 7f0651196b80 10 client.0 _ll_register_callbacks cb 0x557b3f7387a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.471+0000 7f06367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.471+0000 7f06367fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46922/0 (socket says 172.21.15.57:46922) 2023-10-06T22:19:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.471+0000 7f06367fc700 1 -- 172.21.15.57:0/2438197501 learned_addr learned my addr 172.21.15.57:0/2438197501 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.471+0000 7f06377fe700 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.473+0000 7f06367fc700 1 -- 172.21.15.57:0/2438197501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 msgr2=0x557b3f9f5d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:19:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.473+0000 7f06367fc700 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9f5d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.473+0000 7f06367fc700 1 -- 172.21.15.57:0/2438197501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 msgr2=0x557b3f9f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.473+0000 7f06367fc700 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.473+0000 7f06367fc700 1 -- 172.21.15.57:0/2438197501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063811d3b0 con 0x557b3f9eb520 2023-10-06T22:19:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.481+0000 7f06367fc700 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f2ff0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f062c010500 tx=0x7f062c010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774b9e11adc0a9d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.491+0000 7f061e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:19:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.492+0000 7f061e7fc700 1 -- 172.21.15.57:0/2438197501 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c007670 con 0x557b3f9eb520 2023-10-06T22:19:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.492+0000 7f061e7fc700 1 -- 172.21.15.57:0/2438197501 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f062c014640 con 0x557b3f9eb520 2023-10-06T22:19:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.492+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b3f9ded50 con 0x557b3f9eb520 2023-10-06T22:19:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.499+0000 7f061e7fc700 1 -- 172.21.15.57:0/2438197501 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062c014800 con 0x557b3f9eb520 2023-10-06T22:19:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.500+0000 7f0651196b80 10 client.16530 fetch_fsmap learned FSMap version 42 2023-10-06T22:19:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.500+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557b3f752c00 con 0x557b3f9eb520 2023-10-06T22:19:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.503+0000 7f061e7fc700 1 -- 172.21.15.57:0/2438197501 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b3f9ded50 con 0x557b3f9eb520 2023-10-06T22:19:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.503+0000 7f061e7fc700 1 -- 172.21.15.57:0/2438197501 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f062c01dc30 con 0x557b3f9eb520 2023-10-06T22:19:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.505+0000 7f0651196b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:19:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.505+0000 7f0651196b80 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.505+0000 7f0651196b80 20 client.16530 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.505+0000 7f0651196b80 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa05fb0 0x557b3fa083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.505+0000 7f0651196b80 4 client.16530 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:19:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.505+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b3f937db0 con 0x557b3fa05fb0 2023-10-06T22:19:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.506+0000 7f0636ffd700 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa05fb0 0x557b3fa083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.534+0000 7f0636ffd700 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa05fb0 0x557b3fa083d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53797766c93794ff server_cookie=c312b911f8210dca in_seq=0 out_seq=0 2023-10-06T22:19:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.539+0000 7f061e7fc700 10 client.16530 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.543+0000 7f061e7fc700 1 -- 172.21.15.57:0/2438197501 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b3f937db0 con 0x557b3fa05fb0 2023-10-06T22:19:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.543+0000 7f061e7fc700 10 client.16530 handle_command_reply: tid=0 2023-10-06T22:19:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.544+0000 7f0651196b80 1 client.16530 shutdown 2023-10-06T22:19:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 msgr2=0x557b3f9f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f2ff0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f062c010500 tx=0x7f062c010810 comp rx=0 tx=0).stop 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 shutdown_connections 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa05fb0 0x557b3fa083d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 --2- 172.21.15.57:0/2438197501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9f2ff0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 >> 172.21.15.57:0/2438197501 conn(0x557b3f960200 msgr2=0x557b3f9e6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.546+0000 7f061e7fc700 0 client.16530 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.547+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 shutdown_connections 2023-10-06T22:19:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.547+0000 7f0651196b80 1 -- 172.21.15.57:0/2438197501 wait complete. 2023-10-06T22:19:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.548+0000 7f0651196b80 20 client.16530 trim_cache size 0 max 16384 2023-10-06T22:19:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.600+0000 7f0651196b80 1 Processor -- start 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.600+0000 7f0651196b80 1 -- start start 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.601+0000 7f0651196b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eac00 0x557b3f9fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.601+0000 7f0651196b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9eb520 0x557b3f9f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.601+0000 7f0651196b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9f63d0 0x557b3f9f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.601+0000 7f0651196b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f7bf8c0 con 0x557b3f9eac00 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.601+0000 7f0651196b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f9de890 con 0x557b3f9eb520 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.601+0000 7f0651196b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f9ec3f0 con 0x557b3f9f63d0 2023-10-06T22:19:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.602+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eac00 0x557b3f9fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.602+0000 7f06377fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9f63d0 0x557b3f9f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.602+0000 7f06377fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9f63d0 0x557b3f9f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:42738/0 (socket says 172.21.15.57:42738) 2023-10-06T22:19:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.602+0000 7f06377fe700 1 -- 172.21.15.57:0/2113446068 learned_addr learned my addr 172.21.15.57:0/2113446068 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.603+0000 7f06367fc700 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9eb520 0x557b3f9f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.603+0000 7f06377fe700 1 -- 172.21.15.57:0/2113446068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9eb520 msgr2=0x557b3f9f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.603+0000 7f06377fe700 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9eb520 0x557b3f9f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.603+0000 7f06377fe700 1 -- 172.21.15.57:0/2113446068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eac00 msgr2=0x557b3f9fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.603+0000 7f06377fe700 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eac00 0x557b3f9fcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.603+0000 7f06377fe700 1 -- 172.21.15.57:0/2113446068 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b3f7901f0 con 0x557b3f9f63d0 2023-10-06T22:19:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.604+0000 7f06377fe700 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9f63d0 0x557b3f9f27b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f062800b870 tx=0x7f062801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25313d1ef2833169 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.623+0000 7f0625ffb700 1 -- 172.21.15.57:0/2113446068 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628027040 con 0x557b3f9f63d0 2023-10-06T22:19:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.623+0000 7f0625ffb700 1 -- 172.21.15.57:0/2113446068 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0628009290 con 0x557b3f9f63d0 2023-10-06T22:19:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.623+0000 7f0625ffb700 1 -- 172.21.15.57:0/2113446068 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628004980 con 0x557b3f9f63d0 2023-10-06T22:19:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.625+0000 7f0651196b80 1 -- 172.21.15.57:0/2113446068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9f63d0 msgr2=0x557b3f9f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.625+0000 7f0651196b80 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9f63d0 0x557b3f9f27b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f062800b870 tx=0x7f062801c000 comp rx=0 tx=0).stop 2023-10-06T22:19:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.630+0000 7f0651196b80 1 -- 172.21.15.57:0/2113446068 shutdown_connections 2023-10-06T22:19:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.630+0000 7f0651196b80 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9f63d0 0x557b3f9f27b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.630+0000 7f0651196b80 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eac00 0x557b3f9fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.630+0000 7f0651196b80 1 --2- 172.21.15.57:0/2113446068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9eb520 0x557b3f9f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.630+0000 7f0651196b80 1 -- 172.21.15.57:0/2113446068 >> 172.21.15.57:0/2113446068 conn(0x557b3f960200 msgr2=0x557b3f9ee2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.631+0000 7f0651196b80 1 -- 172.21.15.57:0/2113446068 shutdown_connections 2023-10-06T22:19:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.631+0000 7f0651196b80 1 -- 172.21.15.57:0/2113446068 wait complete. 2023-10-06T22:19:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.633+0000 7f0651196b80 1 -- start start 2023-10-06T22:19:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.634+0000 7f0651196b80 1 Processor -- start 2023-10-06T22:19:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.634+0000 7f0651196b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0651196b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0651196b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0651196b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f9f02d0 con 0x557b3f9eb520 2023-10-06T22:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0651196b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f9f0440 con 0x557b3f9f63d0 2023-10-06T22:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0651196b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3f9f0600 con 0x557b3f9eac00 2023-10-06T22:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0651196b80 10 client.0 _ll_register_callbacks cb 0x7f062c007b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9ef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:42752/0 (socket says 172.21.15.57:42752) 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f0636ffd700 1 -- 172.21.15.57:0/3525643007 learned_addr learned my addr 172.21.15.57:0/3525643007 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.636+0000 7f06377fe700 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.637+0000 7f06377fe700 1 -- 172.21.15.57:0/3525643007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 msgr2=0x557b3f9ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.637+0000 7f06377fe700 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.637+0000 7f06377fe700 1 -- 172.21.15.57:0/3525643007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 msgr2=0x557b3f9ef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.637+0000 7f06377fe700 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.637+0000 7f06377fe700 1 -- 172.21.15.57:0/3525643007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b3f769d80 con 0x557b3f9f63d0 2023-10-06T22:19:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.643+0000 7f06377fe700 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9efcf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0628011040 tx=0x7f062800f650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b369c43a7bff6da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:19:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.666+0000 7f061e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:19:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.666+0000 7f061e7fc700 1 -- 172.21.15.57:0/3525643007 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628027040 con 0x557b3f9f63d0 2023-10-06T22:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.666+0000 7f061e7fc700 1 -- 172.21.15.57:0/3525643007 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f0628005600 con 0x557b3f9f63d0 2023-10-06T22:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.666+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f062801e040 con 0x557b3f9f63d0 2023-10-06T22:19:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.670+0000 7f061e7fc700 1 -- 172.21.15.57:0/3525643007 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06280045a0 con 0x557b3f9f63d0 2023-10-06T22:19:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.670+0000 7f0651196b80 10 client.26564 fetch_fsmap learned FSMap version 42 2023-10-06T22:19:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.670+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557b3f768250 con 0x557b3f9f63d0 2023-10-06T22:19:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.674+0000 7f061e7fc700 1 -- 172.21.15.57:0/3525643007 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f062801e040 con 0x557b3f9f63d0 2023-10-06T22:19:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.675+0000 7f061e7fc700 1 -- 172.21.15.57:0/3525643007 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f062804d070 con 0x557b3f9f63d0 2023-10-06T22:19:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.676+0000 7f0651196b80 10 client.26564 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:19:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.676+0000 7f0651196b80 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:19:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.676+0000 7f0651196b80 20 client.26564 populate_metadata read hostname 'smithi057' 2023-10-06T22:19:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.676+0000 7f0651196b80 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa5a4b0 0x557b3fa5c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.676+0000 7f0651196b80 4 client.26564 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.676+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b3f937ad0 con 0x557b3fa5a4b0 2023-10-06T22:19:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.678+0000 7f0636ffd700 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa5a4b0 0x557b3fa5c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.687+0000 7f0636ffd700 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa5a4b0 0x557b3fa5c8d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8151be81b61fa620 server_cookie=b2114f7c54b07355 in_seq=0 out_seq=0 2023-10-06T22:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:52.687+0000 7f061e7fc700 10 client.26564 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:53.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:53 smithi057 ceph-mon[98492]: pgmap v157: 97 pgs: 97 active+clean; 974 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 352 op/s 2023-10-06T22:19:53.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:53 smithi156 ceph-mon[134358]: pgmap v157: 97 pgs: 97 active+clean; 974 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 352 op/s 2023-10-06T22:19:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.545+0000 7f061e7fc700 1 -- 172.21.15.57:0/3525643007 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b3f937ad0 con 0x557b3fa5a4b0 2023-10-06T22:19:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.545+0000 7f061e7fc700 10 client.26564 handle_command_reply: tid=0 2023-10-06T22:19:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.545+0000 7f0651196b80 1 client.26564 shutdown 2023-10-06T22:19:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.546+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 msgr2=0x557b3f9efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.546+0000 7f0651196b80 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9efcf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0628011040 tx=0x7f062800f650 comp rx=0 tx=0).stop 2023-10-06T22:19:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 shutdown_connections 2023-10-06T22:19:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f0651196b80 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557b3fa5a4b0 0x557b3fa5c8d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f0651196b80 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b3f9f63d0 0x557b3f9efcf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f0651196b80 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b3f9eac00 0x557b3f9ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f0651196b80 1 --2- 172.21.15.57:0/3525643007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b3f9eb520 0x557b3f9ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 >> 172.21.15.57:0/3525643007 conn(0x557b3f960200 msgr2=0x557b3f9ecc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f061e7fc700 0 client.26564 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:19:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.547+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 shutdown_connections 2023-10-06T22:19:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.548+0000 7f0651196b80 1 -- 172.21.15.57:0/3525643007 wait complete. 2023-10-06T22:19:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.548+0000 7f0651196b80 20 client.26564 trim_cache size 0 max 16384 2023-10-06T22:19:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:19:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:19:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:19:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:19:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:53 smithi190 ceph-mon[134891]: pgmap v157: 97 pgs: 97 active+clean; 974 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 352 op/s 2023-10-06T22:19:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.654+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06200a1800 msgr2=0x7f06200a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.654+0000 7f063d7ff700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06200a1800 0x7f06200a3cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f062800b2e0 tx=0x7f0628004ef0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.654+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063810f620 msgr2=0x7f0638119b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:19:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.654+0000 7f063d7ff700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063810f620 0x7f0638119b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f063000ef10 tx=0x7f063000c3a0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.655+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 shutdown_connections 2023-10-06T22:19:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.655+0000 7f063d7ff700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f06200a1800 0x7f06200a3cc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.655+0000 7f063d7ff700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f063811a690 0x7f0638113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.655+0000 7f063d7ff700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063810f620 0x7f0638119b00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.655+0000 7f063d7ff700 1 --2- 172.21.15.57:0/3396241986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f063810fa90 0x7f063811a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:19:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.655+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 >> 172.21.15.57:0/3396241986 conn(0x7f063806cfa0 msgr2=0x7f06380718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:19:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.656+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 shutdown_connections 2023-10-06T22:19:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:19:53.657+0000 7f063d7ff700 1 -- 172.21.15.57:0/3396241986 wait complete. 2023-10-06T22:19:55.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:55 smithi057 ceph-mon[98492]: pgmap v158: 97 pgs: 97 active+clean; 980 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 312 op/s 2023-10-06T22:19:55.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:55 smithi156 ceph-mon[134358]: pgmap v158: 97 pgs: 97 active+clean; 980 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 312 op/s 2023-10-06T22:19:55.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:55 smithi190 ceph-mon[134891]: pgmap v158: 97 pgs: 97 active+clean; 980 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 312 op/s 2023-10-06T22:19:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:57 smithi057 ceph-mon[98492]: pgmap v159: 97 pgs: 97 active+clean; 980 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 307 op/s 2023-10-06T22:19:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:57 smithi156 ceph-mon[134358]: pgmap v159: 97 pgs: 97 active+clean; 980 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 307 op/s 2023-10-06T22:19:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:57 smithi190 ceph-mon[134891]: pgmap v159: 97 pgs: 97 active+clean; 980 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 307 op/s 2023-10-06T22:19:59.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:19:59 smithi057 ceph-mon[98492]: pgmap v160: 97 pgs: 97 active+clean; 996 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2023-10-06T22:19:59.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:19:59 smithi156 ceph-mon[134358]: pgmap v160: 97 pgs: 97 active+clean; 996 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2023-10-06T22:19:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:19:59 smithi190 ceph-mon[134891]: pgmap v160: 97 pgs: 97 active+clean; 996 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2023-10-06T22:20:00.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:00 smithi057 ceph-mon[98492]: overall HEALTH_OK 2023-10-06T22:20:00.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:00 smithi156 ceph-mon[134358]: overall HEALTH_OK 2023-10-06T22:20:00.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:00 smithi190 ceph-mon[134891]: overall HEALTH_OK 2023-10-06T22:20:01.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:01 smithi057 ceph-mon[98492]: pgmap v161: 97 pgs: 97 active+clean; 996 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2023-10-06T22:20:01.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:01 smithi156 ceph-mon[134358]: pgmap v161: 97 pgs: 97 active+clean; 996 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2023-10-06T22:20:01.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:01 smithi190 ceph-mon[134891]: pgmap v161: 97 pgs: 97 active+clean; 996 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2023-10-06T22:20:02.079 INFO:tasks.workunit.client.0.smithi057.stdout: 275 815135 728 565417 553 207570 654 2023-10-06T22:20:03.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:03 smithi156 ceph-mon[134358]: pgmap v162: 97 pgs: 97 active+clean; 1014 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 555 op/s 2023-10-06T22:20:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:03 smithi190 ceph-mon[134891]: pgmap v162: 97 pgs: 97 active+clean; 1014 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 555 op/s 2023-10-06T22:20:03.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:03 smithi057 ceph-mon[98492]: pgmap v162: 97 pgs: 97 active+clean; 1014 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 555 op/s 2023-10-06T22:20:05.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:05 smithi156 ceph-mon[134358]: pgmap v163: 97 pgs: 97 active+clean; 1023 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 500 op/s 2023-10-06T22:20:05.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:05 smithi190 ceph-mon[134891]: pgmap v163: 97 pgs: 97 active+clean; 1023 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 500 op/s 2023-10-06T22:20:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:05 smithi057 ceph-mon[98492]: pgmap v163: 97 pgs: 97 active+clean; 1023 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 500 op/s 2023-10-06T22:20:07.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:07 smithi190 ceph-mon[134891]: pgmap v164: 97 pgs: 97 active+clean; 1023 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 462 op/s 2023-10-06T22:20:07.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:07 smithi057 ceph-mon[98492]: pgmap v164: 97 pgs: 97 active+clean; 1023 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 462 op/s 2023-10-06T22:20:07.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:07 smithi156 ceph-mon[134358]: pgmap v164: 97 pgs: 97 active+clean; 1023 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 462 op/s 2023-10-06T22:20:08.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:08 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:20:08.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:08 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:20:08.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:08 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:20:09.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:09 smithi057 ceph-mon[98492]: pgmap v165: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 580 op/s 2023-10-06T22:20:09.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:09 smithi156 ceph-mon[134358]: pgmap v165: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 580 op/s 2023-10-06T22:20:09.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:09 smithi190 ceph-mon[134891]: pgmap v165: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.8 MiB/s wr, 580 op/s 2023-10-06T22:20:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.740+0000 7fd1d33ff700 1 -- 172.21.15.57:0/2697535133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc10d520 msgr2=0x7fd1cc10d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.740+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/2697535133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc10d520 0x7fd1cc10d900 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4007ff0 tx=0x7fd1c4008020 comp rx=0 tx=0).stop 2023-10-06T22:20:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.741+0000 7fd1d33ff700 1 -- 172.21.15.57:0/2697535133 shutdown_connections 2023-10-06T22:20:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.741+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/2697535133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc108870 0x7fd1cc079110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.741+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/2697535133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc10d520 0x7fd1cc10d900 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.741+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/2697535133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1cc107eb0 0x7fd1cc108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.741+0000 7fd1d33ff700 1 -- 172.21.15.57:0/2697535133 >> 172.21.15.57:0/2697535133 conn(0x7fd1cc06cfa0 msgr2=0x7fd1cc06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.759+0000 7fd1d33ff700 1 -- 172.21.15.57:0/2697535133 shutdown_connections 2023-10-06T22:20:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.759+0000 7fd1d33ff700 1 -- 172.21.15.57:0/2697535133 wait complete. 2023-10-06T22:20:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.761+0000 7fd1d33ff700 1 Processor -- start 2023-10-06T22:20:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.761+0000 7fd1d33ff700 1 -- start start 2023-10-06T22:20:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.762+0000 7fd1d33ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc107eb0 0x7fd1cc082690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.762+0000 7fd1d33ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc108870 0x7fd1cc082bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.762+0000 7fd1d33ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1cc10d520 0x7fd1cc07c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.762+0000 7fd1d33ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc07cd10 con 0x7fd1cc107eb0 2023-10-06T22:20:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.762+0000 7fd1d33ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc07ce80 con 0x7fd1cc108870 2023-10-06T22:20:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.762+0000 7fd1d33ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc07d040 con 0x7fd1cc10d520 2023-10-06T22:20:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.763+0000 7fd1d119b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc107eb0 0x7fd1cc082690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.763+0000 7fd1d119b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc107eb0 0x7fd1cc082690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48222/0 (socket says 172.21.15.57:48222) 2023-10-06T22:20:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.763+0000 7fd1d119b700 1 -- 172.21.15.57:0/4013430745 learned_addr learned my addr 172.21.15.57:0/4013430745 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.768+0000 7fd1d099a700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc108870 0x7fd1cc082bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.769+0000 7fd1d199c700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1cc10d520 0x7fd1cc07c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.776+0000 7fd1d119b700 1 -- 172.21.15.57:0/4013430745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1cc10d520 msgr2=0x7fd1cc07c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.776+0000 7fd1d119b700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1cc10d520 0x7fd1cc07c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.776+0000 7fd1d119b700 1 -- 172.21.15.57:0/4013430745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc108870 msgr2=0x7fd1cc082bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.776+0000 7fd1d119b700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc108870 0x7fd1cc082bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.776+0000 7fd1d119b700 1 -- 172.21.15.57:0/4013430745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4007c20 con 0x7fd1cc107eb0 2023-10-06T22:20:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.790+0000 7fd1d119b700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc107eb0 0x7fd1cc082690 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4004fd0 tx=0x7fd1c400f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.837+0000 7fd1c27fc700 1 -- 172.21.15.57:0/4013430745 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400fec0 con 0x7fd1cc107eb0 2023-10-06T22:20:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.837+0000 7fd1c27fc700 1 -- 172.21.15.57:0/4013430745 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c40108f0 con 0x7fd1cc107eb0 2023-10-06T22:20:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.837+0000 7fd1c27fc700 1 -- 172.21.15.57:0/4013430745 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c401a8b0 con 0x7fd1cc107eb0 2023-10-06T22:20:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.840+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc07d2c0 con 0x7fd1cc107eb0 2023-10-06T22:20:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.840+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc1cc440 con 0x7fd1cc107eb0 2023-10-06T22:20:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.861+0000 7fd1e6cdfb80 1 Processor -- start 2023-10-06T22:20:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.861+0000 7fd1e6cdfb80 1 -- start start 2023-10-06T22:20:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.862+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa28c00 0x5603caa28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.862+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.862+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.862+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ca992400 con 0x5603caa29520 2023-10-06T22:20:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.862+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ca983bc0 con 0x5603caa28c00 2023-10-06T22:20:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.862+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ca9814e0 con 0x5603caa343d0 2023-10-06T22:20:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.863+0000 7fd1c27fc700 1 -- 172.21.15.57:0/4013430745 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd1c401aa10 con 0x7fd1cc107eb0 2023-10-06T22:20:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.865+0000 7fd1c27fc700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd1b80a1800 0x7fd1b80a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.865+0000 7fd1c27fc700 1 -- 172.21.15.57:0/4013430745 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd1c40cf860 con 0x7fd1cc107eb0 2023-10-06T22:20:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.868+0000 7fd1d119b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.868+0000 7fd1d099a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa367c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58518/0 (socket says 172.21.15.57:58518) 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 -- 172.21.15.57:0/2139935361 learned_addr learned my addr 172.21.15.57:0/2139935361 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 -- 172.21.15.57:0/2139935361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa28c00 msgr2=0x5603caa28fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 --2- 172.21.15.57:0/2139935361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa28c00 0x5603caa28fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 -- 172.21.15.57:0/2139935361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 msgr2=0x5603caa33e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 --2- 172.21.15.57:0/2139935361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa33e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 -- 172.21.15.57:0/2139935361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603ca7a7d80 con 0x5603caa343d0 2023-10-06T22:20:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.869+0000 7fd1d099a700 1 --2- 172.21.15.57:0/2139935361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa367c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009cf0 tx=0x7fd1c8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c719afefb3e8cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.871+0000 7fd1d199c700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd1b80a1800 0x7fd1b80a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:09.879+0000 7fd1d199c700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd1b80a1800 0x7fd1b80a3cc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005950 tx=0x7fd1bc00b390 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.050+0000 7fd1b77fe700 1 -- 172.21.15.57:0/2139935361 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8011e30 con 0x5603caa343d0 2023-10-06T22:20:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.050+0000 7fd1b77fe700 1 -- 172.21.15.57:0/2139935361 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c8004510 con 0x5603caa343d0 2023-10-06T22:20:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.050+0000 7fd1b77fe700 1 -- 172.21.15.57:0/2139935361 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8005350 con 0x5603caa343d0 2023-10-06T22:20:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.060+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2139935361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 msgr2=0x5603caa367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.061+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2139935361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa367c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009cf0 tx=0x7fd1c8011440 comp rx=0 tx=0).stop 2023-10-06T22:20:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.061+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2139935361 shutdown_connections 2023-10-06T22:20:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.061+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2139935361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa367c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.061+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2139935361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa28c00 0x5603caa28fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.061+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2139935361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa33e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.061+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2139935361 >> 172.21.15.57:0/2139935361 conn(0x5603ca99e200 msgr2=0x5603caa25080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.061+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2139935361 shutdown_connections 2023-10-06T22:20:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.062+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2139935361 wait complete. 2023-10-06T22:20:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.062+0000 7fd1e6cdfb80 1 -- start start 2023-10-06T22:20:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.063+0000 7fd1e6cdfb80 1 Processor -- start 2023-10-06T22:20:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.064+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa28c00 0x5603caa33de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.064+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa29520 0x5603caa310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.064+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.064+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ca97cdd0 con 0x5603caa28c00 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.064+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ca9196b0 con 0x5603caa29520 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.064+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ca919350 con 0x5603caa343d0 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.064+0000 7fd1e6cdfb80 10 client.0 _ll_register_callbacks cb 0x5603ca7767a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.066+0000 7fd1d199c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.066+0000 7fd1d099a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa29520 0x5603caa310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.066+0000 7fd1d199c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa31770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58532/0 (socket says 172.21.15.57:58532) 2023-10-06T22:20:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.066+0000 7fd1d199c700 1 -- 172.21.15.57:0/3035981606 learned_addr learned my addr 172.21.15.57:0/3035981606 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.066+0000 7fd1d119b700 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa28c00 0x5603caa33de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.067+0000 7fd1d099a700 1 -- 172.21.15.57:0/3035981606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 msgr2=0x5603caa31770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.067+0000 7fd1d099a700 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa31770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.067+0000 7fd1d099a700 1 -- 172.21.15.57:0/3035981606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa28c00 msgr2=0x5603caa33de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.067+0000 7fd1d099a700 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa28c00 0x5603caa33de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.067+0000 7fd1d099a700 1 -- 172.21.15.57:0/3035981606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8009920 con 0x5603caa29520 2023-10-06T22:20:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.079+0000 7fd1d099a700 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa29520 0x5603caa310d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8004e70 tx=0x7fd1c8009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3a7ed4f5d21771 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.528+0000 7fd1b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:20:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.528+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/3035981606 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8004aa0 con 0x5603caa29520 2023-10-06T22:20:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.528+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/3035981606 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd1c8004f70 con 0x5603caa29520 2023-10-06T22:20:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.538+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603caa1cd50 con 0x5603caa29520 2023-10-06T22:20:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.544+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/3035981606 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1c8005130 con 0x5603caa29520 2023-10-06T22:20:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.544+0000 7fd1e6cdfb80 10 client.26573 fetch_fsmap learned FSMap version 42 2023-10-06T22:20:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.544+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5603ca790c00 con 0x5603caa29520 2023-10-06T22:20:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.550+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/3035981606 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603caa1cd50 con 0x5603caa29520 2023-10-06T22:20:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.550+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/3035981606 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fd1c8029050 con 0x5603caa29520 2023-10-06T22:20:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.551+0000 7fd1e6cdfb80 10 client.26573 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:20:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.551+0000 7fd1e6cdfb80 10 client.26573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.551+0000 7fd1e6cdfb80 20 client.26573 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.551+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa43ee0 0x5603caa46300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.552+0000 7fd1e6cdfb80 4 client.26573 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:20:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.552+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603ca975db0 con 0x5603caa43ee0 2023-10-06T22:20:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.552+0000 7fd1d119b700 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa43ee0 0x5603caa46300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.565+0000 7fd1d119b700 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa43ee0 0x5603caa46300 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1673d8a8455b32e4 server_cookie=9d7ebabde033a204 in_seq=0 out_seq=0 2023-10-06T22:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.565+0000 7fd1b4ff9700 10 client.26573 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.582+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/3035981606 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603ca975db0 con 0x5603caa43ee0 2023-10-06T22:20:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.582+0000 7fd1b4ff9700 10 client.26573 handle_command_reply: tid=0 2023-10-06T22:20:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.582+0000 7fd1e6cdfb80 1 client.26573 shutdown 2023-10-06T22:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.583+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa29520 msgr2=0x5603caa310d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.583+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa29520 0x5603caa310d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8004e70 tx=0x7fd1c8009790 comp rx=0 tx=0).stop 2023-10-06T22:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 shutdown_connections 2023-10-06T22:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa43ee0 0x5603caa46300 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa343d0 0x5603caa31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa28c00 0x5603caa33de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/3035981606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa29520 0x5603caa310d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 >> 172.21.15.57:0/3035981606 conn(0x5603ca99e200 msgr2=0x5603caa24c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1b4ff9700 0 client.26573 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.591+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 shutdown_connections 2023-10-06T22:20:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.592+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/3035981606 wait complete. 2023-10-06T22:20:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.592+0000 7fd1e6cdfb80 20 client.26573 trim_cache size 0 max 16384 2023-10-06T22:20:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.912+0000 7fd1e6cdfb80 1 Processor -- start 2023-10-06T22:20:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.912+0000 7fd1e6cdfb80 1 -- start start 2023-10-06T22:20:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa2c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa2c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ca7fd8c0 con 0x5603caa29520 2023-10-06T22:20:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603caa21860 con 0x5603caa343d0 2023-10-06T22:20:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603caa31d70 con 0x5603caa28c00 2023-10-06T22:20:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1d119b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1d119b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa3af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58538/0 (socket says 172.21.15.57:58538) 2023-10-06T22:20:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.913+0000 7fd1d119b700 1 -- 172.21.15.57:0/2150382563 learned_addr learned my addr 172.21.15.57:0/2150382563 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.914+0000 7fd1d119b700 1 -- 172.21.15.57:0/2150382563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 msgr2=0x5603caa2c8d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:20:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.914+0000 7fd1d119b700 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa2c8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.914+0000 7fd1d119b700 1 -- 172.21.15.57:0/2150382563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 msgr2=0x5603caa2c390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:20:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.917+0000 7fd1d099a700 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa2c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.917+0000 7fd1d119b700 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa2c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.918+0000 7fd1d119b700 1 -- 172.21.15.57:0/2150382563 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603ca7ce1f0 con 0x5603caa28c00 2023-10-06T22:20:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:10.918+0000 7fd1d119b700 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa3af70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4007f10 tx=0x7fd1c4008020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e1b937350903c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.190+0000 7fd1b7fff700 1 -- 172.21.15.57:0/2150382563 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c409c970 con 0x5603caa28c00 2023-10-06T22:20:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.190+0000 7fd1b7fff700 1 -- 172.21.15.57:0/2150382563 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c40a0ac0 con 0x5603caa28c00 2023-10-06T22:20:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.190+0000 7fd1b7fff700 1 -- 172.21.15.57:0/2150382563 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c409ae30 con 0x5603caa28c00 2023-10-06T22:20:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.194+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2150382563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 msgr2=0x5603caa3af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.194+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa3af70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4007f10 tx=0x7fd1c4008020 comp rx=0 tx=0).stop 2023-10-06T22:20:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.194+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2150382563 shutdown_connections 2023-10-06T22:20:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.194+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa2c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.194+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa3af70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.194+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2150382563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa2c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.194+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2150382563 >> 172.21.15.57:0/2150382563 conn(0x5603ca99e200 msgr2=0x5603caa3f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.195+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2150382563 shutdown_connections 2023-10-06T22:20:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.195+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2150382563 wait complete. 2023-10-06T22:20:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.195+0000 7fd1e6cdfb80 1 -- start start 2023-10-06T22:20:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.200+0000 7fd1e6cdfb80 1 Processor -- start 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.202+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa36f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.202+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa37490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.202+0000 7fd1e6cdfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.202+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603caa96ea0 con 0x5603caa29520 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.202+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603caa97010 con 0x5603caa343d0 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.202+0000 7fd1e6cdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603caa971d0 con 0x5603caa28c00 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.202+0000 7fd1e6cdfb80 10 client.0 _ll_register_callbacks cb 0x7fd1c8009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d119b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa36f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d199c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d119b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa36f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:58544/0 (socket says 172.21.15.57:58544) 2023-10-06T22:20:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d119b700 1 -- 172.21.15.57:0/2343721501 learned_addr learned my addr 172.21.15.57:0/2343721501 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d199c700 1 -- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 msgr2=0x5603caa36f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d199c700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa36f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d199c700 1 -- 172.21.15.57:0/2343721501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 msgr2=0x5603caa37490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.203+0000 7fd1d099a700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa37490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.204+0000 7fd1d199c700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa37490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.204+0000 7fd1d199c700 1 -- 172.21.15.57:0/2343721501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc1cc440 con 0x5603caa343d0 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.204+0000 7fd1d099a700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa37490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.204+0000 7fd1d119b700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa36f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:20:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.204+0000 7fd1d199c700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa379f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00bd50 tx=0x7fd1bc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ec5f48351921d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.241+0000 7fd1b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:20:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.241+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/2343721501 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc011c30 con 0x5603caa343d0 2023-10-06T22:20:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.241+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/2343721501 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fd1bc017070 con 0x5603caa343d0 2023-10-06T22:20:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.243+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1c8004510 con 0x5603caa343d0 2023-10-06T22:20:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.249+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/2343721501 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1bc003e90 con 0x5603caa343d0 2023-10-06T22:20:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.257+0000 7fd1e6cdfb80 10 client.26576 fetch_fsmap learned FSMap version 42 2023-10-06T22:20:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.257+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5603ca7a6250 con 0x5603caa343d0 2023-10-06T22:20:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.259+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/2343721501 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c8004510 con 0x5603caa343d0 2023-10-06T22:20:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.259+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/2343721501 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fd1bc0054e0 con 0x5603caa343d0 2023-10-06T22:20:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.260+0000 7fd1e6cdfb80 10 client.26576 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:20:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.261+0000 7fd1e6cdfb80 10 client.26576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.261+0000 7fd1e6cdfb80 20 client.26576 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.261+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa989b0 0x5603caa9add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.261+0000 7fd1e6cdfb80 4 client.26576 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:20:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.261+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603ca975ad0 con 0x5603caa989b0 2023-10-06T22:20:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.270+0000 7fd1d119b700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa989b0 0x5603caa9add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.281+0000 7fd1d119b700 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa989b0 0x5603caa9add0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8f4bf55f4af570 server_cookie=2d39dbf2ae475d49 in_seq=0 out_seq=0 2023-10-06T22:20:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:11.284+0000 7fd1b4ff9700 10 client.26576 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:11.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:11 smithi057 ceph-mon[98492]: pgmap v166: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2023-10-06T22:20:11.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:11 smithi156 ceph-mon[134358]: pgmap v166: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2023-10-06T22:20:11.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:11 smithi190 ceph-mon[134891]: pgmap v166: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2023-10-06T22:20:12.080 INFO:tasks.workunit.client.0.smithi057.stdout: 287 827250 582 572065 584 214274 610 2023-10-06T22:20:13.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:13 smithi156 ceph-mon[134358]: pgmap v167: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 363 op/s 2023-10-06T22:20:13.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:13 smithi190 ceph-mon[134891]: pgmap v167: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 363 op/s 2023-10-06T22:20:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:13 smithi057 ceph-mon[98492]: pgmap v167: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 363 op/s 2023-10-06T22:20:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.876+0000 7fd1b4ff9700 1 -- 172.21.15.57:0/2343721501 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603ca975ad0 con 0x5603caa989b0 2023-10-06T22:20:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.876+0000 7fd1b4ff9700 10 client.26576 handle_command_reply: tid=0 2023-10-06T22:20:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.877+0000 7fd1e6cdfb80 1 client.26576 shutdown 2023-10-06T22:20:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 msgr2=0x5603caa379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa379f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00bd50 tx=0x7fd1bc0047c0 comp rx=0 tx=0).stop 2023-10-06T22:20:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 shutdown_connections 2023-10-06T22:20:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5603caa989b0 0x5603caa9add0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5603caa343d0 0x5603caa379f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603caa28c00 0x5603caa36f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 --2- 172.21.15.57:0/2343721501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5603caa29520 0x5603caa37490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 >> 172.21.15.57:0/2343721501 conn(0x5603ca99e200 msgr2=0x5603caa2c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.884+0000 7fd1b4ff9700 0 client.26576 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.885+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 shutdown_connections 2023-10-06T22:20:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.885+0000 7fd1e6cdfb80 1 -- 172.21.15.57:0/2343721501 wait complete. 2023-10-06T22:20:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:13.885+0000 7fd1e6cdfb80 20 client.26576 trim_cache size 0 max 16384 2023-10-06T22:20:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:20:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:20:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:20:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.094+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd1b80a1800 msgr2=0x7fd1b80a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.094+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd1b80a1800 0x7fd1b80a3cc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005950 tx=0x7fd1bc00b390 comp rx=0 tx=0).stop 2023-10-06T22:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.094+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc107eb0 msgr2=0x7fd1cc082690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.094+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc107eb0 0x7fd1cc082690 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4004fd0 tx=0x7fd1c400f7e0 comp rx=0 tx=0).stop 2023-10-06T22:20:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.105+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 shutdown_connections 2023-10-06T22:20:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.105+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd1b80a1800 0x7fd1b80a3cc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.105+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1cc10d520 0x7fd1cc07c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.105+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc107eb0 0x7fd1cc082690 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.105+0000 7fd1d33ff700 1 --2- 172.21.15.57:0/4013430745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc108870 0x7fd1cc082bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.105+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 >> 172.21.15.57:0/4013430745 conn(0x7fd1cc06cfa0 msgr2=0x7fd1cc0715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.117+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 shutdown_connections 2023-10-06T22:20:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:14.128+0000 7fd1d33ff700 1 -- 172.21.15.57:0/4013430745 wait complete. 2023-10-06T22:20:15.185 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:20:15.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:15 smithi156 ceph-mon[134358]: pgmap v168: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2023-10-06T22:20:15.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:15 smithi190 ceph-mon[134891]: pgmap v168: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2023-10-06T22:20:15.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:15 smithi057 ceph-mon[98492]: pgmap v168: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2023-10-06T22:20:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:17 smithi190 ceph-mon[134891]: pgmap v169: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2023-10-06T22:20:17.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:17 smithi057 ceph-mon[98492]: pgmap v169: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2023-10-06T22:20:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:17 smithi156 ceph-mon[134358]: pgmap v169: 97 pgs: 97 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2023-10-06T22:20:19.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:19 smithi057 ceph-mon[98492]: pgmap v170: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 507 op/s 2023-10-06T22:20:19.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:19 smithi156 ceph-mon[134358]: pgmap v170: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 507 op/s 2023-10-06T22:20:19.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:19 smithi190 ceph-mon[134891]: pgmap v170: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 507 op/s 2023-10-06T22:20:21.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.172+0000 7fe1001ff700 1 -- 172.21.15.57:0/4123687375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f8108870 msgr2=0x7fe0f8079110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:21.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.172+0000 7fe1001ff700 1 --2- 172.21.15.57:0/4123687375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f8108870 0x7fe0f8079110 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80098f0 tx=0x7fe0e8009c00 comp rx=0 tx=0).stop 2023-10-06T22:20:21.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.173+0000 7fe1001ff700 1 -- 172.21.15.57:0/4123687375 shutdown_connections 2023-10-06T22:20:21.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.173+0000 7fe1001ff700 1 --2- 172.21.15.57:0/4123687375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f8108870 0x7fe0f8079110 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.173+0000 7fe1001ff700 1 --2- 172.21.15.57:0/4123687375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f810d560 0x7fe0f810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.173+0000 7fe1001ff700 1 --2- 172.21.15.57:0/4123687375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f8107eb0 0x7fe0f8108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.173+0000 7fe1001ff700 1 -- 172.21.15.57:0/4123687375 >> 172.21.15.57:0/4123687375 conn(0x7fe0f806d1a0 msgr2=0x7fe0f806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.174+0000 7fe1001ff700 1 -- 172.21.15.57:0/4123687375 shutdown_connections 2023-10-06T22:20:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.176+0000 7fe1001ff700 1 -- 172.21.15.57:0/4123687375 wait complete. 2023-10-06T22:20:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.178+0000 7fe1001ff700 1 Processor -- start 2023-10-06T22:20:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.178+0000 7fe1001ff700 1 -- start start 2023-10-06T22:20:21.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.179+0000 7fe1001ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f8107eb0 0x7fe0f80835d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.179+0000 7fe1001ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f8108870 0x7fe0f8083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.179+0000 7fe1001ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f810d560 0x7fe0f807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.179+0000 7fe1001ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f807df10 con 0x7fe0f810d560 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.179+0000 7fe1001ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f807e080 con 0x7fe0f8108870 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.179+0000 7fe1001ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f807e240 con 0x7fe0f8107eb0 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.180+0000 7fe0fe79c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f810d560 0x7fe0f807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.180+0000 7fe0fe79c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f810d560 0x7fe0f807d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49390/0 (socket says 172.21.15.57:49390) 2023-10-06T22:20:21.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.180+0000 7fe0fe79c700 1 -- 172.21.15.57:0/1183697159 learned_addr learned my addr 172.21.15.57:0/1183697159 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:21.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.183+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f8108870 0x7fe0f8083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.183+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f8107eb0 0x7fe0f80835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.184+0000 7fe0fd79a700 1 -- 172.21.15.57:0/1183697159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f8107eb0 msgr2=0x7fe0f80835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.184+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f8107eb0 0x7fe0f80835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.184+0000 7fe0fd79a700 1 -- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f810d560 msgr2=0x7fe0f807d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.185+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f810d560 0x7fe0f807d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.185+0000 7fe0fd79a700 1 -- 172.21.15.57:0/1183697159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e8009520 con 0x7fe0f8108870 2023-10-06T22:20:21.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.185+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f8108870 0x7fe0f8083b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0007fd0 tx=0x7fe0f000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.476+0000 7fe0eeffd700 1 -- 172.21.15.57:0/1183697159 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000dd20 con 0x7fe0f8108870 2023-10-06T22:20:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.476+0000 7fe0eeffd700 1 -- 172.21.15.57:0/1183697159 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f000f040 con 0x7fe0f8108870 2023-10-06T22:20:21.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.476+0000 7fe0eeffd700 1 -- 172.21.15.57:0/1183697159 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0014600 con 0x7fe0f8108870 2023-10-06T22:20:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.476+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f8136310 con 0x7fe0f8108870 2023-10-06T22:20:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.476+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f81367d0 con 0x7fe0f8108870 2023-10-06T22:20:21.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.489+0000 7fe113afab80 1 Processor -- start 2023-10-06T22:20:21.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.489+0000 7fe0eeffd700 1 -- 172.21.15.57:0/1183697159 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe0f0015400 con 0x7fe0f8108870 2023-10-06T22:20:21.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.492+0000 7fe0eeffd700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe0e40a1800 0x7fe0e40a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.492+0000 7fe0eeffd700 1 -- 172.21.15.57:0/1183697159 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fe0f00cc8c0 con 0x7fe0f8108870 2023-10-06T22:20:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:21 smithi156 ceph-mon[134358]: pgmap v171: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 411 op/s 2023-10-06T22:20:21.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.497+0000 7fe113afab80 1 -- start start 2023-10-06T22:20:21.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.497+0000 7fe113afab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be0c0 0x5595d37be4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.497+0000 7fe113afab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.498+0000 7fe113afab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c9890 0x5595d37cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.498+0000 7fe113afab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d3727460 con 0x5595d37c9890 2023-10-06T22:20:21.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.498+0000 7fe113afab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d3718c20 con 0x5595d37be0c0 2023-10-06T22:20:21.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.498+0000 7fe113afab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d3716540 con 0x5595d37be9e0 2023-10-06T22:20:21.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.507+0000 7fe0fdf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be0c0 0x5595d37be4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.507+0000 7fe0fdf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be0c0 0x5595d37be4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:49100/0 (socket says 172.21.15.57:49100) 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.507+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/1291069551 learned_addr learned my addr 172.21.15.57:0/1291069551 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.508+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.508+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/1291069551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 msgr2=0x5595d37c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.508+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.508+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/1291069551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c9890 msgr2=0x5595d37cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.508+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c9890 0x5595d37cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.508+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/1291069551 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d353cde0 con 0x5595d37be0c0 2023-10-06T22:20:21.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.509+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be0c0 0x5595d37be4a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009ad0 tx=0x7fe0f4009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e622b3fac824f41b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:21.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.509+0000 7fe0fe79c700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe0e40a1800 0x7fe0e40a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.526+0000 7fe0fe79c700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe0e40a1800 0x7fe0e40a3cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009230 tx=0x7fe0e801b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:21 smithi190 ceph-mon[134891]: pgmap v171: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 411 op/s 2023-10-06T22:20:21.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.583+0000 7fe0e3fff700 1 -- 172.21.15.57:0/1291069551 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40095d0 con 0x5595d37be0c0 2023-10-06T22:20:21.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.583+0000 7fe0e3fff700 1 -- 172.21.15.57:0/1291069551 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f400ee50 con 0x5595d37be0c0 2023-10-06T22:20:21.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.583+0000 7fe0e3fff700 1 -- 172.21.15.57:0/1291069551 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40123d0 con 0x5595d37be0c0 2023-10-06T22:20:21.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.584+0000 7fe113afab80 1 -- 172.21.15.57:0/1291069551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be0c0 msgr2=0x5595d37be4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:21.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.584+0000 7fe113afab80 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be0c0 0x5595d37be4a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009ad0 tx=0x7fe0f4009de0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.584+0000 7fe113afab80 1 -- 172.21.15.57:0/1291069551 shutdown_connections 2023-10-06T22:20:21.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.584+0000 7fe113afab80 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c9890 0x5595d37cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.584+0000 7fe113afab80 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be0c0 0x5595d37be4a0 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009ad0 tx=0x7fe0f4009de0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.584+0000 7fe113afab80 1 --2- 172.21.15.57:0/1291069551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.584+0000 7fe113afab80 1 -- 172.21.15.57:0/1291069551 >> 172.21.15.57:0/1291069551 conn(0x5595d3733260 msgr2=0x5595d37ba450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:21.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.598+0000 7fe113afab80 1 -- 172.21.15.57:0/1291069551 shutdown_connections 2023-10-06T22:20:21.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.599+0000 7fe113afab80 1 -- 172.21.15.57:0/1291069551 wait complete. 2023-10-06T22:20:21.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.599+0000 7fe113afab80 1 -- start start 2023-10-06T22:20:21.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.600+0000 7fe113afab80 1 Processor -- start 2023-10-06T22:20:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.601+0000 7fe113afab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.601+0000 7fe113afab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37c9890 0x5595d37c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.601+0000 7fe113afab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.601+0000 7fe113afab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d3711e30 con 0x5595d37c7350 2023-10-06T22:20:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.601+0000 7fe113afab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d36ae710 con 0x5595d37c9890 2023-10-06T22:20:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.601+0000 7fe113afab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d36ae3b0 con 0x5595d37be9e0 2023-10-06T22:20:21.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.601+0000 7fe113afab80 10 client.0 _ll_register_callbacks cb 0x5595d350b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:21.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.602+0000 7fe0fe79c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.602+0000 7fe0fe79c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37d5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49406/0 (socket says 172.21.15.57:49406) 2023-10-06T22:20:21.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.602+0000 7fe0fe79c700 1 -- 172.21.15.57:0/3753572364 learned_addr learned my addr 172.21.15.57:0/3753572364 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:21.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.605+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37c9890 0x5595d37c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.605+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.606+0000 7fe0fe79c700 1 -- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 msgr2=0x5595d37c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:21.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.606+0000 7fe0fe79c700 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.606+0000 7fe0fe79c700 1 -- 172.21.15.57:0/3753572364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37c9890 msgr2=0x5595d37c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:21.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.606+0000 7fe0fe79c700 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37c9890 0x5595d37c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.606+0000 7fe0fe79c700 1 -- 172.21.15.57:0/3753572364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f4009730 con 0x5595d37c7350 2023-10-06T22:20:21.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.614+0000 7fe0fe79c700 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37d5410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80057a0 tx=0x7fe0e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c951b089fef704e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:21.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:21 smithi057 ceph-mon[98492]: pgmap v171: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 411 op/s 2023-10-06T22:20:21.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.766+0000 7fe0e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:20:21.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.766+0000 7fe0e17fa700 1 -- 172.21.15.57:0/3753572364 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800f870 con 0x5595d37c7350 2023-10-06T22:20:21.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.766+0000 7fe0e17fa700 1 -- 172.21.15.57:0/3753572364 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fe0e8003680 con 0x5595d37c7350 2023-10-06T22:20:21.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.766+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595d37b1f10 con 0x5595d37c7350 2023-10-06T22:20:21.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.779+0000 7fe0e17fa700 1 -- 172.21.15.57:0/3753572364 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e8003840 con 0x5595d37c7350 2023-10-06T22:20:21.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.779+0000 7fe113afab80 10 client.16551 fetch_fsmap learned FSMap version 42 2023-10-06T22:20:21.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.779+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5595d3525c60 con 0x5595d37c7350 2023-10-06T22:20:21.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.792+0000 7fe0e17fa700 1 -- 172.21.15.57:0/3753572364 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0e8034070 con 0x5595d37c7350 2023-10-06T22:20:21.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.792+0000 7fe0e17fa700 1 -- 172.21.15.57:0/3753572364 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fe0e8024b20 con 0x5595d37c7350 2023-10-06T22:20:21.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.794+0000 7fe113afab80 10 client.16551 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:20:21.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.794+0000 7fe113afab80 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:21.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.794+0000 7fe113afab80 20 client.16551 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:21.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.794+0000 7fe113afab80 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d37d99e0 0x5595d37dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:21.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.794+0000 7fe113afab80 4 client.16551 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:20:21.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.794+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595d370ae10 con 0x5595d37d99e0 2023-10-06T22:20:21.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.794+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d37d99e0 0x5595d37dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:21.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.820+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d37d99e0 0x5595d37dbe00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c88a576cc4d3d7 server_cookie=230fd6283d2b06d3 in_seq=0 out_seq=0 2023-10-06T22:20:21.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.821+0000 7fe0e17fa700 10 client.16551 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:21.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.822+0000 7fe0e17fa700 1 -- 172.21.15.57:0/3753572364 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5595d370ae10 con 0x5595d37d99e0 2023-10-06T22:20:21.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.822+0000 7fe0e17fa700 10 client.16551 handle_command_reply: tid=0 2023-10-06T22:20:21.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.822+0000 7fe113afab80 1 client.16551 shutdown 2023-10-06T22:20:21.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 msgr2=0x5595d37d5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:21.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37d5410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80057a0 tx=0x7fe0e8009260 comp rx=0 tx=0).stop 2023-10-06T22:20:21.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 shutdown_connections 2023-10-06T22:20:21.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d37d99e0 0x5595d37dbe00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37d5410 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37be9e0 0x5595d37c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 --2- 172.21.15.57:0/3753572364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37c9890 0x5595d37c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:21.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.829+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 >> 172.21.15.57:0/3753572364 conn(0x5595d3733260 msgr2=0x5595d37b9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:21.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.883+0000 7fe0e17fa700 0 client.16551 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:21.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.902+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 shutdown_connections 2023-10-06T22:20:21.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.903+0000 7fe113afab80 1 -- 172.21.15.57:0/3753572364 wait complete. 2023-10-06T22:20:21.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:21.911+0000 7fe113afab80 20 client.16551 trim_cache size 0 max 16384 2023-10-06T22:20:22.128 INFO:tasks.workunit.client.0.smithi057.stdout: 294 791137 616 550677 605 208096 676 2023-10-06T22:20:22.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.209+0000 7fe113afab80 1 Processor -- start 2023-10-06T22:20:22.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.230+0000 7fe113afab80 1 -- start start 2023-10-06T22:20:22.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.231+0000 7fe113afab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:22.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.231+0000 7fe113afab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:22.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.231+0000 7fe113afab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:22.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.231+0000 7fe113afab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d3592920 con 0x5595d37c7350 2023-10-06T22:20:22.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.231+0000 7fe113afab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d37bbd90 con 0x5595d37be9e0 2023-10-06T22:20:22.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.231+0000 7fe113afab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d37cc120 con 0x5595d37c9890 2023-10-06T22:20:22.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.234+0000 7fe0fdf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:22.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.234+0000 7fe0fdf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37d53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:51166/0 (socket says 172.21.15.57:51166) 2023-10-06T22:20:22.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.234+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/4233510390 learned_addr learned my addr 172.21.15.57:0/4233510390 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:22.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.234+0000 7fe0fe79c700 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:22.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.243+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/4233510390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 msgr2=0x5595d37c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.243+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.243+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/4233510390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 msgr2=0x5595d37b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.243+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.243+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/4233510390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d37e0a50 con 0x5595d37be9e0 2023-10-06T22:20:22.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.247+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37d53c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4000ea0 tx=0x7fe0f4006a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733f06e510c0eba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:22.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.269+0000 7fe0ecff9700 1 -- 172.21.15.57:0/4233510390 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40095b0 con 0x5595d37be9e0 2023-10-06T22:20:22.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.269+0000 7fe0ecff9700 1 -- 172.21.15.57:0/4233510390 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f4013c00 con 0x5595d37be9e0 2023-10-06T22:20:22.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.269+0000 7fe0ecff9700 1 -- 172.21.15.57:0/4233510390 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f402ae70 con 0x5595d37be9e0 2023-10-06T22:20:22.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.270+0000 7fe113afab80 1 -- 172.21.15.57:0/4233510390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 msgr2=0x5595d37d53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:22.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.270+0000 7fe113afab80 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37d53c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4000ea0 tx=0x7fe0f4006a70 comp rx=0 tx=0).stop 2023-10-06T22:20:22.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.276+0000 7fe113afab80 1 -- 172.21.15.57:0/4233510390 shutdown_connections 2023-10-06T22:20:22.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.276+0000 7fe113afab80 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.276+0000 7fe113afab80 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37d53c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.276+0000 7fe113afab80 1 --2- 172.21.15.57:0/4233510390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.277+0000 7fe113afab80 1 -- 172.21.15.57:0/4233510390 >> 172.21.15.57:0/4233510390 conn(0x5595d3733260 msgr2=0x5595d37c5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:22.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.277+0000 7fe113afab80 1 -- 172.21.15.57:0/4233510390 shutdown_connections 2023-10-06T22:20:22.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.277+0000 7fe113afab80 1 -- 172.21.15.57:0/4233510390 wait complete. 2023-10-06T22:20:22.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.278+0000 7fe113afab80 1 -- start start 2023-10-06T22:20:22.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.278+0000 7fe113afab80 1 Processor -- start 2023-10-06T22:20:22.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.279+0000 7fe113afab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:22.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.279+0000 7fe113afab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:22.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.279+0000 7fe113afab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:22.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.279+0000 7fe113afab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d382c960 con 0x5595d37c7350 2023-10-06T22:20:22.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.279+0000 7fe113afab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d382cad0 con 0x5595d37be9e0 2023-10-06T22:20:22.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.279+0000 7fe113afab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d382cc90 con 0x5595d37c9890 2023-10-06T22:20:22.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.279+0000 7fe113afab80 10 client.0 _ll_register_callbacks cb 0x7fe0e80053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:22.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.280+0000 7fe0fdf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:22.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.280+0000 7fe0fdf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:51174/0 (socket says 172.21.15.57:51174) 2023-10-06T22:20:22.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.280+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/444804424 learned_addr learned my addr 172.21.15.57:0/444804424 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:22.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.280+0000 7fe0fe79c700 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:22.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.281+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:22.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.291+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 msgr2=0x5595d37ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:22.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.291+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.291+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/444804424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 msgr2=0x5595d37ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:22.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.291+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.291+0000 7fe0fdf9b700 1 -- 172.21.15.57:0/444804424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595d353cde0 con 0x5595d37be9e0 2023-10-06T22:20:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.302+0000 7fe0fdf9b700 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37cdba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4013b50 tx=0x7fe0f40095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c77d49e8c96e43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:22.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.303+0000 7fe0e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:20:22.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.313+0000 7fe0e17fa700 1 -- 172.21.15.57:0/444804424 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400e990 con 0x5595d37be9e0 2023-10-06T22:20:22.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.313+0000 7fe0e17fa700 1 -- 172.21.15.57:0/444804424 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fe0f4021040 con 0x5595d37be9e0 2023-10-06T22:20:22.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.314+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f400ee50 con 0x5595d37be9e0 2023-10-06T22:20:22.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.318+0000 7fe0e17fa700 1 -- 172.21.15.57:0/444804424 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f400eaf0 con 0x5595d37be9e0 2023-10-06T22:20:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.319+0000 7fe113afab80 10 client.26591 fetch_fsmap learned FSMap version 42 2023-10-06T22:20:22.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.319+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5595d3563250 con 0x5595d37be9e0 2023-10-06T22:20:22.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.324+0000 7fe0e17fa700 1 -- 172.21.15.57:0/444804424 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f400ee50 con 0x5595d37be9e0 2023-10-06T22:20:22.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.324+0000 7fe0e17fa700 1 -- 172.21.15.57:0/444804424 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fe0f4011070 con 0x5595d37be9e0 2023-10-06T22:20:22.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.325+0000 7fe113afab80 10 client.26591 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:20:22.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.325+0000 7fe113afab80 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:22.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.325+0000 7fe113afab80 20 client.26591 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:22.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.325+0000 7fe113afab80 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d382deb0 0x5595d38302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:22.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.325+0000 7fe113afab80 4 client.26591 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:20:22.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.325+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595d370ab30 con 0x5595d382deb0 2023-10-06T22:20:22.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.342+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d382deb0 0x5595d38302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:22.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.343+0000 7fe0fd79a700 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d382deb0 0x5595d38302a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c978b7ca02f4dd8a server_cookie=14a917eb801ec36b in_seq=0 out_seq=0 2023-10-06T22:20:22.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.343+0000 7fe0e17fa700 10 client.26591 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:22.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.348+0000 7fe0e17fa700 1 -- 172.21.15.57:0/444804424 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5595d370ab30 con 0x5595d382deb0 2023-10-06T22:20:22.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.349+0000 7fe0e17fa700 10 client.26591 handle_command_reply: tid=0 2023-10-06T22:20:22.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.349+0000 7fe113afab80 1 client.26591 shutdown 2023-10-06T22:20:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 msgr2=0x5595d37cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37cdba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4013b50 tx=0x7fe0f40095b0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 shutdown_connections 2023-10-06T22:20:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5595d382deb0 0x5595d38302a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5595d37c9890 0x5595d37ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595d37be9e0 0x5595d37cdba0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 --2- 172.21.15.57:0/444804424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595d37c7350 0x5595d37ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 >> 172.21.15.57:0/444804424 conn(0x5595d3733260 msgr2=0x5595d37d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:22.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.481+0000 7fe0e17fa700 0 client.26591 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:22.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.482+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 shutdown_connections 2023-10-06T22:20:22.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.482+0000 7fe113afab80 1 -- 172.21.15.57:0/444804424 wait complete. 2023-10-06T22:20:22.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.486+0000 7fe113afab80 20 client.26591 trim_cache size 0 max 16384 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:20:22.489 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:20:22.490 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:20:22.490 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:20:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.551+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe0e40a1800 msgr2=0x7fe0e40a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.551+0000 7fe1001ff700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe0e40a1800 0x7fe0e40a3cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009230 tx=0x7fe0e801b040 comp rx=0 tx=0).stop 2023-10-06T22:20:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.551+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f8108870 msgr2=0x7fe0f8083b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.551+0000 7fe1001ff700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f8108870 0x7fe0f8083b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0007fd0 tx=0x7fe0f000d3b0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.564+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 shutdown_connections 2023-10-06T22:20:22.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.564+0000 7fe1001ff700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe0e40a1800 0x7fe0e40a3cc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.564+0000 7fe1001ff700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0f810d560 0x7fe0f807d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.564+0000 7fe1001ff700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f8107eb0 0x7fe0f80835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.564+0000 7fe1001ff700 1 --2- 172.21.15.57:0/1183697159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f8108870 0x7fe0f8083b10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:22.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.564+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 >> 172.21.15.57:0/1183697159 conn(0x7fe0f806d1a0 msgr2=0x7fe0f8073170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:22.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.565+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 shutdown_connections 2023-10-06T22:20:22.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:20:22.565+0000 7fe1001ff700 1 -- 172.21.15.57:0/1183697159 wait complete. 2023-10-06T22:20:22.654 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:20:23.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:23 smithi156 ceph-mon[134358]: pgmap v172: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 483 op/s 2023-10-06T22:20:23.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:23 smithi190 ceph-mon[134891]: pgmap v172: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 483 op/s 2023-10-06T22:20:23.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:23 smithi057 ceph-mon[98492]: pgmap v172: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 483 op/s 2023-10-06T22:20:25.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:25 smithi190 ceph-mon[134891]: pgmap v173: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 453 op/s 2023-10-06T22:20:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:25 smithi057 ceph-mon[98492]: pgmap v173: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 453 op/s 2023-10-06T22:20:25.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:25 smithi156 ceph-mon[134358]: pgmap v173: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 453 op/s 2023-10-06T22:20:27.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:27 smithi057 ceph-mon[98492]: pgmap v174: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 391 op/s 2023-10-06T22:20:27.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:27 smithi156 ceph-mon[134358]: pgmap v174: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 391 op/s 2023-10-06T22:20:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:27 smithi190 ceph-mon[134891]: pgmap v174: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 391 op/s 2023-10-06T22:20:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:29 smithi190 ceph-mon[134891]: pgmap v175: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 478 op/s 2023-10-06T22:20:29.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:29 smithi057 ceph-mon[98492]: pgmap v175: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 478 op/s 2023-10-06T22:20:29.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:29 smithi156 ceph-mon[134358]: pgmap v175: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 478 op/s 2023-10-06T22:20:31.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:31 smithi190 ceph-mon[134891]: pgmap v176: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 273 op/s 2023-10-06T22:20:31.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:31 smithi057 ceph-mon[98492]: pgmap v176: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 273 op/s 2023-10-06T22:20:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:31 smithi156 ceph-mon[134358]: pgmap v176: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 273 op/s 2023-10-06T22:20:32.090 INFO:tasks.workunit.client.0.smithi057.stdout: 307 796456 657 555205 597 214035 1337 2023-10-06T22:20:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.328+0000 7fab417ff700 1 -- 172.21.15.57:0/3136778370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 msgr2=0x7fab3c11ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.328+0000 7fab417ff700 1 --2- 172.21.15.57:0/3136778370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 0x7fab3c11ca80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab30009990 tx=0x7fab30009ca0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.372+0000 7fab417ff700 1 -- 172.21.15.57:0/3136778370 shutdown_connections 2023-10-06T22:20:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.372+0000 7fab417ff700 1 --2- 172.21.15.57:0/3136778370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 0x7fab3c11ca80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.372+0000 7fab417ff700 1 --2- 172.21.15.57:0/3136778370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab3c10f620 0x7fab3c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.372+0000 7fab417ff700 1 --2- 172.21.15.57:0/3136778370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 0x7fab3c1083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.372+0000 7fab417ff700 1 -- 172.21.15.57:0/3136778370 >> 172.21.15.57:0/3136778370 conn(0x7fab3c06cfa0 msgr2=0x7fab3c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.373+0000 7fab417ff700 1 -- 172.21.15.57:0/3136778370 shutdown_connections 2023-10-06T22:20:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.373+0000 7fab417ff700 1 -- 172.21.15.57:0/3136778370 wait complete. 2023-10-06T22:20:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.376+0000 7fab417ff700 1 Processor -- start 2023-10-06T22:20:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.376+0000 7fab417ff700 1 -- start start 2023-10-06T22:20:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.377+0000 7fab417ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab3c10f620 0x7fab3c118a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.377+0000 7fab417ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 0x7fab3c118f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.377+0000 7fab417ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 0x7fab3c111860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.377+0000 7fab417ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c112030 con 0x7fab3c10f620 2023-10-06T22:20:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.377+0000 7fab417ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c1121a0 con 0x7fab3c10fa90 2023-10-06T22:20:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.377+0000 7fab417ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c112360 con 0x7fab3c11a690 2023-10-06T22:20:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 0x7fab3c118f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 0x7fab3c118f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:45308/0 (socket says 172.21.15.57:45308) 2023-10-06T22:20:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 -- 172.21.15.57:0/74885744 learned_addr learned my addr 172.21.15.57:0/74885744 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3b7fe700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 0x7fab3c111860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 -- 172.21.15.57:0/74885744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 msgr2=0x7fab3c111860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 0x7fab3c111860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 -- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab3c10f620 msgr2=0x7fab3c118a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab3c10f620 0x7fab3c118a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.378+0000 7fab3a7fc700 1 -- 172.21.15.57:0/74885744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab300095f0 con 0x7fab3c10fa90 2023-10-06T22:20:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.379+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 0x7fab3c118f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab3400ef10 tx=0x7fab3400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.447+0000 7fab2bfff700 1 -- 172.21.15.57:0/74885744 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400cad0 con 0x7fab3c10fa90 2023-10-06T22:20:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.447+0000 7fab2bfff700 1 -- 172.21.15.57:0/74885744 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab3400f040 con 0x7fab3c10fa90 2023-10-06T22:20:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.447+0000 7fab2bfff700 1 -- 172.21.15.57:0/74885744 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340146e0 con 0x7fab3c10fa90 2023-10-06T22:20:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.448+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c1125e0 con 0x7fab3c10fa90 2023-10-06T22:20:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.448+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c116830 con 0x7fab3c10fa90 2023-10-06T22:20:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.455+0000 7fab5503cb80 1 Processor -- start 2023-10-06T22:20:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.456+0000 7fab5503cb80 1 -- start start 2023-10-06T22:20:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.457+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x55971352efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.457+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713539e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.457+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x55971353c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.457+0000 7fab5503cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559713498400 con 0x55971352f520 2023-10-06T22:20:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.457+0000 7fab5503cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559713489bc0 con 0x55971353a3d0 2023-10-06T22:20:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.457+0000 7fab5503cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597134874e0 con 0x55971352ec00 2023-10-06T22:20:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.458+0000 7fab3b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x55971352efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.458+0000 7fab3b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x55971352efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33856/0 (socket says 172.21.15.57:33856) 2023-10-06T22:20:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.458+0000 7fab3b7fe700 1 -- 172.21.15.57:0/1164962442 learned_addr learned my addr 172.21.15.57:0/1164962442 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.458+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x55971353c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.459+0000 7fab3b7fe700 1 -- 172.21.15.57:0/1164962442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 msgr2=0x55971353c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.459+0000 7fab3b7fe700 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x55971353c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.459+0000 7fab3b7fe700 1 -- 172.21.15.57:0/1164962442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 msgr2=0x559713539e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.459+0000 7fab3b7fe700 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713539e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.459+0000 7fab3b7fe700 1 -- 172.21.15.57:0/1164962442 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597132add80 con 0x55971352ec00 2023-10-06T22:20:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.459+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x55971353c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:20:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.465+0000 7fab3b7fe700 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x55971352efe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab30005b00 tx=0x7fab30005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719c2c36c1c6a9ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.471+0000 7fab2bfff700 1 -- 172.21.15.57:0/74885744 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fab34004ad0 con 0x7fab3c10fa90 2023-10-06T22:20:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.474+0000 7fab2bfff700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fab240a1800 0x7fab240a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.474+0000 7fab2bfff700 1 -- 172.21.15.57:0/74885744 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fab340ccf70 con 0x7fab3c10fa90 2023-10-06T22:20:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.476+0000 7fab3affd700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fab240a1800 0x7fab240a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.493+0000 7fab3affd700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fab240a1800 0x7fab240a3cc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005fd0 tx=0x7fab2c005ee0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.521+0000 7fab297fa700 1 -- 172.21.15.57:0/1164962442 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000bde0 con 0x55971352ec00 2023-10-06T22:20:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.521+0000 7fab297fa700 1 -- 172.21.15.57:0/1164962442 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab3001e040 con 0x55971352ec00 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.521+0000 7fab297fa700 1 -- 172.21.15.57:0/1164962442 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30004b60 con 0x55971352ec00 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.524+0000 7fab5503cb80 1 -- 172.21.15.57:0/1164962442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 msgr2=0x55971352efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.524+0000 7fab5503cb80 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x55971352efe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab30005b00 tx=0x7fab30005c00 comp rx=0 tx=0).stop 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.528+0000 7fab5503cb80 1 -- 172.21.15.57:0/1164962442 shutdown_connections 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.528+0000 7fab5503cb80 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x55971353c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.528+0000 7fab5503cb80 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x55971352efe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.528+0000 7fab5503cb80 1 --2- 172.21.15.57:0/1164962442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713539e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.528+0000 7fab5503cb80 1 -- 172.21.15.57:0/1164962442 >> 172.21.15.57:0/1164962442 conn(0x5597134a4200 msgr2=0x55971352b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.537+0000 7fab5503cb80 1 -- 172.21.15.57:0/1164962442 shutdown_connections 2023-10-06T22:20:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.537+0000 7fab5503cb80 1 -- 172.21.15.57:0/1164962442 wait complete. 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.538+0000 7fab5503cb80 1 -- start start 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.539+0000 7fab5503cb80 1 Processor -- start 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.540+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713537200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.540+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352f520 0x559713537740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.540+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x559713537c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.540+0000 7fab5503cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559713482dd0 con 0x55971353a3d0 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.540+0000 7fab5503cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971341f6b0 con 0x55971352f520 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.540+0000 7fab5503cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971341f350 con 0x55971352ec00 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.540+0000 7fab5503cb80 10 client.0 _ll_register_callbacks cb 0x55971327c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.541+0000 7fab3affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713537200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.544+0000 7fab3affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713537200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33870/0 (socket says 172.21.15.57:33870) 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.544+0000 7fab3affd700 1 -- 172.21.15.57:0/3814939187 learned_addr learned my addr 172.21.15.57:0/3814939187 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.544+0000 7fab3b7fe700 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x559713537c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.544+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352f520 0x559713537740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.557+0000 7fab3affd700 1 -- 172.21.15.57:0/3814939187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352f520 msgr2=0x559713537740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.557+0000 7fab3affd700 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352f520 0x559713537740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.557+0000 7fab3affd700 1 -- 172.21.15.57:0/3814939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 msgr2=0x559713537c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.557+0000 7fab3affd700 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x559713537c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.558+0000 7fab3affd700 1 -- 172.21.15.57:0/3814939187 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c11d3b0 con 0x55971352ec00 2023-10-06T22:20:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.558+0000 7fab3affd700 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713537200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0096e0 tx=0x7fab2c005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e47d61ca382d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.594+0000 7fab1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:20:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.594+0000 7fab1e7fc700 1 -- 172.21.15.57:0/3814939187 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c010a40 con 0x55971352ec00 2023-10-06T22:20:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.594+0000 7fab1e7fc700 1 -- 172.21.15.57:0/3814939187 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fab2c010ba0 con 0x55971352ec00 2023-10-06T22:20:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.595+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559713522d50 con 0x55971352ec00 2023-10-06T22:20:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.600+0000 7fab1e7fc700 1 -- 172.21.15.57:0/3814939187 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab2c010d60 con 0x55971352ec00 2023-10-06T22:20:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.600+0000 7fab5503cb80 10 client.26587 fetch_fsmap learned FSMap version 42 2023-10-06T22:20:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.600+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559713296c00 con 0x55971352ec00 2023-10-06T22:20:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.605+0000 7fab1e7fc700 1 -- 172.21.15.57:0/3814939187 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559713522d50 con 0x55971352ec00 2023-10-06T22:20:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.605+0000 7fab1e7fc700 1 -- 172.21.15.57:0/3814939187 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fab2c026500 con 0x55971352ec00 2023-10-06T22:20:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.607+0000 7fab5503cb80 10 client.26587 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:20:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.607+0000 7fab5503cb80 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.607+0000 7fab5503cb80 20 client.26587 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.607+0000 7fab5503cb80 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55971356c010 0x55971356e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.607+0000 7fab5503cb80 4 client.26587 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:20:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.607+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55971347bdb0 con 0x55971356c010 2023-10-06T22:20:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.607+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55971356c010 0x55971356e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.611+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55971356c010 0x55971356e430 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2bfe173bc0edad server_cookie=8b84ea355846eb6 in_seq=0 out_seq=0 2023-10-06T22:20:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.612+0000 7fab1e7fc700 10 client.26587 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.616+0000 7fab1e7fc700 1 -- 172.21.15.57:0/3814939187 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55971347bdb0 con 0x55971356c010 2023-10-06T22:20:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.616+0000 7fab1e7fc700 10 client.26587 handle_command_reply: tid=0 2023-10-06T22:20:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.616+0000 7fab5503cb80 1 client.26587 shutdown 2023-10-06T22:20:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 msgr2=0x559713537200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713537200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0096e0 tx=0x7fab2c005d20 comp rx=0 tx=0).stop 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 shutdown_connections 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55971356c010 0x55971356e430 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x559713537c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713537200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 --2- 172.21.15.57:0/3814939187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352f520 0x559713537740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 >> 172.21.15.57:0/3814939187 conn(0x5597134a4200 msgr2=0x55971352ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.736+0000 7fab1e7fc700 0 client.26587 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:32.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.737+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 shutdown_connections 2023-10-06T22:20:32.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.737+0000 7fab5503cb80 1 -- 172.21.15.57:0/3814939187 wait complete. 2023-10-06T22:20:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:32.744+0000 7fab5503cb80 20 client.26587 trim_cache size 0 max 16384 2023-10-06T22:20:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.225+0000 7fab5503cb80 1 Processor -- start 2023-10-06T22:20:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.243+0000 7fab5503cb80 1 -- start start 2023-10-06T22:20:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.243+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713562ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.243+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713531aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.243+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x5597135325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.243+0000 7fab5503cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559713568960 con 0x55971352f520 2023-10-06T22:20:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.243+0000 7fab5503cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597133038c0 con 0x55971353a3d0 2023-10-06T22:20:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.243+0000 7fab5503cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559713563490 con 0x55971352ec00 2023-10-06T22:20:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.245+0000 7fab3a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713531aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.245+0000 7fab3a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713531aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44964/0 (socket says 172.21.15.57:44964) 2023-10-06T22:20:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.245+0000 7fab3a7fc700 1 -- 172.21.15.57:0/458516287 learned_addr learned my addr 172.21.15.57:0/458516287 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.245+0000 7fab3b7fe700 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x5597135325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.245+0000 7fab3affd700 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713562ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.250+0000 7fab3a7fc700 1 -- 172.21.15.57:0/458516287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 msgr2=0x559713562ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.250+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713562ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.250+0000 7fab3a7fc700 1 -- 172.21.15.57:0/458516287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 msgr2=0x5597135325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.250+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x5597135325d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.250+0000 7fab3a7fc700 1 -- 172.21.15.57:0/458516287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597132d41f0 con 0x55971352f520 2023-10-06T22:20:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.265+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713531aa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fab340090a0 tx=0x7fab34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1350a1dfdb96a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.324+0000 7fab29ffb700 1 -- 172.21.15.57:0/458516287 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340b8270 con 0x55971352f520 2023-10-06T22:20:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.324+0000 7fab29ffb700 1 -- 172.21.15.57:0/458516287 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab34099ff0 con 0x55971352f520 2023-10-06T22:20:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.324+0000 7fab29ffb700 1 -- 172.21.15.57:0/458516287 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34098440 con 0x55971352f520 2023-10-06T22:20:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.330+0000 7fab5503cb80 1 -- 172.21.15.57:0/458516287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 msgr2=0x559713531aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.330+0000 7fab5503cb80 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713531aa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fab340090a0 tx=0x7fab34009bd0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.331+0000 7fab5503cb80 1 -- 172.21.15.57:0/458516287 shutdown_connections 2023-10-06T22:20:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.331+0000 7fab5503cb80 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971353a3d0 0x5597135325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.331+0000 7fab5503cb80 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352ec00 0x559713562ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.331+0000 7fab5503cb80 1 --2- 172.21.15.57:0/458516287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971352f520 0x559713531aa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.331+0000 7fab5503cb80 1 -- 172.21.15.57:0/458516287 >> 172.21.15.57:0/458516287 conn(0x5597134a4200 msgr2=0x55971356e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.331+0000 7fab5503cb80 1 -- 172.21.15.57:0/458516287 shutdown_connections 2023-10-06T22:20:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.331+0000 7fab5503cb80 1 -- 172.21.15.57:0/458516287 wait complete. 2023-10-06T22:20:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.332+0000 7fab5503cb80 1 -- start start 2023-10-06T22:20:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.332+0000 7fab5503cb80 1 Processor -- start 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.333+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352ec00 0x5597135b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.333+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352f520 0x5597135b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.333+0000 7fab5503cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x5597135b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.333+0000 7fab5503cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597135b9320 con 0x55971353a3d0 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.333+0000 7fab5503cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597135b9460 con 0x55971352ec00 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.333+0000 7fab5503cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597135b9620 con 0x55971352f520 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.333+0000 7fab5503cb80 10 client.0 _ll_register_callbacks cb 0x7fab2c00d860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.336+0000 7fab3a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352f520 0x5597135b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.336+0000 7fab3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352ec00 0x5597135b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.337+0000 7fab3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352ec00 0x5597135b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:45356/0 (socket says 172.21.15.57:45356) 2023-10-06T22:20:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.337+0000 7fab3affd700 1 -- 172.21.15.57:0/2130902318 learned_addr learned my addr 172.21.15.57:0/2130902318 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.337+0000 7fab3b7fe700 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x5597135b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.353+0000 7fab3a7fc700 1 -- 172.21.15.57:0/2130902318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352ec00 msgr2=0x5597135b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.353+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352ec00 0x5597135b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.353+0000 7fab3a7fc700 1 -- 172.21.15.57:0/2130902318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 msgr2=0x5597135b8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.353+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x5597135b8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.353+0000 7fab3a7fc700 1 -- 172.21.15.57:0/2130902318 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c116830 con 0x55971352f520 2023-10-06T22:20:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.377+0000 7fab3a7fc700 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352f520 0x5597135b44c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x559713567f50 tx=0x7fab34007c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ed09588450c866 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.381+0000 7fab1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:20:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.382+0000 7fab1e7fc700 1 -- 172.21.15.57:0/2130902318 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340b8270 con 0x55971352f520 2023-10-06T22:20:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.382+0000 7fab1e7fc700 1 -- 172.21.15.57:0/2130902318 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fab340a22f0 con 0x55971352f520 2023-10-06T22:20:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.382+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab3001e040 con 0x55971352f520 2023-10-06T22:20:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.393+0000 7fab1e7fc700 1 -- 172.21.15.57:0/2130902318 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab3409e140 con 0x55971352f520 2023-10-06T22:20:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.393+0000 7fab5503cb80 10 client.26593 fetch_fsmap learned FSMap version 42 2023-10-06T22:20:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.393+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5597132ac250 con 0x55971352f520 2023-10-06T22:20:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.405+0000 7fab1e7fc700 1 -- 172.21.15.57:0/2130902318 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab3001e040 con 0x55971352f520 2023-10-06T22:20:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.405+0000 7fab1e7fc700 1 -- 172.21.15.57:0/2130902318 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 42) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fab34080070 con 0x55971352f520 2023-10-06T22:20:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.406+0000 7fab5503cb80 10 client.26593 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T22:20:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.406+0000 7fab5503cb80 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.406+0000 7fab5503cb80 20 client.26593 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.406+0000 7fab5503cb80 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5597135c0a40 0x5597135c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.406+0000 7fab5503cb80 4 client.26593 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:20:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.406+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55971347bad0 con 0x5597135c0a40 2023-10-06T22:20:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.407+0000 7fab3affd700 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5597135c0a40 0x5597135c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.421+0000 7fab3affd700 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5597135c0a40 0x5597135c2e60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7cdf41d85777959 server_cookie=c687c0d2daecd07a in_seq=0 out_seq=0 2023-10-06T22:20:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:33.421+0000 7fab1e7fc700 10 client.26593 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:33.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:33 smithi057 ceph-mon[98492]: pgmap v177: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.1 MiB/s wr, 333 op/s 2023-10-06T22:20:33.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:33 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:20:33.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:33 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:20:33.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:33 smithi156 ceph-mon[134358]: pgmap v177: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.1 MiB/s wr, 333 op/s 2023-10-06T22:20:33.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:33 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:20:33.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:33 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:20:33.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:33 smithi190 ceph-mon[134891]: pgmap v177: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.1 MiB/s wr, 333 op/s 2023-10-06T22:20:33.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:33 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:20:33.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:33 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:20:34.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:34 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:20:34.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:34 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:20:34.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:34 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:20:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.858+0000 7fab1e7fc700 1 -- 172.21.15.57:0/2130902318 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55971347bad0 con 0x5597135c0a40 2023-10-06T22:20:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.858+0000 7fab1e7fc700 10 client.26593 handle_command_reply: tid=0 2023-10-06T22:20:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.858+0000 7fab5503cb80 1 client.26593 shutdown 2023-10-06T22:20:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.858+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352f520 msgr2=0x5597135b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.858+0000 7fab5503cb80 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352f520 0x5597135b44c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x559713567f50 tx=0x7fab34007c40 comp rx=0 tx=0).stop 2023-10-06T22:20:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 shutdown_connections 2023-10-06T22:20:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab5503cb80 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5597135c0a40 0x5597135c2e60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab5503cb80 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55971353a3d0 0x5597135b8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab5503cb80 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55971352ec00 0x5597135b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab5503cb80 1 --2- 172.21.15.57:0/2130902318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971352f520 0x5597135b44c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 >> 172.21.15.57:0/2130902318 conn(0x5597134a4200 msgr2=0x5597135325d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab1e7fc700 0 client.26593 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.859+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 shutdown_connections 2023-10-06T22:20:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.860+0000 7fab5503cb80 1 -- 172.21.15.57:0/2130902318 wait complete. 2023-10-06T22:20:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:34.860+0000 7fab5503cb80 20 client.26593 trim_cache size 0 max 16384 2023-10-06T22:20:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:20:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:20:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:20:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:20:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.107+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fab240a1800 msgr2=0x7fab240a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.107+0000 7fab417ff700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fab240a1800 0x7fab240a3cc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005fd0 tx=0x7fab2c005ee0 comp rx=0 tx=0).stop 2023-10-06T22:20:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.107+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 msgr2=0x7fab3c118f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.107+0000 7fab417ff700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 0x7fab3c118f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab3400ef10 tx=0x7fab3400c3a0 comp rx=0 tx=0).stop 2023-10-06T22:20:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.127+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 shutdown_connections 2023-10-06T22:20:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.127+0000 7fab417ff700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fab240a1800 0x7fab240a3cc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.127+0000 7fab417ff700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab3c11a690 0x7fab3c111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.127+0000 7fab417ff700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab3c10f620 0x7fab3c118a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.127+0000 7fab417ff700 1 --2- 172.21.15.57:0/74885744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c10fa90 0x7fab3c118f80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.127+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 >> 172.21.15.57:0/74885744 conn(0x7fab3c06cfa0 msgr2=0x7fab3c06f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.127+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 shutdown_connections 2023-10-06T22:20:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:35.128+0000 7fab417ff700 1 -- 172.21.15.57:0/74885744 wait complete. 2023-10-06T22:20:35.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:35 smithi190 ceph-mon[134891]: pgmap v178: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 297 op/s 2023-10-06T22:20:35.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:35 smithi057 ceph-mon[98492]: pgmap v178: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 297 op/s 2023-10-06T22:20:36.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:35 smithi156 ceph-mon[134358]: pgmap v178: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 297 op/s 2023-10-06T22:20:37.322 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:37 smithi190 ceph-mon[134891]: pgmap v179: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 284 op/s 2023-10-06T22:20:37.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:37 smithi057 ceph-mon[98492]: pgmap v179: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 284 op/s 2023-10-06T22:20:37.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:37 smithi156 ceph-mon[134358]: pgmap v179: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 284 op/s 2023-10-06T22:20:39.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:39 smithi156 ceph-mon[134358]: pgmap v180: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 616 op/s 2023-10-06T22:20:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:39 smithi190 ceph-mon[134891]: pgmap v180: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 616 op/s 2023-10-06T22:20:39.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:39 smithi057 ceph-mon[98492]: pgmap v180: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 616 op/s 2023-10-06T22:20:41.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:41 smithi156 ceph-mon[134358]: pgmap v181: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 579 op/s 2023-10-06T22:20:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:41 smithi190 ceph-mon[134891]: pgmap v181: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 579 op/s 2023-10-06T22:20:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:41 smithi057 ceph-mon[98492]: pgmap v181: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 579 op/s 2023-10-06T22:20:42.093 INFO:tasks.workunit.client.0.smithi057.stdout: 318 775945 539 541852 624 233427 1305 2023-10-06T22:20:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:43 smithi190 ceph-mon[134891]: pgmap v182: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2023-10-06T22:20:43.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:43 smithi057 ceph-mon[98492]: pgmap v182: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2023-10-06T22:20:43.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:43 smithi156 ceph-mon[134358]: pgmap v182: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2023-10-06T22:20:45.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:45 smithi057 ceph-mon[98492]: pgmap v183: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 567 op/s 2023-10-06T22:20:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:45 smithi156 ceph-mon[134358]: pgmap v183: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 567 op/s 2023-10-06T22:20:45.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-mon[134891]: pgmap v183: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.4 MiB/s wr, 567 op/s 2023-10-06T22:20:45.820 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.400+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001535 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-62.xml auth{0=1} v980 ap=1 s=8564 n(v0 rc2023-10-06T22:19:26.816187+0000 b8564 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b11ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001535:[<0x200000012f2/comment-62.xml v878>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001535:[<0x200000012f2/comment-62.xml v980>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.820 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.401+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001539 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-59.xml auth{0=1} v976 ap=1 s=4866 n(v0 rc2023-10-06T22:19:26.925186+0000 b4866 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5bbd4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001539:[<0x200000012f2/comment-59.xml v882>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001539:[<0x200000012f2/comment-59.xml v976>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.820 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.403+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001526 [7,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-77.xml auth{0=1} v968 ap=1 s=7252 n(v0 rc2023-10-06T22:19:26.304195+0000 b7252 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@6} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5bbd5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001526:[<0x200000012f2/comment-77.xml v869>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001526:[<0x200000012f2/comment-77.xml v968>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.403+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001545 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-48.xml auth{0=1} v972 ap=1 s=3729 n(v0 rc2023-10-06T22:19:27.276181+0000 b3729 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b62e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001545:[<0x200000012f2/comment-48.xml v918>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001545:[<0x200000012f2/comment-48.xml v972>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.405+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x100000017fa [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/picture-25.jpg auth{0=1} v1057 ap=1 s=39728 n(v0 rc2023-10-06T22:19:45.090924+0000 b39728 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@6} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b8c4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000017fa:[<0x200000012f2/picture-25.jpg v1003>,<0x20000000207/blog-25 v149636>,<0x10000000206/blogtest_in v2065>,<0x10000000201/src v1839>,<0x10000000006/blogbench-1.0 v1676>,<0x10000000005/tmp v94>,<0x100000001fb/client.0 v92>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v110>,<0x10000000001/sv_1 v125>,<0x10000000000/_nogroup v112>,<0x1/volumes v109>]//[]","memoryvalue":"(3)0x100000017fa:[<0x200000012f2/picture-25.jpg v1057>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.405+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001542 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-50.xml auth{0=1} v984 ap=1 s=5190 n(v0 rc2023-10-06T22:19:27.186182+0000 b5190 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5bbd7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001542:[<0x200000012f2/comment-50.xml v916>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001542:[<0x200000012f2/comment-50.xml v984>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.406+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001546 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-47.xml auth{0=1} v1034 ap=1 s=7383 n(v0 rc2023-10-06T22:19:27.323180+0000 b7383 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b8c5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001546:[<0x200000012f2/comment-47.xml v920>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001546:[<0x200000012f2/comment-47.xml v1034>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.407+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000152d [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-70.xml auth{0=1} v1014 ap=1 s=10056 n(v0 rc2023-10-06T22:19:26.534191+0000 b10056 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b4c9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000152d:[<0x200000012f2/comment-70.xml v900>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x1000000152d:[<0x200000012f2/comment-70.xml v1014>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.408+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001547 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-46.xml auth{0=1} v1042 ap=1 s=6166 n(v0 rc2023-10-06T22:19:27.380179+0000 b6166 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b4c8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001547:[<0x200000012f2/comment-46.xml v922>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001547:[<0x200000012f2/comment-46.xml v1042>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.409+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000152e [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-69.xml auth{0=1} v1010 ap=1 s=7556 n(v0 rc2023-10-06T22:19:26.571191+0000 b7556 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b4cb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000152e:[<0x200000012f2/comment-69.xml v902>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x1000000152e:[<0x200000012f2/comment-69.xml v1010>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.416+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001529 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-74.xml auth{0=1} v1026 ap=1 s=9947 n(v0 rc2023-10-06T22:19:26.382194+0000 b9947 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b962b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001529:[<0x200000012f2/comment-74.xml v884>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001529:[<0x200000012f2/comment-74.xml v1026>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.418+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001533 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-64.xml auth{0=1} v1050 ap=1 s=7039 n(v0 rc2023-10-06T22:19:26.763188+0000 b7039 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b964680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001533:[<0x200000012f2/comment-64.xml v906>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001533:[<0x200000012f2/comment-64.xml v1050>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.419+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000152b [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-72.xml auth{0=1} v1018 ap=1 s=8943 n(v0 rc2023-10-06T22:19:26.441193+0000 b8943 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5b965180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000152b:[<0x200000012f2/comment-72.xml v898>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x1000000152b:[<0x200000012f2/comment-72.xml v1018>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.822 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.420+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001530 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-67.xml auth{0=1} v1030 ap=1 s=9218 n(v0 rc2023-10-06T22:19:26.650190+0000 b9218 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5abef700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001530:[<0x200000012f2/comment-67.xml v904>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001530:[<0x200000012f2/comment-67.xml v1030>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.822 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.523+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001549 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-44.xml auth{0=1} v1019 ap=1 s=10002 n(v0 rc2023-10-06T22:19:27.437178+0000 b10002 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5ba86580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001549:[<0x200000012f2/comment-44.xml v924>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001549:[<0x200000012f2/comment-44.xml v1019>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.822 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.525+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001527 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-76.xml auth{0=1} v1027 ap=1 s=4814 n(v0 rc2023-10-06T22:19:26.344194+0000 b4814 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5ba88680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001527:[<0x200000012f2/comment-76.xml v888>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001527:[<0x200000012f2/comment-76.xml v1027>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.822 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.534+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001540 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-52.xml auth{0=1} v1039 ap=1 s=6261 n(v0 rc2023-10-06T22:19:27.131183+0000 b6261 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5ba89180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001540:[<0x200000012f2/comment-52.xml v920>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001540:[<0x200000012f2/comment-52.xml v1039>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v68>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:45.822 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:20:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:20:45.536+0000 7f57fcb43700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001531 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-66.xml auth{0=1} v1031 ap=1 s=6957 n(v0 rc2023-10-06T22:19:26.695189+0000 b6957 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55de5bba4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001531:[<0x200000012f2/comment-66.xml v892>,<0x20000000207/blog-25 v143026>,<0x10000000206/blogtest_in v1986>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1672>,<0x10000000005/tmp v90>,<0x100000001fb/client.0 v88>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v106>,<0x10000000001/sv_1 v121>,<0x10000000000/_nogroup v108>,<0x1/volumes v105>]//[]","memoryvalue":"(3)0x10000001531:[<0x200000012f2/comment-66.xml v1031>,<0x20000000207/blog-25 v160732>,<0x10000000206/blogtest_in v2363>,<0x10000000201/src v1851>,<0x10000000006/blogbench-1.0 v1636>,<0x10000000005/tmp v54>,<0x100000001fb/client.0 v52>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v69>,<0x10000000001/sv_1 v85>,<0x10000000000/_nogroup v71>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:20:46.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001535 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-62.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001539 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-59.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001526 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-77.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001545 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-48.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000017fa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/picture-25.jpg) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001542 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-50.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001546 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-47.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000152d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-70.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001547 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-46.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000152e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-69.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001529 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-74.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001533 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-64.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000152b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-72.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001530 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-67.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001549 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-44.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001527 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-76.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001540 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-52.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.719 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:46 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001531 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-66.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001535 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-62.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001539 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-59.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001526 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-77.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001545 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-48.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000017fa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/picture-25.jpg) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001542 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-50.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001546 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-47.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000152d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-70.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001547 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-46.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000152e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-69.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001529 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-74.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001533 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-64.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000152b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-72.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001530 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-67.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001549 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-44.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001527 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-76.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001540 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-52.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:46 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001531 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-66.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001535 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-62.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.873 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001539 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-59.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.874 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001526 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-77.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.874 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001545 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-48.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.874 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000017fa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/picture-25.jpg) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.874 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001542 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-50.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.874 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001546 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-47.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.874 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000152d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-70.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001547 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-46.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000152e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-69.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001529 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-74.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001533 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-64.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000152b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-72.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001530 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-67.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001549 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-44.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001527 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-76.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001540 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-52.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:46.875 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:46 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001531 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-66.xml) see mds.f log and `damage ls` output for details 2023-10-06T22:20:47.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:47 smithi057 ceph-mon[98492]: pgmap v184: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 540 op/s 2023-10-06T22:20:47.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:47 smithi156 ceph-mon[134358]: pgmap v184: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 540 op/s 2023-10-06T22:20:47.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:47 smithi190 ceph-mon[134891]: pgmap v184: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 540 op/s 2023-10-06T22:20:49.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a[98468]: 2023-10-06T22:20:49.395+0000 7f57da29d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:20:49.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:49 smithi057 ceph-mon[98492]: pgmap v185: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 612 op/s 2023-10-06T22:20:49.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:49 smithi156 ceph-mon[134358]: pgmap v185: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 612 op/s 2023-10-06T22:20:49.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:49 smithi190 ceph-mon[134891]: pgmap v185: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 612 op/s 2023-10-06T22:20:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.328+0000 7f3f747ff700 1 -- 172.21.15.57:0/3779947294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f6c10d510 msgr2=0x7f3f6c10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.328+0000 7f3f747ff700 1 --2- 172.21.15.57:0/3779947294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f6c10d510 0x7f3f6c10d8f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009990 tx=0x7f3f68009ca0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.329+0000 7f3f747ff700 1 -- 172.21.15.57:0/3779947294 shutdown_connections 2023-10-06T22:20:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.329+0000 7f3f747ff700 1 --2- 172.21.15.57:0/3779947294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c108650 0x7f3f6c078ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.329+0000 7f3f747ff700 1 --2- 172.21.15.57:0/3779947294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f6c10d510 0x7f3f6c10d8f0 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009990 tx=0x7f3f68009ca0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.329+0000 7f3f747ff700 1 --2- 172.21.15.57:0/3779947294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6c107c90 0x7f3f6c108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.329+0000 7f3f747ff700 1 -- 172.21.15.57:0/3779947294 >> 172.21.15.57:0/3779947294 conn(0x7f3f6c06cf90 msgr2=0x7f3f6c06d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.342+0000 7f3f747ff700 1 -- 172.21.15.57:0/3779947294 shutdown_connections 2023-10-06T22:20:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.342+0000 7f3f747ff700 1 -- 172.21.15.57:0/3779947294 wait complete. 2023-10-06T22:20:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.344+0000 7f3f747ff700 1 Processor -- start 2023-10-06T22:20:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.351+0000 7f3f747ff700 1 -- start start 2023-10-06T22:20:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.351+0000 7f3f747ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c107c90 0x7f3f6c083490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.352+0000 7f3f747ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f6c108650 0x7f3f6c07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.352+0000 7f3f747ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6c07d9b0 0x7f3f6c07de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.352+0000 7f3f747ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c083d00 con 0x7f3f6c108650 2023-10-06T22:20:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.352+0000 7f3f747ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1bb010 con 0x7f3f6c07d9b0 2023-10-06T22:20:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.352+0000 7f3f747ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1bb180 con 0x7f3f6c107c90 2023-10-06T22:20:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.352+0000 7f3f7259b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c107c90 0x7f3f6c083490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.353+0000 7f3f7259b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c107c90 0x7f3f6c083490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33630/0 (socket says 172.21.15.57:33630) 2023-10-06T22:20:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.353+0000 7f3f7259b700 1 -- 172.21.15.57:0/1971972839 learned_addr learned my addr 172.21.15.57:0/1971972839 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.353+0000 7f3f7259b700 1 -- 172.21.15.57:0/1971972839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6c07d9b0 msgr2=0x7f3f6c07de60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:20:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.353+0000 7f3f7259b700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6c07d9b0 0x7f3f6c07de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.353+0000 7f3f7259b700 1 -- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f6c108650 msgr2=0x7f3f6c07d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.353+0000 7f3f7259b700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f6c108650 0x7f3f6c07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.353+0000 7f3f7259b700 1 -- 172.21.15.57:0/1971972839 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680095f0 con 0x7f3f6c107c90 2023-10-06T22:20:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.354+0000 7f3f7259b700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c107c90 0x7f3f6c083490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3f68003730 tx=0x7f3f6800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.585+0000 7f3f637fe700 1 -- 172.21.15.57:0/1971972839 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6801d070 con 0x7f3f6c107c90 2023-10-06T22:20:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.585+0000 7f3f637fe700 1 -- 172.21.15.57:0/1971972839 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f680054f0 con 0x7f3f6c107c90 2023-10-06T22:20:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.585+0000 7f3f637fe700 1 -- 172.21.15.57:0/1971972839 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68004a40 con 0x7f3f6c107c90 2023-10-06T22:20:50.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.585+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c1bb400 con 0x7f3f6c107c90 2023-10-06T22:20:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.585+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c1bb770 con 0x7f3f6c107c90 2023-10-06T22:20:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.602+0000 7f3f88109b80 1 Processor -- start 2023-10-06T22:20:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.603+0000 7f3f88109b80 1 -- start start 2023-10-06T22:20:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.608+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b65f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.608+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 0x5633b6601e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.608+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b66047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.608+0000 7f3f88109b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b6560400 con 0x5633b66023d0 2023-10-06T22:20:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.608+0000 7f3f88109b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b6551bc0 con 0x5633b65f7520 2023-10-06T22:20:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.608+0000 7f3f88109b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b654f4e0 con 0x5633b65f6c00 2023-10-06T22:20:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.611+0000 7f3f637fe700 1 -- 172.21.15.57:0/1971972839 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3f68004ba0 con 0x7f3f6c107c90 2023-10-06T22:20:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.614+0000 7f3f637fe700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3f580a1800 0x7f3f580a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.614+0000 7f3f637fe700 1 -- 172.21.15.57:0/1971972839 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f3f680ce2b0 con 0x7f3f6c107c90 2023-10-06T22:20:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.615+0000 7f3f72d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b65f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.615+0000 7f3f72d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b65f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33646/0 (socket says 172.21.15.57:33646) 2023-10-06T22:20:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.615+0000 7f3f72d9c700 1 -- 172.21.15.57:0/498781988 learned_addr learned my addr 172.21.15.57:0/498781988 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.615+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b66047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:50.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.617+0000 7f3f72d9c700 1 -- 172.21.15.57:0/498781988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 msgr2=0x5633b6601e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:20:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.617+0000 7f3f72d9c700 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 0x5633b6601e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.617+0000 7f3f72d9c700 1 -- 172.21.15.57:0/498781988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 msgr2=0x5633b66047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.617+0000 7f3f72d9c700 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b66047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.617+0000 7f3f72d9c700 1 -- 172.21.15.57:0/498781988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b6375d80 con 0x5633b65f6c00 2023-10-06T22:20:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.617+0000 7f3f72d9c700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3f580a1800 0x7f3f580a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.618+0000 7f3f72d9c700 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b65f6fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00bb30 tx=0x7f3f5c00be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59b87b6c567d957e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.642+0000 7f3f72d9c700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3f580a1800 0x7f3f580a3cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c01dfd0 tx=0x7f3f5c01db40 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:50.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:50 smithi057 ceph-mon[98492]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:20:50.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:50 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:active 2023-10-06T22:20:50.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:50 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:20:50.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:50 smithi156 ceph-mon[134358]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:20:50.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:50 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:active 2023-10-06T22:20:50.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:50 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:20:50.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:50 smithi190 ceph-mon[134891]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:20:50.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:50 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] up:active 2023-10-06T22:20:50.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:50 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:20:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.854+0000 7f3f60ff9700 1 -- 172.21.15.57:0/498781988 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00fc10 con 0x5633b65f6c00 2023-10-06T22:20:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.854+0000 7f3f60ff9700 1 -- 172.21.15.57:0/498781988 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f5c00fd70 con 0x5633b65f6c00 2023-10-06T22:20:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.854+0000 7f3f60ff9700 1 -- 172.21.15.57:0/498781988 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c01d900 con 0x5633b65f6c00 2023-10-06T22:20:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.855+0000 7f3f88109b80 1 -- 172.21.15.57:0/498781988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 msgr2=0x5633b65f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.855+0000 7f3f88109b80 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b65f6fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00bb30 tx=0x7f3f5c00be40 comp rx=0 tx=0).stop 2023-10-06T22:20:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.887+0000 7f3f88109b80 1 -- 172.21.15.57:0/498781988 shutdown_connections 2023-10-06T22:20:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.887+0000 7f3f88109b80 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b66047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.887+0000 7f3f88109b80 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b65f6fe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.887+0000 7f3f88109b80 1 --2- 172.21.15.57:0/498781988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 0x5633b6601e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.887+0000 7f3f88109b80 1 -- 172.21.15.57:0/498781988 >> 172.21.15.57:0/498781988 conn(0x5633b656c200 msgr2=0x5633b65f3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.887+0000 7f3f88109b80 1 -- 172.21.15.57:0/498781988 shutdown_connections 2023-10-06T22:20:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.887+0000 7f3f88109b80 1 -- 172.21.15.57:0/498781988 wait complete. 2023-10-06T22:20:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.888+0000 7f3f88109b80 1 -- start start 2023-10-06T22:20:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.889+0000 7f3f88109b80 1 Processor -- start 2023-10-06T22:20:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.890+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f6c00 0x5633b6601e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.890+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b65ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.890+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.890+0000 7f3f88109b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b654add0 con 0x5633b66023d0 2023-10-06T22:20:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.890+0000 7f3f88109b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b64e76b0 con 0x5633b65f6c00 2023-10-06T22:20:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.890+0000 7f3f88109b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b64e7350 con 0x5633b65f7520 2023-10-06T22:20:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.890+0000 7f3f88109b80 10 client.0 _ll_register_callbacks cb 0x5633b63447a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.895+0000 7f3f72d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.895+0000 7f3f72d9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65ff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43646/0 (socket says 172.21.15.57:43646) 2023-10-06T22:20:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.895+0000 7f3f72d9c700 1 -- 172.21.15.57:0/4133798668 learned_addr learned my addr 172.21.15.57:0/4133798668 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.899+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b65ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.900+0000 7f3f7259b700 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f6c00 0x5633b6601e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.900+0000 7f3f71d9a700 1 -- 172.21.15.57:0/4133798668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f6c00 msgr2=0x5633b6601e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.900+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f6c00 0x5633b6601e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.900+0000 7f3f71d9a700 1 -- 172.21.15.57:0/4133798668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 msgr2=0x5633b65ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.900+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.900+0000 7f3f71d9a700 1 -- 172.21.15.57:0/4133798668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c00b760 con 0x5633b65f7520 2023-10-06T22:20:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:50.901+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b65ff090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c06d700 tx=0x7f3f6400e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa24ed0013ff791 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.107+0000 7f3f55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.107+0000 7f3f55ffb700 1 -- 172.21.15.57:0/4133798668 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64004bb0 con 0x5633b65f7520 2023-10-06T22:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.107+0000 7f3f55ffb700 1 -- 172.21.15.57:0/4133798668 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f3f64004d10 con 0x5633b65f7520 2023-10-06T22:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.107+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633b65ead50 con 0x5633b65f7520 2023-10-06T22:20:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.111+0000 7f3f55ffb700 1 -- 172.21.15.57:0/4133798668 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f6400ea40 con 0x5633b65f7520 2023-10-06T22:20:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.113+0000 7f3f88109b80 10 client.26605 fetch_fsmap learned FSMap version 43 2023-10-06T22:20:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.113+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5633b635ec00 con 0x5633b65f7520 2023-10-06T22:20:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.133+0000 7f3f55ffb700 1 -- 172.21.15.57:0/4133798668 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f64022030 con 0x5633b65f7520 2023-10-06T22:20:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.133+0000 7f3f55ffb700 1 -- 172.21.15.57:0/4133798668 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f3f64013070 con 0x5633b65f7520 2023-10-06T22:20:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.144+0000 7f3f88109b80 10 client.26605 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T22:20:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.144+0000 7f3f88109b80 10 client.26605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.144+0000 7f3f88109b80 20 client.26605 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.144+0000 7f3f88109b80 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b661a920 0x5633b661cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.144+0000 7f3f88109b80 4 client.26605 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.144+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633b6543db0 con 0x5633b661a920 2023-10-06T22:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.145+0000 7f3f7259b700 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b661a920 0x5633b661cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.177+0000 7f3f7259b700 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b661a920 0x5633b661cd40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aecd705c27e44f16 server_cookie=feb2737370ad5147 in_seq=0 out_seq=0 2023-10-06T22:20:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.177+0000 7f3f55ffb700 10 client.26605 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.180+0000 7f3f55ffb700 1 -- 172.21.15.57:0/4133798668 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5633b6543db0 con 0x5633b661a920 2023-10-06T22:20:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.180+0000 7f3f55ffb700 10 client.26605 handle_command_reply: tid=0 2023-10-06T22:20:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.180+0000 7f3f88109b80 1 client.26605 shutdown 2023-10-06T22:20:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 msgr2=0x5633b65ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b65ff090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c06d700 tx=0x7f3f6400e430 comp rx=0 tx=0).stop 2023-10-06T22:20:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 shutdown_connections 2023-10-06T22:20:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b661a920 0x5633b661cd40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f6c00 0x5633b6601e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 --2- 172.21.15.57:0/4133798668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b65ff090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.181+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 >> 172.21.15.57:0/4133798668 conn(0x5633b656c200 msgr2=0x5633b65f2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.182+0000 7f3f55ffb700 0 client.26605 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.182+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 shutdown_connections 2023-10-06T22:20:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.182+0000 7f3f88109b80 1 -- 172.21.15.57:0/4133798668 wait complete. 2023-10-06T22:20:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.183+0000 7f3f88109b80 20 client.26605 trim_cache size 0 max 16384 2023-10-06T22:20:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.539+0000 7f3f88109b80 1 Processor -- start 2023-10-06T22:20:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.549+0000 7f3f88109b80 1 -- start start 2023-10-06T22:20:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.549+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b66019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.549+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 0x5633b65fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.549+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65fc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.549+0000 7f3f88109b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b63cb8c0 con 0x5633b66023d0 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.549+0000 7f3f88109b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b66047c0 con 0x5633b65f7520 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.549+0000 7f3f88109b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b65faed0 con 0x5633b65f6c00 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.550+0000 7f3f7259b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b66019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.550+0000 7f3f7259b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b66019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33660/0 (socket says 172.21.15.57:33660) 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.550+0000 7f3f7259b700 1 -- 172.21.15.57:0/2657287560 learned_addr learned my addr 172.21.15.57:0/2657287560 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.550+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 0x5633b65fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.554+0000 7f3f7259b700 1 -- 172.21.15.57:0/2657287560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 msgr2=0x5633b65fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.554+0000 7f3f7259b700 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 0x5633b65fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.554+0000 7f3f7259b700 1 -- 172.21.15.57:0/2657287560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 msgr2=0x5633b65fc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.554+0000 7f3f7259b700 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.554+0000 7f3f7259b700 1 -- 172.21.15.57:0/2657287560 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b639c1f0 con 0x5633b65f6c00 2023-10-06T22:20:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.560+0000 7f3f7259b700 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b66019b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f680970b0 tx=0x7f3f68097040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e553afea9d3996a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:51.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:51 smithi156 ceph-mon[134358]: pgmap v186: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 307 op/s 2023-10-06T22:20:51.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:51 smithi190 ceph-mon[134891]: pgmap v186: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 307 op/s 2023-10-06T22:20:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.852+0000 7f3f617fa700 1 -- 172.21.15.57:0/2657287560 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6809f480 con 0x5633b65f6c00 2023-10-06T22:20:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.852+0000 7f3f617fa700 1 -- 172.21.15.57:0/2657287560 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f680a3630 con 0x5633b65f6c00 2023-10-06T22:20:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.852+0000 7f3f617fa700 1 -- 172.21.15.57:0/2657287560 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6809f150 con 0x5633b65f6c00 2023-10-06T22:20:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.853+0000 7f3f88109b80 1 -- 172.21.15.57:0/2657287560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 msgr2=0x5633b66019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.853+0000 7f3f88109b80 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b66019b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f680970b0 tx=0x7f3f68097040 comp rx=0 tx=0).stop 2023-10-06T22:20:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.860+0000 7f3f88109b80 1 -- 172.21.15.57:0/2657287560 shutdown_connections 2023-10-06T22:20:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.860+0000 7f3f88109b80 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b66023d0 0x5633b65fc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.860+0000 7f3f88109b80 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f6c00 0x5633b66019b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.860+0000 7f3f88109b80 1 --2- 172.21.15.57:0/2657287560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b65f7520 0x5633b65fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.860+0000 7f3f88109b80 1 -- 172.21.15.57:0/2657287560 >> 172.21.15.57:0/2657287560 conn(0x5633b656c200 msgr2=0x5633b65fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.864+0000 7f3f88109b80 1 -- 172.21.15.57:0/2657287560 shutdown_connections 2023-10-06T22:20:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.865+0000 7f3f88109b80 1 -- 172.21.15.57:0/2657287560 wait complete. 2023-10-06T22:20:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.865+0000 7f3f88109b80 1 -- start start 2023-10-06T22:20:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.866+0000 7f3f88109b80 1 Processor -- start 2023-10-06T22:20:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.866+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b65f6c00 0x5633b6611790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.867+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b66666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.867+0000 7f3f88109b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b66023d0 0x5633b6666c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.867+0000 7f3f88109b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b666a4a0 con 0x5633b65f6c00 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.867+0000 7f3f88109b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b6667160 con 0x5633b66023d0 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.867+0000 7f3f88109b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b6667320 con 0x5633b65f7520 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.867+0000 7f3f88109b80 10 client.0 _ll_register_callbacks cb 0x7f3f64010570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.867+0000 7f3f72d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b66023d0 0x5633b6666c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.870+0000 7f3f71d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b66666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.870+0000 7f3f72d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b66023d0 0x5633b6666c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46830/0 (socket says 172.21.15.57:46830) 2023-10-06T22:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.870+0000 7f3f72d9c700 1 -- 172.21.15.57:0/1346461825 learned_addr learned my addr 172.21.15.57:0/1346461825 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.871+0000 7f3f7259b700 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b65f6c00 0x5633b6611790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.871+0000 7f3f71d9a700 1 -- 172.21.15.57:0/1346461825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b66023d0 msgr2=0x5633b6666c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.871+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b66023d0 0x5633b6666c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.871+0000 7f3f71d9a700 1 -- 172.21.15.57:0/1346461825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b65f6c00 msgr2=0x5633b6611790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.871+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b65f6c00 0x5633b6611790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.871+0000 7f3f71d9a700 1 -- 172.21.15.57:0/1346461825 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f680cd040 con 0x5633b65f7520 2023-10-06T22:20:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:51.875+0000 7f3f71d9a700 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b66666e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f64015c50 tx=0x7f3f640086e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132a726ec2d43666 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:20:51.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:51 smithi057 ceph-mon[98492]: pgmap v186: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 307 op/s 2023-10-06T22:20:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.059+0000 7f3f55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:20:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.059+0000 7f3f55ffb700 1 -- 172.21.15.57:0/1346461825 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64008310 con 0x5633b65f7520 2023-10-06T22:20:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.059+0000 7f3f55ffb700 1 -- 172.21.15.57:0/1346461825 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f3f6400ea40 con 0x5633b65f7520 2023-10-06T22:20:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.059+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f5c00fd70 con 0x5633b65f7520 2023-10-06T22:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.082+0000 7f3f55ffb700 1 -- 172.21.15.57:0/1346461825 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f6400ec40 con 0x5633b65f7520 2023-10-06T22:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.082+0000 7f3f88109b80 10 client.26611 fetch_fsmap learned FSMap version 43 2023-10-06T22:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.082+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5633b6374250 con 0x5633b65f7520 2023-10-06T22:20:52.093 INFO:tasks.workunit.client.0.smithi057.stdout: 326 784018 603 545963 516 244919 1558 2023-10-06T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.091+0000 7f3f55ffb700 1 -- 172.21.15.57:0/1346461825 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f64034030 con 0x5633b65f7520 2023-10-06T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.091+0000 7f3f55ffb700 1 -- 172.21.15.57:0/1346461825 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f3f6400e150 con 0x5633b65f7520 2023-10-06T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.093+0000 7f3f88109b80 10 client.26611 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.093+0000 7f3f88109b80 10 client.26611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.093+0000 7f3f88109b80 20 client.26611 populate_metadata read hostname 'smithi057' 2023-10-06T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.093+0000 7f3f88109b80 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b6609eb0 0x5633b660c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.093+0000 7f3f88109b80 4 client.26611 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.093+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633b6543ad0 con 0x5633b6609eb0 2023-10-06T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.093+0000 7f3f7259b700 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b6609eb0 0x5633b660c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.094+0000 7f3f7259b700 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b6609eb0 0x5633b660c2d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd39b7b3bbe798 server_cookie=f6cb7f3678ef788a in_seq=0 out_seq=0 2023-10-06T22:20:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:52.106+0000 7f3f55ffb700 10 client.26611 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:52.655 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:20:53.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:53 smithi057 ceph-mon[98492]: pgmap v187: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 331 op/s 2023-10-06T22:20:54.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:53 smithi156 ceph-mon[134358]: pgmap v187: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 331 op/s 2023-10-06T22:20:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:53 smithi190 ceph-mon[134891]: pgmap v187: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 331 op/s 2023-10-06T22:20:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.944+0000 7f3f55ffb700 1 -- 172.21.15.57:0/1346461825 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f3f680b7070 con 0x5633b6609eb0 2023-10-06T22:20:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.944+0000 7f3f55ffb700 10 client.26611 handle_command_reply: tid=0 2023-10-06T22:20:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.944+0000 7f3f88109b80 1 client.26611 shutdown 2023-10-06T22:20:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 msgr2=0x5633b66666e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b66666e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f64015c50 tx=0x7f3f640086e0 comp rx=0 tx=0).stop 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 shutdown_connections 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5633b6609eb0 0x5633b660c2d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5633b66023d0 0x5633b6666c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633b65f6c00 0x5633b6611790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 --2- 172.21.15.57:0/1346461825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b65f7520 0x5633b66666e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 >> 172.21.15.57:0/1346461825 conn(0x5633b656c200 msgr2=0x5633b65f8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.945+0000 7f3f55ffb700 0 client.26611 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:20:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.946+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 shutdown_connections 2023-10-06T22:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.946+0000 7f3f88109b80 1 -- 172.21.15.57:0/1346461825 wait complete. 2023-10-06T22:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:54.946+0000 7f3f88109b80 20 client.26611 trim_cache size 0 max 16384 2023-10-06T22:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:20:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.071+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3f580a1800 msgr2=0x7f3f580a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.071+0000 7f3f747ff700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3f580a1800 0x7f3f580a3cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c01dfd0 tx=0x7f3f5c01db40 comp rx=0 tx=0).stop 2023-10-06T22:20:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.071+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c107c90 msgr2=0x7f3f6c083490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:20:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.071+0000 7f3f747ff700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c107c90 0x7f3f6c083490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3f68003730 tx=0x7f3f6800f740 comp rx=0 tx=0).stop 2023-10-06T22:20:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.076+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 shutdown_connections 2023-10-06T22:20:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.076+0000 7f3f747ff700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3f580a1800 0x7f3f580a3cc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.076+0000 7f3f747ff700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6c07d9b0 0x7f3f6c07de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.076+0000 7f3f747ff700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f6c107c90 0x7f3f6c083490 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.076+0000 7f3f747ff700 1 --2- 172.21.15.57:0/1971972839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3f6c108650 0x7f3f6c07d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:20:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.076+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 >> 172.21.15.57:0/1971972839 conn(0x7f3f6c06cf90 msgr2=0x7f3f6c072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:20:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.084+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 shutdown_connections 2023-10-06T22:20:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:20:55.109+0000 7f3f747ff700 1 -- 172.21.15.57:0/1971972839 wait complete. 2023-10-06T22:20:55.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:55 smithi057 ceph-mon[98492]: pgmap v188: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 276 op/s 2023-10-06T22:20:56.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:55 smithi156 ceph-mon[134358]: pgmap v188: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 276 op/s 2023-10-06T22:20:56.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:55 smithi190 ceph-mon[134891]: pgmap v188: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 276 op/s 2023-10-06T22:20:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:57 smithi057 ceph-mon[98492]: pgmap v189: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 322 op/s 2023-10-06T22:20:57.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:57 smithi156 ceph-mon[134358]: pgmap v189: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 322 op/s 2023-10-06T22:20:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:57 smithi190 ceph-mon[134891]: pgmap v189: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 322 op/s 2023-10-06T22:20:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:20:59 smithi190 ceph-mon[134891]: pgmap v190: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 595 op/s 2023-10-06T22:20:59.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:20:59 smithi057 ceph-mon[98492]: pgmap v190: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 595 op/s 2023-10-06T22:20:59.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:20:59 smithi156 ceph-mon[134358]: pgmap v190: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 595 op/s 2023-10-06T22:21:00.720 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.431+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c4 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-18.jpg auth{0=1} v1279 ap=1 s=48659 n(v0 rc2023-10-06T22:16:41.380592+0000 b48659 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149b9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c4:[<0x200000013f1/picture-18.jpg v1279>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003c4:[<0x200000013f1/picture-18.jpg v1279>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.191 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.431+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d0 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-13.jpg auth{0=1} v1277 ap=1 s=14927 n(v0 rc2023-10-06T22:16:41.667588+0000 b14927 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149b8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d0:[<0x200000013f1/picture-13.jpg v1277>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d0:[<0x200000013f1/picture-13.jpg v1277>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.191 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.432+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c1 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-19.jpg auth{0=1} v1281 ap=1 s=29581 n(v0 rc2023-10-06T22:16:41.201595+0000 b29581 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149d9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c1:[<0x200000013f1/picture-19.jpg v1281>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003c1:[<0x200000013f1/picture-19.jpg v1281>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.191 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.432+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046c [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-29.xml auth{0=1} v1362 ap=1 s=7027 n(v0 rc2023-10-06T22:17:16.178085+0000 b7027 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149d9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000046c:[<0x200000013f1/comment-29.xml v1317>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000046c:[<0x200000013f1/comment-29.xml v1362>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.191 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.435+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a0 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-35.jpg auth{0=1} v1283 ap=1 s=32846 n(v0 rc2023-10-06T22:16:39.867614+0000 b32846 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149db180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a0:[<0x200000013f1/picture-35.jpg v1283>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003a0:[<0x200000013f1/picture-35.jpg v1283>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.437+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000493 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-5.xml auth{0=1} v1368 ap=1 s=6668 n(v0 rc2023-10-06T22:17:16.606079+0000 b6668 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149d8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000493:[<0x200000013f1/comment-5.xml v1354>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000493:[<0x200000013f1/comment-5.xml v1368>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.438+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ce [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-14.jpg auth{0=1} v1267 ap=1 s=55979 n(v0 rc2023-10-06T22:16:41.651588+0000 b55979 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ce:[<0x200000013f1/picture-14.jpg v1267>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003ce:[<0x200000013f1/picture-14.jpg v1267>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.441+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dd [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-2.jpg auth{0=1} v1269 ap=1 s=9143 n(v0 rc2023-10-06T22:16:41.714587+0000 b9143 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149db700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003dd:[<0x200000013f1/picture-2.jpg v1269>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003dd:[<0x200000013f1/picture-2.jpg v1269>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.442+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003db [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-4.jpg auth{0=1} v1271 ap=1 s=21039 n(v0 rc2023-10-06T22:16:41.710587+0000 b21039 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149dac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003db:[<0x200000013f1/picture-4.jpg v1271>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003db:[<0x200000013f1/picture-4.jpg v1271>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.443+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d6 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-7.jpg auth{0=1} v1273 ap=1 s=13248 n(v0 rc2023-10-06T22:16:41.686587+0000 b13248 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149f3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d6:[<0x200000013f1/picture-7.jpg v1273>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d6:[<0x200000013f1/picture-7.jpg v1273>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.445+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000048d [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-11.xml auth{0=1} v1366 ap=1 s=2311 n(v0 rc2023-10-06T22:17:16.512080+0000 b2311 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149f4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000048d:[<0x200000013f1/comment-11.xml v1335>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000048d:[<0x200000013f1/comment-11.xml v1366>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.445+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d4 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-9.jpg auth{0=1} v1275 ap=1 s=40509 n(v0 rc2023-10-06T22:16:41.680588+0000 b40509 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149f3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d4:[<0x200000013f1/picture-9.jpg v1275>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d4:[<0x200000013f1/picture-9.jpg v1275>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.446+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000484 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-20.xml auth{0=1} v1364 ap=1 s=8864 n(v0 rc2023-10-06T22:17:16.323083+0000 b8864 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149f4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000484:[<0x200000013f1/comment-20.xml v1333>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000484:[<0x200000013f1/comment-20.xml v1364>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.447+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d3 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-10.jpg auth{0=1} v1263 ap=1 s=32521 n(v0 rc2023-10-06T22:16:41.678588+0000 b32521 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a27600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d3:[<0x200000013f1/picture-10.jpg v1263>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d3:[<0x200000013f1/picture-10.jpg v1263>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.192 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.448+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000460 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-37.xml auth{0=1} v1367 ap=1 s=4538 n(v0 rc2023-10-06T22:17:16.109086+0000 b4538 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a27080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000460:[<0x200000013f1/comment-37.xml v1299>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000460:[<0x200000013f1/comment-37.xml v1367>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.448+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000488 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-16.xml auth{0=1} v1361 ap=1 s=7342 n(v0 rc2023-10-06T22:17:16.379082+0000 b7342 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a28100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000488:[<0x200000013f1/comment-16.xml v1347>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000488:[<0x200000013f1/comment-16.xml v1361>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.458+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000d9b [7,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-69.xml auth{0=1} v1390 ap=1 s=15985 n(v0 rc2023-10-06T22:18:53.295673+0000 b15985 1=1+0) (iversion lock) caps={15228=pAsLsXsFscr/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149f2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000d9b:[<0x200000013f1/article-69.xml v1383>,<0x20000000207/blog-26 v144230>,<0x10000000206/blogtest_in v1758>,<0x10000000201/src v1829>,<0x10000000006/blogbench-1.0 v1666>,<0x10000000005/tmp v84>,<0x100000001fb/client.0 v82>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v100>,<0x10000000001/sv_1 v115>,<0x10000000000/_nogroup v102>,<0x1/volumes v99>]//[]","memoryvalue":"(3)0x10000000d9b:[<0x200000013f1/article-69.xml v1390>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.458+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003de [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-1.jpg auth{0=1} v1265 ap=1 s=62380 n(v0 rc2023-10-06T22:16:41.716587+0000 b62380 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a27b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003de:[<0x200000013f1/picture-1.jpg v1265>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003de:[<0x200000013f1/picture-1.jpg v1265>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.460+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d7 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-6.jpg auth{0=1} v1267 ap=1 s=20491 n(v0 rc2023-10-06T22:16:41.704587+0000 b20491 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a29700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d7:[<0x200000013f1/picture-6.jpg v1267>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d7:[<0x200000013f1/picture-6.jpg v1267>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.461+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000475 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-25.xml auth{0=1} v1359 ap=1 s=2875 n(v0 rc2023-10-06T22:17:16.214084+0000 b2875 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a2f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000475:[<0x200000013f1/comment-25.xml v1321>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000475:[<0x200000013f1/comment-25.xml v1359>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.461+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b1 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-28.jpg auth{0=1} v1273 ap=1 s=55860 n(v0 rc2023-10-06T22:16:40.398606+0000 b55860 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a2fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b1:[<0x200000013f1/picture-28.jpg v1273>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003b1:[<0x200000013f1/picture-28.jpg v1273>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.509+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000465 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-34.xml auth{0=1} v1365 ap=1 s=9959 n(v0 rc2023-10-06T22:17:16.134086+0000 b9959 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a2f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000465:[<0x200000013f1/comment-34.xml v1307>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000465:[<0x200000013f1/comment-34.xml v1365>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.510+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c6 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-17.jpg auth{0=1} v1269 ap=1 s=59541 n(v0 rc2023-10-06T22:16:41.541590+0000 b59541 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a38580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003c6:[<0x200000013f1/picture-17.jpg v1269>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003c6:[<0x200000013f1/picture-17.jpg v1269>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.512+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b8 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-24.jpg auth{0=1} v1271 ap=1 s=48119 n(v0 rc2023-10-06T22:16:40.709602+0000 b48119 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a39b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b8:[<0x200000013f1/picture-24.jpg v1271>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003b8:[<0x200000013f1/picture-24.jpg v1271>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.193 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.513+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a7 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-33.jpg auth{0=1} v1261 ap=1 s=16132 n(v0 rc2023-10-06T22:16:40.009612+0000 b16132 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a3a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a7:[<0x200000013f1/picture-33.jpg v1261>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003a7:[<0x200000013f1/picture-33.jpg v1261>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.194 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.515+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000381 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-42.jpg auth{0=1} v1257 ap=1 s=52900 n(v0 rc2023-10-06T22:16:39.319622+0000 b52900 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a3b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000381:[<0x200000013f1/picture-42.jpg v1257>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000381:[<0x200000013f1/picture-42.jpg v1257>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.194 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.516+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000479 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-23.xml auth{0=1} v1363 ap=1 s=6052 n(v0 rc2023-10-06T22:17:16.249084+0000 b6052 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a40580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000479:[<0x200000013f1/comment-23.xml v1327>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000479:[<0x200000013f1/comment-23.xml v1363>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.194 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.516+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045c [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-41.xml auth{0=1} v1369 ap=1 s=6516 n(v0 rc2023-10-06T22:17:16.082086+0000 b6516 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a31180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000045c:[<0x200000013f1/comment-41.xml v1293>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000045c:[<0x200000013f1/comment-41.xml v1369>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.194 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.517+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000396 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-37.jpg auth{0=1} v1275 ap=1 s=17404 n(v0 rc2023-10-06T22:16:39.666617+0000 b17404 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a38000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000396:[<0x200000013f1/picture-37.jpg v1275>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000396:[<0x200000013f1/picture-37.jpg v1275>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.194 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.517+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038e [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-39.jpg auth{0=1} v1259 ap=1 s=59131 n(v0 rc2023-10-06T22:16:39.522619+0000 b59131 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a40000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000038e:[<0x200000013f1/picture-39.jpg v1259>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000038e:[<0x200000013f1/picture-39.jpg v1259>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.194 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.716+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000480 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-22.xml auth{0=1} v1369 ap=1 s=6341 n(v0 rc2023-10-06T22:17:16.278083+0000 b6341 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a73080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000480:[<0x200000013f1/comment-22.xml v1329>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000480:[<0x200000013f1/comment-22.xml v1369>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.194 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.717+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000496 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-2.xml auth{0=1} v1379 ap=1 s=3113 n(v0 rc2023-10-06T22:17:16.635078+0000 b3113 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a73600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000496:[<0x200000013f1/comment-2.xml v1357>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000496:[<0x200000013f1/comment-2.xml v1379>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.095+0000 7f4be6dff700 1 -- 172.21.15.57:0/2143004581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be010f640 msgr2=0x7f4be0107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.095+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2143004581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be010f640 0x7f4be0107eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00098f0 tx=0x7f4bd0009c00 comp rx=0 tx=0).stop 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.104+0000 7f4be6dff700 1 -- 172.21.15.57:0/2143004581 shutdown_connections 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.104+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2143004581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be011a690 0x7f4be011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.104+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2143004581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be010f640 0x7f4be0107eb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.104+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2143004581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be01083f0 0x7f4be0108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.104+0000 7f4be6dff700 1 -- 172.21.15.57:0/2143004581 >> 172.21.15.57:0/2143004581 conn(0x7f4be006d1a0 msgr2=0x7f4be006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.104+0000 7f4be6dff700 1 -- 172.21.15.57:0/2143004581 shutdown_connections 2023-10-06T22:21:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.104+0000 7f4be6dff700 1 -- 172.21.15.57:0/2143004581 wait complete. 2023-10-06T22:21:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.106+0000 7f4be6dff700 1 Processor -- start 2023-10-06T22:21:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.106+0000 7f4be6dff700 1 -- start start 2023-10-06T22:21:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4be6dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be01083f0 0x7f4be01159b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4be6dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be010f640 0x7f4be0115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4be6dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be011a690 0x7f4be01a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4be6dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01a9940 con 0x7f4be01083f0 2023-10-06T22:21:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4be6dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01a9ab0 con 0x7f4be011a690 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4be6dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01a9c70 con 0x7f4be010f640 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4bdffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be010f640 0x7f4be0115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4bdffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be010f640 0x7f4be0115ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57462/0 (socket says 172.21.15.57:57462) 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.107+0000 7f4bdffff700 1 -- 172.21.15.57:0/2965087665 learned_addr learned my addr 172.21.15.57:0/2965087665 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.108+0000 7f4bdffff700 1 -- 172.21.15.57:0/2965087665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be011a690 msgr2=0x7f4be01a93d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.108+0000 7f4bdffff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be011a690 0x7f4be01a93d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.108+0000 7f4bdffff700 1 -- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be01083f0 msgr2=0x7f4be01159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.108+0000 7f4bdffff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be01083f0 0x7f4be01159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.108+0000 7f4bdffff700 1 -- 172.21.15.57:0/2965087665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0009520 con 0x7f4be010f640 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.109+0000 7f4bdffff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be010f640 0x7f4be0115ef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4bd401fa60 tx=0x7f4bd401fe20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.149+0000 7f4bddffb700 1 -- 172.21.15.57:0/2965087665 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd401da40 con 0x7f4be010f640 2023-10-06T22:21:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.149+0000 7f4bddffb700 1 -- 172.21.15.57:0/2965087665 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd40044e0 con 0x7f4be010f640 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.149+0000 7f4bddffb700 1 -- 172.21.15.57:0/2965087665 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4005020 con 0x7f4be010f640 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.149+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be01a9f50 con 0x7f4be010f640 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.150+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be01aa3b0 con 0x7f4be010f640 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.174+0000 7f4bfa802b80 1 Processor -- start 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.175+0000 7f4bfa802b80 1 -- start start 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.176+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.176+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.176+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067d890 0x55f33067fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.176+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3305db460 con 0x55f33067d890 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.176+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3305ccc20 con 0x55f3306729e0 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.176+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3305ca540 con 0x55f3306720c0 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.186+0000 7f4be4b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.186+0000 7f4be4b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46838/0 (socket says 172.21.15.57:46838) 2023-10-06T22:21:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.186+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2427290348 learned_addr learned my addr 172.21.15.57:0/2427290348 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.187+0000 7f4be539c700 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.202 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.888+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000476 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-24.xml auth{0=1} v1371 ap=1 s=10235 n(v0 rc2023-10-06T22:17:16.229084+0000 b10235 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a40b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000476:[<0x200000013f1/comment-24.xml v1325>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000476:[<0x200000013f1/comment-24.xml v1371>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.202 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.889+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000459 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-44.xml auth{0=1} v1371 ap=1 s=9032 n(v0 rc2023-10-06T22:17:16.053087+0000 b9032 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a74100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000459:[<0x200000013f1/comment-44.xml v1289>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000459:[<0x200000013f1/comment-44.xml v1371>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.202 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.890+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d9 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-5.jpg auth{0=1} v1254 ap=1 s=33461 n(v0 rc2023-10-06T22:16:41.707587+0000 b33461 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a7a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d9:[<0x200000013f1/picture-5.jpg v1254>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d9:[<0x200000013f1/picture-5.jpg v1254>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.890+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000369 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-50.jpg auth{0=1} v1252 ap=1 s=11545 n(v0 rc2023-10-06T22:16:38.348636+0000 b11545 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a7ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000369:[<0x200000013f1/picture-50.jpg v1252>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000369:[<0x200000013f1/picture-50.jpg v1252>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.891+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000491 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-7.xml auth{0=1} v1365 ap=1 s=9905 n(v0 rc2023-10-06T22:17:16.557079+0000 b9905 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a7c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000491:[<0x200000013f1/comment-7.xml v1350>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000491:[<0x200000013f1/comment-7.xml v1365>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.891+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000471 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-27.xml auth{0=1} v1375 ap=1 s=2946 n(v0 rc2023-10-06T22:17:16.194085+0000 b2946 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a7a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000471:[<0x200000013f1/comment-27.xml v1313>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000471:[<0x200000013f1/comment-27.xml v1375>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.893+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045a [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-43.xml auth{0=1} v1367 ap=1 s=5060 n(v0 rc2023-10-06T22:17:16.075086+0000 b5060 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a88000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000045a:[<0x200000013f1/comment-43.xml v1291>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000045a:[<0x200000013f1/comment-43.xml v1367>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.893+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000489 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-15.xml auth{0=1} v1377 ap=1 s=4300 n(v0 rc2023-10-06T22:17:16.394082+0000 b4300 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a7cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000489:[<0x200000013f1/comment-15.xml v1336>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000489:[<0x200000013f1/comment-15.xml v1377>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.893+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000048f [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-9.xml auth{0=1} v1363 ap=1 s=7001 n(v0 rc2023-10-06T22:17:16.538080+0000 b7001 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a89080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000048f:[<0x200000013f1/comment-9.xml v1342>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000048f:[<0x200000013f1/comment-9.xml v1363>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.897+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000367 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-51.jpg auth{0=1} v1248 ap=1 s=62952 n(v0 rc2023-10-06T22:16:38.249638+0000 b62952 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a8a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000367:[<0x200000013f1/picture-51.jpg v1248>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000367:[<0x200000013f1/picture-51.jpg v1248>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.898+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000458 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-45.xml auth{0=1} v1361 ap=1 s=8548 n(v0 rc2023-10-06T22:17:16.049087+0000 b8548 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a8ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000458:[<0x200000013f1/comment-45.xml v1285>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000458:[<0x200000013f1/comment-45.xml v1361>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.899+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000375 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-46.jpg auth{0=1} v1250 ap=1 s=25612 n(v0 rc2023-10-06T22:16:38.984627+0000 b25612 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a8b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000375:[<0x200000013f1/picture-46.jpg v1250>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000375:[<0x200000013f1/picture-46.jpg v1250>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.203 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.900+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ba [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-23.jpg auth{0=1} v1276 ap=1 s=33844 n(v0 rc2023-10-06T22:16:40.770601+0000 b33844 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414953700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ba:[<0x200000013f1/picture-23.jpg v1276>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003ba:[<0x200000013f1/picture-23.jpg v1276>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.902+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000487 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-17.xml auth{0=1} v1373 ap=1 s=9178 n(v0 rc2023-10-06T22:17:16.353082+0000 b9178 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a88580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000487:[<0x200000013f1/comment-17.xml v1334>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000487:[<0x200000013f1/comment-17.xml v1373>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.904+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000377 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-45.jpg auth{0=1} v1270 ap=1 s=32991 n(v0 rc2023-10-06T22:16:39.072626+0000 b32991 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c41499eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000377:[<0x200000013f1/picture-45.jpg v1270>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000377:[<0x200000013f1/picture-45.jpg v1270>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.904+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000387 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-41.jpg auth{0=1} v1266 ap=1 s=11254 n(v0 rc2023-10-06T22:16:39.376621+0000 b11254 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c41499f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000387:[<0x200000013f1/picture-41.jpg v1266>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000387:[<0x200000013f1/picture-41.jpg v1266>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.904+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000625 [6,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-46.xml auth{0=1} v1383 ap=1 s=2683 n(v0 rc2023-10-06T22:17:58.402470+0000 b2683 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@5} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149a0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000625:[<0x200000013f1/article-46.xml v1373>,<0x20000000207/blog-26 v136483>,<0x10000000206/blogtest_in v1451>,<0x10000000201/src v1819>,<0x10000000006/blogbench-1.0 v1656>,<0x10000000005/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v90>,<0x10000000001/sv_1 v105>,<0x10000000000/_nogroup v92>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000000625:[<0x200000013f1/article-46.xml v1383>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.905+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bc [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-22.jpg auth{0=1} v1274 ap=1 s=34283 n(v0 rc2023-10-06T22:16:40.834600+0000 b34283 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149a7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003bc:[<0x200000013f1/picture-22.jpg v1274>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003bc:[<0x200000013f1/picture-22.jpg v1274>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.910+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d19 [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-16.xml auth{0=1} v1410 ap=1 s=1259 n(v0 rc2023-10-06T22:20:18.608442+0000 b1259 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55c4149a8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001d19:[<0x200000013f1/article-16.xml v1404>,<0x20000000207/blog-26 v157521>,<0x10000000206/blogtest_in v2273>,<0x10000000201/src v1849>,<0x10000000006/blogbench-1.0 v1686>,<0x10000000005/tmp v104>,<0x100000001fb/client.0 v102>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v120>,<0x10000000001/sv_1 v135>,<0x10000000000/_nogroup v122>,<0x1/volumes v119>]//[]","memoryvalue":"(3)0x10000001d19:[<0x200000013f1/article-16.xml v1410>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.911+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cd [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-15.jpg auth{0=1} v1280 ap=1 s=16691 n(v0 rc2023-10-06T22:16:41.645588+0000 b16691 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149a9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003cd:[<0x200000013f1/picture-15.jpg v1280>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003cd:[<0x200000013f1/picture-15.jpg v1280>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.911+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000373 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-47.jpg auth{0=1} v1272 ap=1 s=51056 n(v0 rc2023-10-06T22:16:38.910628+0000 b51056 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149aeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000373:[<0x200000013f1/picture-47.jpg v1272>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000373:[<0x200000013f1/picture-47.jpg v1272>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.912+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037a [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-44.jpg auth{0=1} v1268 ap=1 s=49957 n(v0 rc2023-10-06T22:16:39.111625+0000 b49957 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149afb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037a:[<0x200000013f1/picture-44.jpg v1268>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000037a:[<0x200000013f1/picture-44.jpg v1268>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.204 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.912+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d1 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-12.jpg auth{0=1} v1278 ap=1 s=43624 n(v0 rc2023-10-06T22:16:41.674588+0000 b43624 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149af600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d1:[<0x200000013f1/picture-12.jpg v1278>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d1:[<0x200000013f1/picture-12.jpg v1278>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.913+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000624 [6,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-59.jpg auth{0=1} v1385 ap=1 s=59362 n(v0 rc2023-10-06T22:17:58.377471+0000 b59362 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149b6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000624:[<0x200000013f1/picture-59.jpg v1368>,<0x20000000207/blog-26 v136483>,<0x10000000206/blogtest_in v1451>,<0x10000000201/src v1819>,<0x10000000006/blogbench-1.0 v1656>,<0x10000000005/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v90>,<0x10000000001/sv_1 v105>,<0x10000000000/_nogroup v92>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000000624:[<0x200000013f1/picture-59.jpg v1385>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.915+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000048b [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-13.xml auth{0=1} v1368 ap=1 s=10013 n(v0 rc2023-10-06T22:17:16.472081+0000 b10013 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149f5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000048b:[<0x200000013f1/comment-13.xml v1352>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000048b:[<0x200000013f1/comment-13.xml v1368>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.915+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000473 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-26.xml auth{0=1} v1374 ap=1 s=3263 n(v0 rc2023-10-06T22:17:16.211084+0000 b3263 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149fa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000473:[<0x200000013f1/comment-26.xml v1320>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000473:[<0x200000013f1/comment-26.xml v1374>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.919+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000482 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-21.xml auth{0=1} v1376 ap=1 s=6707 n(v0 rc2023-10-06T22:17:16.303083+0000 b6707 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149fa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000482:[<0x200000013f1/comment-21.xml v1330>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000482:[<0x200000013f1/comment-21.xml v1376>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.919+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b3 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-27.jpg auth{0=1} v1256 ap=1 s=32056 n(v0 rc2023-10-06T22:16:40.513605+0000 b32056 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149fb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b3:[<0x200000013f1/picture-27.jpg v1256>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003b3:[<0x200000013f1/picture-27.jpg v1256>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.920+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000469 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-31.xml auth{0=1} v1327 ap=1 s=2331 n(v0 rc2023-10-06T22:17:16.162085+0000 b2331 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149b7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000469:[<0x200000013f1/comment-31.xml v1311>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000469:[<0x200000013f1/comment-31.xml v1327>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.920+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005aa [6,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-73.xml auth{0=1} v1392 ap=1 s=31552 n(v0 rc2023-10-06T22:17:41.863711+0000 b31552 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c4149fc680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005aa:[<0x200000013f1/article-73.xml v1382>,<0x20000000207/blog-26 v135019>,<0x10000000206/blogtest_in v1276>,<0x10000000201/src v1815>,<0x10000000006/blogbench-1.0 v1652>,<0x10000000005/tmp v70>,<0x100000001fb/client.0 v68>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v86>,<0x10000000001/sv_1 v101>,<0x10000000000/_nogroup v88>,<0x1/volumes v85>]//[]","memoryvalue":"(3)0x100000005aa:[<0x200000013f1/article-73.xml v1392>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.921+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000360 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-52.jpg auth{0=1} v1248 ap=1 s=34841 n(v0 rc2023-10-06T22:16:38.085640+0000 b34841 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a12000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000360:[<0x200000013f1/picture-52.jpg v1248>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000360:[<0x200000013f1/picture-52.jpg v1248>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.921+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046b [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-30.xml auth{0=1} v1372 ap=1 s=4216 n(v0 rc2023-10-06T22:17:16.166085+0000 b4216 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a12b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000046b:[<0x200000013f1/comment-30.xml v1314>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000046b:[<0x200000013f1/comment-30.xml v1372>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.205 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.921+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a3 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-34.jpg auth{0=1} v1252 ap=1 s=45039 n(v0 rc2023-10-06T22:16:39.967612+0000 b45039 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a13080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a3:[<0x200000013f1/picture-34.jpg v1252>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003a3:[<0x200000013f1/picture-34.jpg v1252>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.922+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000495 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-3.xml auth{0=1} v1378 ap=1 s=2093 n(v0 rc2023-10-06T22:17:16.631078+0000 b2093 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a14680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000495:[<0x200000013f1/comment-3.xml v1358>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000495:[<0x200000013f1/comment-3.xml v1378>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.923+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000486 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-18.xml auth{0=1} v1366 ap=1 s=5274 n(v0 rc2023-10-06T22:17:16.340083+0000 b5274 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a14c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000486:[<0x200000013f1/comment-18.xml v1342>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000486:[<0x200000013f1/comment-18.xml v1366>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.924+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000363 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-77.xml auth{0=1} v1246 ap=1 s=10045 n(v0 rc2023-10-06T22:16:38.148639+0000 b10045 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a1c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000363:[<0x200000013f1/article-77.xml v1246>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000363:[<0x200000013f1/article-77.xml v1246>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.931+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cb [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-16.jpg auth{0=1} v1254 ap=1 s=57312 n(v0 rc2023-10-06T22:16:41.613589+0000 b57312 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a15700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003cb:[<0x200000013f1/picture-16.jpg v1254>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003cb:[<0x200000013f1/picture-16.jpg v1254>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.932+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035b [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-53.jpg auth{0=1} v1244 ap=1 s=45091 n(v0 rc2023-10-06T22:16:37.897643+0000 b45091 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a13600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000035b:[<0x200000013f1/picture-53.jpg v1244>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000035b:[<0x200000013f1/picture-53.jpg v1244>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.933+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000048c [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-12.xml auth{0=1} v1380 ap=1 s=2867 n(v0 rc2023-10-06T22:17:16.494080+0000 b2867 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a1c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000048c:[<0x200000013f1/comment-12.xml v1356>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000048c:[<0x200000013f1/comment-12.xml v1380>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.933+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045b [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-42.xml auth{0=1} v1370 ap=1 s=4890 n(v0 rc2023-10-06T22:17:16.078086+0000 b4890 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a1d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000045b:[<0x200000013f1/comment-42.xml v1289>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000045b:[<0x200000013f1/comment-42.xml v1370>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.934+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a9 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-32.jpg auth{0=1} v1258 ap=1 s=51544 n(v0 rc2023-10-06T22:16:40.046611+0000 b51544 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a1f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003a9:[<0x200000013f1/picture-32.jpg v1258>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003a9:[<0x200000013f1/picture-32.jpg v1258>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.934+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036f [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-48.jpg auth{0=1} v1250 ap=1 s=59999 n(v0 rc2023-10-06T22:16:38.857629+0000 b59999 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a26000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036f:[<0x200000013f1/picture-48.jpg v1250>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000036f:[<0x200000013f1/picture-48.jpg v1250>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.206 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.934+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000017de [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-55.xml auth{0=1} v1390 ap=1 s=1226 n(v0 rc2023-10-06T22:19:39.058010+0000 b1226 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@7} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a41b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000017de:[<0x200000013f1/article-55.xml v1381>,<0x20000000207/blog-26 v151076>,<0x10000000206/blogtest_in v2065>,<0x10000000201/src v1839>,<0x10000000006/blogbench-1.0 v1676>,<0x10000000005/tmp v94>,<0x100000001fb/client.0 v92>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v110>,<0x10000000001/sv_1 v125>,<0x10000000000/_nogroup v112>,<0x1/volumes v109>]//[]","memoryvalue":"(3)0x100000017de:[<0x200000013f1/article-55.xml v1390>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.207 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.936+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003aa [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-31.jpg auth{0=1} v1257 ap=1 s=46521 n(v0 rc2023-10-06T22:16:40.106610+0000 b46521 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a43180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003aa:[<0x200000013f1/picture-31.jpg v1257>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003aa:[<0x200000013f1/picture-31.jpg v1257>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.207 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.937+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036c [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-49.jpg auth{0=1} v1255 ap=1 s=53026 n(v0 rc2023-10-06T22:16:38.731630+0000 b53026 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a4a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036c:[<0x200000013f1/picture-49.jpg v1255>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000036c:[<0x200000013f1/picture-49.jpg v1255>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.207 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.937+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d5 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-8.jpg auth{0=1} v1261 ap=1 s=18134 n(v0 rc2023-10-06T22:16:41.682588+0000 b18134 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a4ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d5:[<0x200000013f1/picture-8.jpg v1261>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d5:[<0x200000013f1/picture-8.jpg v1261>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.660 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.938+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040b [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-88.jpg auth{0=1} v1275 ap=1 s=28891 n(v0 rc2023-10-06T22:17:00.114319+0000 b28891 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a4b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040b:[<0x200000013f1/picture-88.jpg v1268>,<0x20000000207/blog-26 v69812>,<0x10000000206/blogtest_in v621>,<0x10000000201/src v1807>,<0x10000000006/blogbench-1.0 v1644>,<0x10000000005/tmp v62>,<0x100000001fb/client.0 v60>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v78>,<0x10000000001/sv_1 v93>,<0x10000000000/_nogroup v80>,<0x1/volumes v77>]//[]","memoryvalue":"(3)0x1000000040b:[<0x200000013f1/picture-88.jpg v1275>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.660 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.938+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b5 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-26.jpg auth{0=1} v1259 ap=1 s=52144 n(v0 rc2023-10-06T22:16:40.583604+0000 b52144 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a4b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b5:[<0x200000013f1/picture-26.jpg v1259>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003b5:[<0x200000013f1/picture-26.jpg v1259>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.660 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.938+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036b [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-32.xml auth{0=1} v1253 ap=1 s=13281 n(v0 rc2023-10-06T22:16:38.638632+0000 b13281 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a4bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036b:[<0x200000013f1/article-32.xml v1253>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000036b:[<0x200000013f1/article-32.xml v1253>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.660 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.941+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005c1 [6,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-23.xml auth{0=1} v1397 ap=1 s=5065 n(v0 rc2023-10-06T22:17:51.611569+0000 b5065 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a1e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005c1:[<0x200000013f1/article-23.xml v1387>,<0x20000000207/blog-26 v136483>,<0x10000000206/blogtest_in v1451>,<0x10000000201/src v1819>,<0x10000000006/blogbench-1.0 v1656>,<0x10000000005/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v90>,<0x10000000001/sv_1 v105>,<0x10000000000/_nogroup v92>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x100000005c1:[<0x200000013f1/article-23.xml v1397>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.660 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.941+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b7 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-25.jpg auth{0=1} v1263 ap=1 s=42565 n(v0 rc2023-10-06T22:16:40.646603+0000 b42565 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a42100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003b7:[<0x200000013f1/picture-25.jpg v1263>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003b7:[<0x200000013f1/picture-25.jpg v1263>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.660 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.941+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000494 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-4.xml auth{0=1} v1359 ap=1 s=8589 n(v0 rc2023-10-06T22:17:16.621078+0000 b8589 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a54000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000494:[<0x200000013f1/comment-4.xml v1353>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000494:[<0x200000013f1/comment-4.xml v1359>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.942+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000461 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-36.xml auth{0=1} v1357 ap=1 s=3982 n(v0 rc2023-10-06T22:17:16.122086+0000 b3982 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a4d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000461:[<0x200000013f1/comment-36.xml v1302>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000461:[<0x200000013f1/comment-36.xml v1357>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.943+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000048a [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-14.xml auth{0=1} v1361 ap=1 s=5984 n(v0 rc2023-10-06T22:17:16.447081+0000 b5984 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a55600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000048a:[<0x200000013f1/comment-14.xml v1339>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000048a:[<0x200000013f1/comment-14.xml v1361>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.943+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ae [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-29.jpg auth{0=1} v1265 ap=1 s=43160 n(v0 rc2023-10-06T22:16:40.293608+0000 b43160 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a56680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ae:[<0x200000013f1/picture-29.jpg v1265>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003ae:[<0x200000013f1/picture-29.jpg v1265>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.944+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000485 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-19.xml auth{0=1} v1363 ap=1 s=8823 n(v0 rc2023-10-06T22:17:16.333083+0000 b8823 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a56c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000485:[<0x200000013f1/comment-19.xml v1337>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000485:[<0x200000013f1/comment-19.xml v1363>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.944+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000964 [6,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-72.xml auth{0=1} v1396 ap=1 s=32033 n(v0 rc2023-10-06T22:18:29.892012+0000 b32033 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@5} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a5e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000964:[<0x200000013f1/article-72.xml v1391>,<0x20000000207/blog-26 v142335>,<0x10000000206/blogtest_in v1684>,<0x10000000201/src v1825>,<0x10000000006/blogbench-1.0 v1662>,<0x10000000005/tmp v80>,<0x100000001fb/client.0 v78>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v96>,<0x10000000001/sv_1 v111>,<0x10000000000/_nogroup v98>,<0x1/volumes v95>]//[]","memoryvalue":"(3)0x10000000964:[<0x200000013f1/article-72.xml v1396>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.945+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bd [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-21.jpg auth{0=1} v1243 ap=1 s=30735 n(v0 rc2023-10-06T22:16:40.900599+0000 b30735 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a5eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003bd:[<0x200000013f1/picture-21.jpg v1243>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003bd:[<0x200000013f1/picture-21.jpg v1243>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.946+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000190d [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-75.jpg auth{0=1} v1411 ap=1 s=27247 n(v0 rc2023-10-06T22:19:53.633801+0000 b27247 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@6} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a60100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000190d:[<0x200000013f1/picture-75.jpg v1404>,<0x20000000207/blog-26 v152257>,<0x10000000206/blogtest_in v2180>,<0x10000000201/src v1843>,<0x10000000006/blogbench-1.0 v1680>,<0x10000000005/tmp v98>,<0x100000001fb/client.0 v96>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v114>,<0x10000000001/sv_1 v129>,<0x10000000000/_nogroup v116>,<0x1/volumes v113>]//[]","memoryvalue":"(3)0x1000000190d:[<0x200000013f1/picture-75.jpg v1411>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.946+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040f [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-97.xml auth{0=1} v1298 ap=1 s=5995 n(v0 rc2023-10-06T22:17:00.253317+0000 b5995 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a61180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000040f:[<0x200000013f1/article-97.xml v1287>,<0x20000000207/blog-26 v69812>,<0x10000000206/blogtest_in v621>,<0x10000000201/src v1807>,<0x10000000006/blogbench-1.0 v1644>,<0x10000000005/tmp v62>,<0x100000001fb/client.0 v60>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v78>,<0x10000000001/sv_1 v93>,<0x10000000000/_nogroup v80>,<0x1/volumes v77>]//[]","memoryvalue":"(3)0x1000000040f:[<0x200000013f1/article-97.xml v1298>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.661 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.947+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000497 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-1.xml auth{0=1} v1371 ap=1 s=2246 n(v0 rc2023-10-06T22:17:16.637078+0000 b2246 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a60c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000497:[<0x200000013f1/comment-1.xml v1361>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000497:[<0x200000013f1/comment-1.xml v1371>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.662 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.948+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-30.jpg auth{0=1} v1245 ap=1 s=19967 n(v0 rc2023-10-06T22:16:40.177609+0000 b19967 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a68000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003ac:[<0x200000013f1/picture-30.jpg v1245>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003ac:[<0x200000013f1/picture-30.jpg v1245>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.662 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.948+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045d [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-40.xml auth{0=1} v1369 ap=1 s=6847 n(v0 rc2023-10-06T22:17:16.098086+0000 b6847 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a69080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000045d:[<0x200000013f1/comment-40.xml v1300>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000045d:[<0x200000013f1/comment-40.xml v1369>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.662 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.948+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045e [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-39.xml auth{0=1} v1367 ap=1 s=10101 n(v0 rc2023-10-06T22:17:16.102086+0000 b10101 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a69b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000045e:[<0x200000013f1/comment-39.xml v1302>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000045e:[<0x200000013f1/comment-39.xml v1367>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.662 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.948+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000466 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-33.xml auth{0=1} v1373 ap=1 s=4138 n(v0 rc2023-10-06T22:17:16.137086+0000 b4138 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a69600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000466:[<0x200000013f1/comment-33.xml v1306>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000466:[<0x200000013f1/comment-33.xml v1373>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.662 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.950+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000467 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-32.xml auth{0=1} v1365 ap=1 s=9028 n(v0 rc2023-10-06T22:17:16.151085+0000 b9028 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a72000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000467:[<0x200000013f1/comment-32.xml v1308>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000467:[<0x200000013f1/comment-32.xml v1365>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.951+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000394 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-38.jpg auth{0=1} v1249 ap=1 s=14770 n(v0 rc2023-10-06T22:16:39.573618+0000 b14770 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a72b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000394:[<0x200000013f1/picture-38.jpg v1249>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x10000000394:[<0x200000013f1/picture-38.jpg v1249>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.952+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000048e [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-10.xml auth{0=1} v1352 ap=1 s=8763 n(v0 rc2023-10-06T22:17:16.527080+0000 b8763 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a90000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000048e:[<0x200000013f1/comment-10.xml v1336>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000048e:[<0x200000013f1/comment-10.xml v1352>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.952+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039a [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-36.jpg auth{0=1} v1247 ap=1 s=22292 n(v0 rc2023-10-06T22:16:39.751616+0000 b22292 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a6a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000039a:[<0x200000013f1/picture-36.jpg v1247>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000039a:[<0x200000013f1/picture-36.jpg v1247>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.952+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dc [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-3.jpg auth{0=1} v1256 ap=1 s=48299 n(v0 rc2023-10-06T22:16:41.712587+0000 b48299 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a90580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003dc:[<0x200000013f1/picture-3.jpg v1256>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003dc:[<0x200000013f1/picture-3.jpg v1256>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.952+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000492 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-6.xml auth{0=1} v1358 ap=1 s=3920 n(v0 rc2023-10-06T22:17:16.577079+0000 b3920 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a90b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000492:[<0x200000013f1/comment-6.xml v1348>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000492:[<0x200000013f1/comment-6.xml v1358>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.953+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037c [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-43.jpg auth{0=1} v1239 ap=1 s=21319 n(v0 rc2023-10-06T22:16:39.249623+0000 b21319 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a6a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037c:[<0x200000013f1/picture-43.jpg v1239>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000037c:[<0x200000013f1/picture-43.jpg v1239>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.953+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bf [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-20.jpg auth{0=1} v1241 ap=1 s=40992 n(v0 rc2023-10-06T22:16:41.077596+0000 b40992 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a6b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003bf:[<0x200000013f1/picture-20.jpg v1241>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003bf:[<0x200000013f1/picture-20.jpg v1241>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.672 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.954+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000190e [8,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-5.xml auth{0=1} v1407 ap=1 s=14025 n(v0 rc2023-10-06T22:19:53.657801+0000 b14025 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@6} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a92c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000190e:[<0x200000013f1/article-5.xml v1396>,<0x20000000207/blog-26 v152257>,<0x10000000206/blogtest_in v2180>,<0x10000000201/src v1843>,<0x10000000006/blogbench-1.0 v1680>,<0x10000000005/tmp v98>,<0x100000001fb/client.0 v96>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v114>,<0x10000000001/sv_1 v129>,<0x10000000000/_nogroup v116>,<0x1/volumes v113>]//[]","memoryvalue":"(3)0x1000000190e:[<0x200000013f1/article-5.xml v1407>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.673 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.955+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000470 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-28.xml auth{0=1} v1354 ap=1 s=7483 n(v0 rc2023-10-06T22:17:16.181085+0000 b7483 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a9c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000470:[<0x200000013f1/comment-28.xml v1310>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000470:[<0x200000013f1/comment-28.xml v1354>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.697 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.956+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000463 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-35.xml auth{0=1} v1362 ap=1 s=9942 n(v0 rc2023-10-06T22:17:16.128086+0000 b9942 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a93700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000463:[<0x200000013f1/comment-35.xml v1302>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000463:[<0x200000013f1/comment-35.xml v1362>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.697 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.957+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d2 [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-11.jpg auth{0=1} v1258 ap=1 s=34477 n(v0 rc2023-10-06T22:16:41.676588+0000 b34477 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a91600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000003d2:[<0x200000013f1/picture-11.jpg v1258>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x100000003d2:[<0x200000013f1/picture-11.jpg v1258>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.697 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.958+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045f [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-38.xml auth{0=1} v1356 ap=1 s=8273 n(v0 rc2023-10-06T22:17:16.105086+0000 b8273 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a9f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000045f:[<0x200000013f1/comment-38.xml v1296>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000045f:[<0x200000013f1/comment-38.xml v1356>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.698 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.959+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038b [4,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-40.jpg auth{0=1} v1254 ap=1 s=31753 n(v0 rc2023-10-06T22:16:39.459620+0000 b31753 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@3} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414a9e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000038b:[<0x200000013f1/picture-40.jpg v1254>,<0x20000000207/blog-26 v61065>,<0x10000000206/blogtest_in v516>,<0x10000000201/src v1803>,<0x10000000006/blogbench-1.0 v1640>,<0x10000000005/tmp v58>,<0x100000001fb/client.0 v56>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v74>,<0x10000000001/sv_1 v89>,<0x10000000000/_nogroup v76>,<0x1/volumes v72>]//[]","memoryvalue":"(3)0x1000000038b:[<0x200000013f1/picture-40.jpg v1254>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.698 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:00.960+0000 7f75c850a700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000490 [5,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-8.xml auth{0=1} v1360 ap=1 s=8360 n(v0 rc2023-10-06T22:17:16.541080+0000 b8360 1=1+0) (ifile excl) (iversion lock) cr={15228=0-4194304@4} caps={15228=pAsLsXsFsxcrwb/pFscr@1},l=15228 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55c414aa6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000490:[<0x200000013f1/comment-8.xml v1344>,<0x20000000207/blog-26 v110429>,<0x10000000206/blogtest_in v853>,<0x10000000201/src v1809>,<0x10000000006/blogbench-1.0 v1646>,<0x10000000005/tmp v64>,<0x100000001fb/client.0 v62>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v80>,<0x10000000001/sv_1 v95>,<0x10000000000/_nogroup v82>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x10000000490:[<0x200000013f1/comment-8.xml v1360>,<0x20000000207/blog-26 v160959>,<0x10000000206/blogtest_in v2417>,<0x10000000201/src v1853>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v137>,<0x10000000000/_nogroup v74>,<0x1/volumes v121>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T22:21:01.699 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:01 smithi190 ceph-mon[134891]: pgmap v191: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2023-10-06T22:21:01.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.197+0000 7f4be539c700 1 -- 172.21.15.57:0/2427290348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 msgr2=0x55f33067d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:01.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.197+0000 7f4be539c700 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.197+0000 7f4be539c700 1 -- 172.21.15.57:0/2427290348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067d890 msgr2=0x55f33067fc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:21:01.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.197+0000 7f4be539c700 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067d890 0x55f33067fc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.197+0000 7f4be539c700 1 -- 172.21.15.57:0/2427290348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3303f0de0 con 0x55f3306720c0 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.203+0000 7f4be539c700 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306724a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009510 tx=0x7f4bd8009d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf629b41681341a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.203+0000 7f4bceffd700 1 -- 172.21.15.57:0/2427290348 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8017070 con 0x55f3306720c0 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.203+0000 7f4bceffd700 1 -- 172.21.15.57:0/2427290348 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd800f040 con 0x55f3306720c0 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.203+0000 7f4bceffd700 1 -- 172.21.15.57:0/2427290348 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8005740 con 0x55f3306720c0 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.204+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2427290348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 msgr2=0x55f3306724a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.204+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306724a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009510 tx=0x7f4bd8009d70 comp rx=0 tx=0).stop 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.205+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2427290348 shutdown_connections 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.205+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067d890 0x55f33067fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.205+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306724a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.205+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2427290348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.205+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2427290348 >> 172.21.15.57:0/2427290348 conn(0x55f3305e7260 msgr2=0x55f33066e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.226+0000 7f4bddffb700 1 -- 172.21.15.57:0/2965087665 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4bd401e040 con 0x7f4be010f640 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.232+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2427290348 shutdown_connections 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.232+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2427290348 wait complete. 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.233+0000 7f4bfa802b80 1 -- start start 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.233+0000 7f4bfa802b80 1 Processor -- start 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.237+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306720c0 0x55f33067a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.238+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.238+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f33067b1f0 0x55f330689470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.238+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3305c5e30 con 0x55f3306720c0 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.238+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f330562710 con 0x55f3306729e0 2023-10-06T22:21:01.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.238+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3305623b0 con 0x55f33067b1f0 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.238+0000 7f4bfa802b80 10 client.0 _ll_register_callbacks cb 0x55f3303bf800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.239+0000 7f4bddffb700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4bc80a1800 0x7f4bc80a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.240+0000 7f4bddffb700 1 -- 172.21.15.57:0/2965087665 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4bd40ddbb0 con 0x7f4be010f640 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.241+0000 7f4be4b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306720c0 0x55f33067a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.241+0000 7f4be4b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306720c0 0x55f33067a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59908/0 (socket says 172.21.15.57:59908) 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.241+0000 7f4be4b9b700 1 -- 172.21.15.57:0/516992168 learned_addr learned my addr 172.21.15.57:0/516992168 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.241+0000 7f4bdffff700 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.244+0000 7f4be539c700 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f33067b1f0 0x55f330689470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.244+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4bc80a1800 0x7f4bc80a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.246+0000 7f4be4b9b700 1 -- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f33067b1f0 msgr2=0x55f330689470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:01.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.246+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f33067b1f0 0x55f330689470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.246+0000 7f4be4b9b700 1 -- 172.21.15.57:0/516992168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 msgr2=0x55f33067acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:01.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.246+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.246+0000 7f4be4b9b700 1 -- 172.21.15.57:0/516992168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd8009380 con 0x55f3306720c0 2023-10-06T22:21:01.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.251+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306720c0 0x55f33067a770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0010f50 tx=0x7f4bd000fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be30d325f645f02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.254+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4bc80a1800 0x7f4bc80a3cc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0005000 tx=0x7f4bd0029000 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.278+0000 7f4bc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.278+0000 7f4bc3fff700 1 -- 172.21.15.57:0/516992168 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00052a0 con 0x55f3306720c0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.278+0000 7f4bc3fff700 1 -- 172.21.15.57:0/516992168 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4bd0005400 con 0x55f3306720c0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.278+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f330665f10 con 0x55f3306720c0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.281+0000 7f4bc3fff700 1 -- 172.21.15.57:0/516992168 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bd0003680 con 0x55f3306720c0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.281+0000 7f4bfa802b80 10 client.16584 fetch_fsmap learned FSMap version 43 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.281+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f3303d9c60 con 0x55f3306720c0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.294+0000 7f4bc3fff700 1 -- 172.21.15.57:0/516992168 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f330665f10 con 0x55f3306720c0 2023-10-06T22:21:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.294+0000 7f4bc3fff700 1 -- 172.21.15.57:0/516992168 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 43) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f4bd00396d0 con 0x55f3306720c0 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.296+0000 7f4bfa802b80 10 client.16584 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.296+0000 7f4bfa802b80 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.296+0000 7f4bfa802b80 20 client.16584 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.296+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f33068d8f0 0x55f33068fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.296+0000 7f4bfa802b80 4 client.16584 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.296+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3305bee10 con 0x55f33068d8f0 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.309+0000 7f4bdffff700 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f33068d8f0 0x55f33068fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.313+0000 7f4bdffff700 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f33068d8f0 0x55f33068fd10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7dafebcb5aefc3 server_cookie=da59ed7722863b82 in_seq=0 out_seq=0 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.314+0000 7f4bc3fff700 10 client.16584 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.315+0000 7f4bc3fff700 1 -- 172.21.15.57:0/516992168 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3305bee10 con 0x55f33068d8f0 2023-10-06T22:21:01.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.315+0000 7f4bc3fff700 10 client.16584 handle_command_reply: tid=0 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.315+0000 7f4bfa802b80 1 client.16584 shutdown 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306720c0 msgr2=0x55f33067a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306720c0 0x55f33067a770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0010f50 tx=0x7f4bd000fbc0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 shutdown_connections 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f33068d8f0 0x55f33068fd10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f33067b1f0 0x55f330689470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306720c0 0x55f33067a770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/516992168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f33067acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.400+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 >> 172.21.15.57:0/516992168 conn(0x55f3305e7260 msgr2=0x55f33066dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.401+0000 7f4bc3fff700 0 client.16584 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.410+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 shutdown_connections 2023-10-06T22:21:01.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.410+0000 7f4bfa802b80 1 -- 172.21.15.57:0/516992168 wait complete. 2023-10-06T22:21:01.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.411+0000 7f4bfa802b80 20 client.16584 trim_cache size 0 max 16384 2023-10-06T22:21:01.756 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:01 smithi057 ceph-mon[98492]: pgmap v191: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2023-10-06T22:21:01.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:01 smithi156 ceph-mon[134358]: pgmap v191: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2023-10-06T22:21:01.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.896+0000 7f4bfa802b80 1 Processor -- start 2023-10-06T22:21:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.897+0000 7f4bfa802b80 1 -- start start 2023-10-06T22:21:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.898+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.898+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306729e0 0x55f330684440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.898+0000 7f4be4b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.898+0000 7f4be4b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36734/0 (socket says 172.21.15.57:36734) 2023-10-06T22:21:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.899+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2973645299 learned_addr learned my addr 172.21.15.57:0/2973645299 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.899+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33067b1f0 0x55f3306779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.899+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f330446920 con 0x55f3306729e0 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.899+0000 7f4be539c700 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33067b1f0 0x55f3306779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.900+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33066aa20 con 0x55f33067b1f0 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.900+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33066f090 con 0x55f3306720c0 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.902+0000 7f4bdffff700 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306729e0 0x55f330684440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.912+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2973645299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33067b1f0 msgr2=0x55f3306779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.912+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33067b1f0 0x55f3306779e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.912+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2973645299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306729e0 msgr2=0x55f330684440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.912+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306729e0 0x55f330684440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.912+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2973645299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33067ff50 con 0x55f3306720c0 2023-10-06T22:21:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:01.923+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306892a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0005a80 tx=0x7f4bd0009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57ee551260a27e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:02.094 INFO:tasks.workunit.client.0.smithi057.stdout: 335 735564 402 520930 461 253784 1219 2023-10-06T22:21:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.113+0000 7f4bcf7fe700 1 -- 172.21.15.57:0/2973645299 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd003e070 con 0x55f3306720c0 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.113+0000 7f4bcf7fe700 1 -- 172.21.15.57:0/2973645299 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd0005030 con 0x55f3306720c0 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.113+0000 7f4bcf7fe700 1 -- 172.21.15.57:0/2973645299 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00038e0 con 0x55f3306720c0 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.114+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 msgr2=0x55f3306892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.114+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306892a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0005a80 tx=0x7f4bd0009890 comp rx=0 tx=0).stop 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.114+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 shutdown_connections 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.114+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f33067b1f0 0x55f3306779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.114+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306892a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.114+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2973645299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3306729e0 0x55f330684440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.114+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 >> 172.21.15.57:0/2973645299 conn(0x55f3305e7260 msgr2=0x55f330688b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.115+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 shutdown_connections 2023-10-06T22:21:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.115+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2973645299 wait complete. 2023-10-06T22:21:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.115+0000 7f4bfa802b80 1 -- start start 2023-10-06T22:21:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.116+0000 7f4bfa802b80 1 Processor -- start 2023-10-06T22:21:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.116+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f3306d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4bfa802b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067b1f0 0x55f3306e0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3306e0c70 con 0x55f33067b1f0 2023-10-06T22:21:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3306e0db0 con 0x55f3306729e0 2023-10-06T22:21:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4bfa802b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3306e0f70 con 0x55f3306720c0 2023-10-06T22:21:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4bfa802b80 10 client.0 _ll_register_callbacks cb 0x7f4bd000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4bdffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f3306d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.117+0000 7f4be4b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.118+0000 7f4bdffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f3306d5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46616/0 (socket says 172.21.15.57:46616) 2023-10-06T22:21:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.118+0000 7f4bdffff700 1 -- 172.21.15.57:0/2148354169 learned_addr learned my addr 172.21.15.57:0/2148354169 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.120+0000 7f4be539c700 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067b1f0 0x55f3306e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.121+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2148354169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 msgr2=0x55f3306d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.121+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f3306d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.121+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2148354169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067b1f0 msgr2=0x55f3306e0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.121+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067b1f0 0x55f3306e0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.121+0000 7f4be4b9b700 1 -- 172.21.15.57:0/2148354169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3303d9c60 con 0x55f3306720c0 2023-10-06T22:21:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.126+0000 7f4be4b9b700 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306d58a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b2e0 tx=0x7f4bd0039650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51319834d93a7834 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:02.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.213+0000 7f4bc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:21:02.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.213+0000 7f4bc3fff700 1 -- 172.21.15.57:0/2148354169 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd003e070 con 0x55f3306720c0 2023-10-06T22:21:02.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.213+0000 7f4bc3fff700 1 -- 172.21.15.57:0/2148354169 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8448+0+0 (secure 0 0 0) 0x7f4bd0000e40 con 0x55f3306720c0 2023-10-06T22:21:02.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.213+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bd800f040 con 0x55f3306720c0 2023-10-06T22:21:02.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.231+0000 7f4bc3fff700 1 -- 172.21.15.57:0/2148354169 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bd0039c50 con 0x55f3306720c0 2023-10-06T22:21:02.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.231+0000 7f4bfa802b80 10 client.26629 fetch_fsmap learned FSMap version 43 2023-10-06T22:21:02.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.231+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f33068a320 con 0x55f3306720c0 2023-10-06T22:21:02.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.241+0000 7f4bc3fff700 1 -- 172.21.15.57:0/2148354169 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd800f040 con 0x55f3306720c0 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.241+0000 7f4bc3fff700 1 -- 172.21.15.57:0/2148354169 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f4bd002f930 con 0x55f3306720c0 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.241+0000 7f4bfa802b80 10 client.26629 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.241+0000 7f4bfa802b80 10 client.26629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.241+0000 7f4bfa802b80 20 client.26629 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.242+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f3306e22a0 0x55f3306e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.242+0000 7f4bfa802b80 4 client.26629 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.242+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f3305beb30 con 0x55f3306e22a0 2023-10-06T22:21:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.243+0000 7f4bdffff700 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f3306e22a0 0x55f3306e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.275+0000 7f4bdffff700 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f3306e22a0 0x55f3306e46c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f6876b208f902b server_cookie=5ac63a10ad04dfa0 in_seq=0 out_seq=0 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.276+0000 7f4bc3fff700 10 client.26629 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.276+0000 7f4bc3fff700 1 -- 172.21.15.57:0/2148354169 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f3305beb30 con 0x55f3306e22a0 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.276+0000 7f4bc3fff700 10 client.26629 handle_command_reply: tid=0 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.276+0000 7f4bfa802b80 1 client.26629 shutdown 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 msgr2=0x55f3306d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306d58a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b2e0 tx=0x7f4bd0039650 comp rx=0 tx=0).stop 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 shutdown_connections 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55f3306e22a0 0x55f3306e46c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f33067b1f0 0x55f3306e0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3306720c0 0x55f3306d58a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 --2- 172.21.15.57:0/2148354169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f3306729e0 0x55f3306d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 >> 172.21.15.57:0/2148354169 conn(0x55f3305e7260 msgr2=0x55f330674030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.280+0000 7f4bc3fff700 0 client.26629 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.281+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 shutdown_connections 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.281+0000 7f4bfa802b80 1 -- 172.21.15.57:0/2148354169 wait complete. 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.281+0000 7f4bfa802b80 20 client.26629 trim_cache size 0 max 16384 2023-10-06T22:21:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:21:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:21:02.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.377+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4bc80a1800 msgr2=0x7f4bc80a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:02.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.377+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4bc80a1800 0x7f4bc80a3cc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0005000 tx=0x7f4bd0029000 comp rx=0 tx=0).stop 2023-10-06T22:21:02.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.377+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be010f640 msgr2=0x7f4be0115ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:02.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.377+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be010f640 0x7f4be0115ef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4bd401fa60 tx=0x7f4bd401fe20 comp rx=0 tx=0).stop 2023-10-06T22:21:02.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.378+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 shutdown_connections 2023-10-06T22:21:02.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.378+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4bc80a1800 0x7f4bc80a3cc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.378+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be011a690 0x7f4be01a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.378+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be01083f0 0x7f4be01159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:02.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.378+0000 7f4be6dff700 1 --2- 172.21.15.57:0/2965087665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be010f640 0x7f4be0115ef0 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4bd401fa60 tx=0x7f4bd401fe20 comp rx=0 tx=0).stop 2023-10-06T22:21:02.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.378+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 >> 172.21.15.57:0/2965087665 conn(0x7f4be006d1a0 msgr2=0x7f4be011aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:02.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.382+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 shutdown_connections 2023-10-06T22:21:02.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:02.387+0000 7f4be6dff700 1 -- 172.21.15.57:0/2965087665 wait complete. 2023-10-06T22:21:02.580 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003c4 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-18.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d0 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-13.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003c1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-19.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000046c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-29.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003a0 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-35.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000493 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-5.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003ce (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-14.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003dd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-2.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003db (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-4.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d6 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-7.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000048d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-11.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d4 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-9.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000484 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-20.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-10.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000460 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-37.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000488 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-16.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000d9b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-69.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003de (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-1.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-6.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000475 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-25.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003b1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-28.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000465 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-34.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003c6 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-17.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003b8 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-24.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003a7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-33.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000381 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-42.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000479 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-23.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000045c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-41.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000396 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-37.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000038e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-39.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000480 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-22.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000496 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-2.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000476 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-24.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000459 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-44.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d9 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-5.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000369 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-50.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000491 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-7.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000471 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-27.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000045a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-43.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000489 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-15.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000048f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-9.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000367 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-51.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000458 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-45.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000375 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-46.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003ba (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-23.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000487 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-17.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000377 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-45.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000387 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-41.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000625 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-46.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003bc (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-22.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000001d19 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-16.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003cd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-15.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000373 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-47.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000037a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-44.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-12.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000624 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-59.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000048b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-13.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000473 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-26.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000482 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-21.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003b3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-27.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000469 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-31.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000005aa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-73.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000360 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-52.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000046b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-30.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003a3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-34.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000495 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-3.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000486 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-18.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000363 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-77.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003cb (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-16.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000035b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-53.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000048c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-12.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000045b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-42.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003a9 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-32.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000036f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-48.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000017de (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-55.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.682 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003aa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-31.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000036c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-49.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d5 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-8.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000040b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-88.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003b5 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-26.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000036b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-32.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000005c1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-23.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003b7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-25.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000494 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-4.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000461 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-36.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000048a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-14.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.683 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003ae (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-29.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000485 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-19.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000964 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-72.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003bd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-21.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000190d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-75.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000040f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-97.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000497 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-1.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003ac (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-30.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000045d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-40.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000045e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-39.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000466 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-33.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000467 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-32.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000394 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-38.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000048e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-10.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000039a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-36.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003dc (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-3.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000492 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-6.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000037c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-43.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003bf (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-20.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000190e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-5.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000470 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-28.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000463 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-35.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x100000003d2 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-11.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000045f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-38.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x1000000038b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-40.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: Scrub error on inode 0x10000000490 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-8.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:02 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:02.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003c4 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-18.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d0 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-13.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003c1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-19.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000046c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-29.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003a0 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-35.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000493 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-5.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003ce (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-14.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003dd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-2.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003db (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-4.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d6 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-7.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000048d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-11.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d4 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-9.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000484 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-20.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-10.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000460 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-37.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000488 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-16.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000d9b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-69.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003de (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-1.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-6.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000475 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-25.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003b1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-28.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000465 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-34.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003c6 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-17.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003b8 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-24.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003a7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-33.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000381 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-42.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000479 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-23.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000045c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-41.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000396 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-37.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000038e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-39.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000480 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-22.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000496 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-2.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000476 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-24.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000459 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-44.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d9 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-5.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000369 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-50.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000491 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-7.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000471 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-27.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000045a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-43.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000489 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-15.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000048f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-9.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000367 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-51.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000458 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-45.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000375 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-46.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003ba (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-23.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000487 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-17.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000377 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-45.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000387 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-41.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000625 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-46.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003bc (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-22.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000001d19 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-16.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003cd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-15.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000373 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-47.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000037a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-44.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-12.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000624 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-59.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000048b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-13.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000473 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-26.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000482 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-21.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003b3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-27.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000469 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-31.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000005aa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-73.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000360 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-52.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000046b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-30.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003a3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-34.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000495 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-3.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000486 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-18.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000363 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-77.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003cb (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-16.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000035b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-53.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000048c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-12.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000045b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-42.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003a9 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-32.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000036f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-48.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000017de (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-55.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003aa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-31.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000036c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-49.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d5 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-8.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000040b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-88.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003b5 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-26.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000036b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-32.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000005c1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-23.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003b7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-25.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000494 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-4.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000461 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-36.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000048a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-14.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003ae (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-29.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000485 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-19.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000964 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-72.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003bd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-21.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000190d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-75.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000040f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-97.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000497 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-1.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.762 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003ac (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-30.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000045d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-40.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000045e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-39.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000466 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-33.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000467 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-32.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000394 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-38.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000048e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-10.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000039a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-36.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003dc (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-3.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000492 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-6.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000037c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-43.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003bf (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-20.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.763 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000190e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-5.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.764 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000470 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-28.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.764 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000463 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-35.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.764 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x100000003d2 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-11.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.764 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000045f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-38.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.764 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x1000000038b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-40.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.764 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: Scrub error on inode 0x10000000490 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-8.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.764 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:02 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:02.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003c4 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-18.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d0 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-13.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003c1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-19.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000046c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-29.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003a0 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-35.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000493 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-5.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003ce (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-14.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003dd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-2.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003db (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-4.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d6 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-7.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000048d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-11.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d4 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-9.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000484 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-20.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-10.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000460 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-37.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000488 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-16.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000d9b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-69.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003de (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-1.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-6.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000475 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-25.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003b1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-28.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000465 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-34.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003c6 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-17.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003b8 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-24.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003a7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-33.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000381 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-42.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000479 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-23.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000045c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-41.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000396 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-37.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000038e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-39.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000480 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-22.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000496 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-2.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000476 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-24.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000459 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-44.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.824 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d9 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-5.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000369 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-50.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000491 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-7.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000471 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-27.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000045a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-43.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000489 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-15.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000048f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-9.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000367 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-51.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000458 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-45.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000375 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-46.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003ba (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-23.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000487 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-17.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.825 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000377 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-45.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000387 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-41.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000625 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-46.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003bc (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-22.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000001d19 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-16.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003cd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-15.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000373 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-47.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000037a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-44.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-12.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000624 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-59.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000048b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-13.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000473 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-26.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000482 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-21.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003b3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-27.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000469 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-31.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000005aa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-73.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000360 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-52.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000046b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-30.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003a3 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-34.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000495 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-3.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000486 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-18.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000363 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-77.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003cb (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-16.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000035b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-53.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000048c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-12.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000045b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-42.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003a9 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-32.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000036f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-48.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000017de (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-55.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003aa (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-31.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000036c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-49.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d5 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-8.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000040b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-88.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003b5 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-26.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000036b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-32.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000005c1 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-23.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.828 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003b7 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-25.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000494 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-4.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000461 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-36.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000048a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-14.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003ae (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-29.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000485 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-19.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000964 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-72.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003bd (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-21.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000190d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-75.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000040f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-97.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000497 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-1.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003ac (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-30.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000045d (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-40.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000045e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-39.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000466 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-33.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000467 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-32.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000394 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-38.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000048e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-10.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000039a (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-36.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003dc (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-3.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000492 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-6.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000037c (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-43.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003bf (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-20.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000190e (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-5.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.830 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000470 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-28.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.831 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000463 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-35.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.831 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x100000003d2 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-11.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.831 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000045f (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-38.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.831 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x1000000038b (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-40.jpg) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.831 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: Scrub error on inode 0x10000000490 (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-8.xml) see mds.h log and `damage ls` output for details 2023-10-06T22:21:02.831 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:02 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:03.722 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:03 smithi057 ceph-mon[98492]: pgmap v192: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 686 op/s 2023-10-06T22:21:03.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:03 smithi156 ceph-mon[134358]: pgmap v192: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 686 op/s 2023-10-06T22:21:03.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:03 smithi190 ceph-mon[134891]: pgmap v192: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 686 op/s 2023-10-06T22:21:05.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:05 smithi057 ceph-mon[98492]: pgmap v193: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.7 MiB/s wr, 633 op/s 2023-10-06T22:21:05.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:05 smithi156 ceph-mon[134358]: pgmap v193: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.7 MiB/s wr, 633 op/s 2023-10-06T22:21:05.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:05 smithi190 ceph-mon[134891]: pgmap v193: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.7 MiB/s wr, 633 op/s 2023-10-06T22:21:06.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:06 smithi156 ceph-mon[134358]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T22:21:06.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:06 smithi190 ceph-mon[134891]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T22:21:06.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:06 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a[98468]: 2023-10-06T22:21:06.421+0000 7f57da29d700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:21:06.931 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:06 smithi057 ceph-mon[98492]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T22:21:07.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:07 smithi156 ceph-mon[134358]: pgmap v195: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.5 MiB/s wr, 708 op/s 2023-10-06T22:21:07.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:07 smithi156 ceph-mon[134358]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:21:07.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:07 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:active 2023-10-06T22:21:07.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:07 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:07.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:07 smithi190 ceph-mon[134891]: pgmap v195: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.5 MiB/s wr, 708 op/s 2023-10-06T22:21:07.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:07 smithi190 ceph-mon[134891]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:21:07.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:07 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:active 2023-10-06T22:21:07.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:07 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:07.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:07 smithi057 ceph-mon[98492]: pgmap v195: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.5 MiB/s wr, 708 op/s 2023-10-06T22:21:07.925 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:07 smithi057 ceph-mon[98492]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T22:21:07.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:07 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] up:active 2023-10-06T22:21:07.926 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:07 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:09.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:09 smithi190 ceph-mon[134891]: pgmap v196: 97 pgs: 14 active+clean+snaptrim, 59 active+clean, 24 active+clean+snaptrim_wait; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 503 op/s 2023-10-06T22:21:09.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:09 smithi057 ceph-mon[98492]: pgmap v196: 97 pgs: 14 active+clean+snaptrim, 59 active+clean, 24 active+clean+snaptrim_wait; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 503 op/s 2023-10-06T22:21:10.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:09 smithi156 ceph-mon[134358]: pgmap v196: 97 pgs: 14 active+clean+snaptrim, 59 active+clean, 24 active+clean+snaptrim_wait; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.0 MiB/s wr, 503 op/s 2023-10-06T22:21:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.565+0000 7f3dd279d700 1 -- 172.21.15.57:0/4148985109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc10d8b0 msgr2=0x7f3dcc119e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.565+0000 7f3dd279d700 1 --2- 172.21.15.57:0/4148985109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc10d8b0 0x7f3dcc119e00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009990 tx=0x7f3dbc009ca0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.608+0000 7f3dd279d700 1 -- 172.21.15.57:0/4148985109 shutdown_connections 2023-10-06T22:21:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.608+0000 7f3dd279d700 1 --2- 172.21.15.57:0/4148985109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dcc11a340 0x7f3dcc11c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.608+0000 7f3dd279d700 1 --2- 172.21.15.57:0/4148985109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dcc10cf90 0x7f3dcc10d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.608+0000 7f3dd279d700 1 --2- 172.21.15.57:0/4148985109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc10d8b0 0x7f3dcc119e00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.608+0000 7f3dd279d700 1 -- 172.21.15.57:0/4148985109 >> 172.21.15.57:0/4148985109 conn(0x7f3dcc0721d0 msgr2=0x7f3dcc0725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.642+0000 7f3dd279d700 1 -- 172.21.15.57:0/4148985109 shutdown_connections 2023-10-06T22:21:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.643+0000 7f3dd279d700 1 -- 172.21.15.57:0/4148985109 wait complete. 2023-10-06T22:21:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.644+0000 7f3dd279d700 1 Processor -- start 2023-10-06T22:21:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.645+0000 7f3dd279d700 1 -- start start 2023-10-06T22:21:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.645+0000 7f3dd279d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dcc10cf90 0x7f3dcc1100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.645+0000 7f3dd279d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dcc10d8b0 0x7f3dcc110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.645+0000 7f3dd279d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc11a340 0x7f3dcc1ba160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.645+0000 7f3dd279d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1ba6a0 con 0x7f3dcc10cf90 2023-10-06T22:21:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.645+0000 7f3dd279d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1ba810 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.645+0000 7f3dd279d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1ba9d0 con 0x7f3dcc11a340 2023-10-06T22:21:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.653+0000 7f3dd0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dcc10d8b0 0x7f3dcc110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.653+0000 7f3dd1f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc11a340 0x7f3dcc1ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.654+0000 7f3dd1f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc11a340 0x7f3dcc1ba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36758/0 (socket says 172.21.15.57:36758) 2023-10-06T22:21:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.654+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/2101645400 learned_addr learned my addr 172.21.15.57:0/2101645400 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.679+0000 7f3dd179b700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dcc10cf90 0x7f3dcc1100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.679+0000 7f3dd0f9a700 1 -- 172.21.15.57:0/2101645400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc11a340 msgr2=0x7f3dcc1ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.679+0000 7f3dd0f9a700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc11a340 0x7f3dcc1ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.679+0000 7f3dd0f9a700 1 -- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dcc10cf90 msgr2=0x7f3dcc1100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.679+0000 7f3dd0f9a700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dcc10cf90 0x7f3dcc1100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.679+0000 7f3dd0f9a700 1 -- 172.21.15.57:0/2101645400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0095f0 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.682+0000 7f3dd0f9a700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dcc10d8b0 0x7f3dcc110610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc000c00 tx=0x7f3dbc005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.683+0000 7f3dc27fc700 1 -- 172.21.15.57:0/2101645400 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc01d070 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.683+0000 7f3dc27fc700 1 -- 172.21.15.57:0/2101645400 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dbc0173f0 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.683+0000 7f3dc27fc700 1 -- 172.21.15.57:0/2101645400 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00e640 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.683+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc1bac50 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.683+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc1bb020 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.690+0000 7f3de73f1b80 1 Processor -- start 2023-10-06T22:21:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.691+0000 7f3de73f1b80 1 -- start start 2023-10-06T22:21:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.691+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e8ad60 0x55b677e8b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.691+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8b680 0x55b677e95ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.691+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e96530 0x55b677e98920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.691+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677df4400 con 0x55b677e8ad60 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.691+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677de5bc0 con 0x55b677e96530 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.691+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677de34e0 con 0x55b677e8b680 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.692+0000 7f3dd179b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8b680 0x55b677e95ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.692+0000 7f3dd179b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8b680 0x55b677e95ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36766/0 (socket says 172.21.15.57:36766) 2023-10-06T22:21:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.692+0000 7f3dd179b700 1 -- 172.21.15.57:0/450745311 learned_addr learned my addr 172.21.15.57:0/450745311 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.693+0000 7f3dd179b700 1 -- 172.21.15.57:0/450745311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e96530 msgr2=0x55b677e98920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:21:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.693+0000 7f3dd179b700 1 --2- 172.21.15.57:0/450745311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e96530 0x55b677e98920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.693+0000 7f3dd179b700 1 -- 172.21.15.57:0/450745311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e8ad60 msgr2=0x55b677e8b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.693+0000 7f3dd179b700 1 --2- 172.21.15.57:0/450745311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e8ad60 0x55b677e8b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.693+0000 7f3dd179b700 1 -- 172.21.15.57:0/450745311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b677c09d80 con 0x55b677e8b680 2023-10-06T22:21:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.693+0000 7f3dd179b700 1 --2- 172.21.15.57:0/450745311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8b680 0x55b677e95ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009d10 tx=0x7f3dc80116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f02754518eb4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.739+0000 7f3dc27fc700 1 -- 172.21.15.57:0/2101645400 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3dbc00e7a0 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.742+0000 7f3dc27fc700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3db80a1800 0x7f3db80a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.742+0000 7f3dc27fc700 1 -- 172.21.15.57:0/2101645400 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f3dbc0cf370 con 0x7f3dcc10d8b0 2023-10-06T22:21:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.748+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3db80a1800 0x7f3db80a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.802+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3db80a1800 0x7f3db80a3cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e190 tx=0x7f3dc400b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.902+0000 7f3db77fe700 1 -- 172.21.15.57:0/450745311 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8011ed0 con 0x55b677e8b680 2023-10-06T22:21:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.902+0000 7f3db77fe700 1 -- 172.21.15.57:0/450745311 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dc8009e60 con 0x55b677e8b680 2023-10-06T22:21:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.902+0000 7f3db77fe700 1 -- 172.21.15.57:0/450745311 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc800f900 con 0x55b677e8b680 2023-10-06T22:21:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.903+0000 7f3de73f1b80 1 -- 172.21.15.57:0/450745311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8b680 msgr2=0x55b677e95ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.903+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/450745311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8b680 0x55b677e95ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009d10 tx=0x7f3dc80116c0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.903+0000 7f3de73f1b80 1 -- 172.21.15.57:0/450745311 shutdown_connections 2023-10-06T22:21:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.903+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/450745311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e96530 0x55b677e98920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.903+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/450745311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e8ad60 0x55b677e8b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.903+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/450745311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8b680 0x55b677e95ff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.903+0000 7f3de73f1b80 1 -- 172.21.15.57:0/450745311 >> 172.21.15.57:0/450745311 conn(0x55b677e00200 msgr2=0x55b677e871b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.904+0000 7f3de73f1b80 1 -- 172.21.15.57:0/450745311 shutdown_connections 2023-10-06T22:21:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.904+0000 7f3de73f1b80 1 -- 172.21.15.57:0/450745311 wait complete. 2023-10-06T22:21:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.904+0000 7f3de73f1b80 1 -- start start 2023-10-06T22:21:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.905+0000 7f3de73f1b80 1 Processor -- start 2023-10-06T22:21:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.906+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.906+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e93240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.906+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e93980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.906+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677ddedd0 con 0x55b677e96530 2023-10-06T22:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.906+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677d7b6b0 con 0x55b677e8b680 2023-10-06T22:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.906+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677d7b350 con 0x55b677e8ad60 2023-10-06T22:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.906+0000 7f3de73f1b80 10 client.0 _ll_register_callbacks cb 0x55b677bd87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.907+0000 7f3dd0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e93240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.907+0000 7f3dd179b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.907+0000 7f3dd0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e93240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:47542/0 (socket says 172.21.15.57:47542) 2023-10-06T22:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.907+0000 7f3dd0f9a700 1 -- 172.21.15.57:0/360839199 learned_addr learned my addr 172.21.15.57:0/360839199 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.908+0000 7f3dd179b700 1 -- 172.21.15.57:0/360839199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 msgr2=0x55b677e93240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.908+0000 7f3dd179b700 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e93240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.921+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e93980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.921+0000 7f3dd179b700 1 -- 172.21.15.57:0/360839199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 msgr2=0x55b677e93980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.921+0000 7f3dd179b700 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e93980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.921+0000 7f3dd179b700 1 -- 172.21.15.57:0/360839199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc8009940 con 0x55b677e8ad60 2023-10-06T22:21:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.921+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e93980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:21:11.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:11 smithi057 ceph-mon[98492]: pgmap v197: 97 pgs: 20 active+clean+snaptrim, 41 active+clean, 36 active+clean+snaptrim_wait; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 425 op/s 2023-10-06T22:21:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:11.981+0000 7f3dd179b700 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8011ac0 tx=0x7f3dc80043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30559c00c794502b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:12.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:11 smithi156 ceph-mon[134358]: pgmap v197: 97 pgs: 20 active+clean+snaptrim, 41 active+clean, 36 active+clean+snaptrim_wait; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 425 op/s 2023-10-06T22:21:12.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:11 smithi190 ceph-mon[134891]: pgmap v197: 97 pgs: 20 active+clean+snaptrim, 41 active+clean, 36 active+clean+snaptrim_wait; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 425 op/s 2023-10-06T22:21:12.103 INFO:tasks.workunit.client.0.smithi057.stdout: 344 669490 646 466345 258 226462 442 2023-10-06T22:21:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.107+0000 7f3db4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:21:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.107+0000 7f3db4ff9700 1 -- 172.21.15.57:0/360839199 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8005240 con 0x55b677e8ad60 2023-10-06T22:21:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.108+0000 7f3db4ff9700 1 -- 172.21.15.57:0/360839199 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f3dc80053a0 con 0x55b677e8ad60 2023-10-06T22:21:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.108+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b677e7eeb0 con 0x55b677e8ad60 2023-10-06T22:21:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.111+0000 7f3db4ff9700 1 -- 172.21.15.57:0/360839199 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc8005560 con 0x55b677e8ad60 2023-10-06T22:21:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.111+0000 7f3de73f1b80 10 client.26641 fetch_fsmap learned FSMap version 44 2023-10-06T22:21:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.111+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b677bf2c00 con 0x55b677e8ad60 2023-10-06T22:21:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.114+0000 7f3db4ff9700 1 -- 172.21.15.57:0/360839199 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dc8030030 con 0x55b677e8ad60 2023-10-06T22:21:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.114+0000 7f3db4ff9700 1 -- 172.21.15.57:0/360839199 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 44) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f3dc8022070 con 0x55b677e8ad60 2023-10-06T22:21:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.134+0000 7f3de73f1b80 10 client.26641 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T22:21:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.134+0000 7f3de73f1b80 10 client.26641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.134+0000 7f3de73f1b80 20 client.26641 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.134+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677ea60f0 0x55b677ea8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.134+0000 7f3de73f1b80 4 client.26641 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:21:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.134+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b677dd7db0 con 0x55b677ea60f0 2023-10-06T22:21:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.135+0000 7f3dd0f9a700 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677ea60f0 0x55b677ea8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.142+0000 7f3dd0f9a700 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677ea60f0 0x55b677ea8510 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709d24997728dca1 server_cookie=8937eba0c7bf9630 in_seq=0 out_seq=0 2023-10-06T22:21:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.145+0000 7f3db4ff9700 10 client.26641 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.146+0000 7f3db4ff9700 1 -- 172.21.15.57:0/360839199 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b677dd7db0 con 0x55b677ea60f0 2023-10-06T22:21:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.146+0000 7f3db4ff9700 10 client.26641 handle_command_reply: tid=0 2023-10-06T22:21:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.146+0000 7f3de73f1b80 1 client.26641 shutdown 2023-10-06T22:21:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.147+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 msgr2=0x55b677e95fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.147+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8011ac0 tx=0x7f3dc80043b0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.148+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 shutdown_connections 2023-10-06T22:21:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.148+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677ea60f0 0x55b677ea8510 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.149+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e93980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.149+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95fc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.149+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/360839199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e93240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.149+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 >> 172.21.15.57:0/360839199 conn(0x55b677e00200 msgr2=0x55b677e86d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.149+0000 7f3db4ff9700 0 client.26641 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.149+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 shutdown_connections 2023-10-06T22:21:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.150+0000 7f3de73f1b80 1 -- 172.21.15.57:0/360839199 wait complete. 2023-10-06T22:21:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.150+0000 7f3de73f1b80 20 client.26641 trim_cache size 0 max 16384 2023-10-06T22:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.181+0000 7f3de73f1b80 1 Processor -- start 2023-10-06T22:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.181+0000 7f3de73f1b80 1 -- start start 2023-10-06T22:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.181+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.181+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e8e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e8ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677c5f8c0 con 0x55b677e96530 2023-10-06T22:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677e87ef0 con 0x55b677e8b680 2023-10-06T22:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677e8f0d0 con 0x55b677e8ad60 2023-10-06T22:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3dd1f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3dd179b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3dd1f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e8ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32920/0 (socket says 172.21.15.57:32920) 2023-10-06T22:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.182+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/951262460 learned_addr learned my addr 172.21.15.57:0/951262460 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.183+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/951262460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 msgr2=0x55b677e95b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.183+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/951262460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.183+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/951262460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 msgr2=0x55b677e8e590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T22:21:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.183+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/951262460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e8e590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.183+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/951262460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b677c301f0 con 0x55b677e96530 2023-10-06T22:21:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.184+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/951262460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e8ead0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4004bc0 tx=0x7f3dc4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ce624bfdf6aea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.185+0000 7f3db7fff700 1 -- 172.21.15.57:0/951262460 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400ee00 con 0x55b677e96530 2023-10-06T22:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.185+0000 7f3db7fff700 1 -- 172.21.15.57:0/951262460 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dc400f040 con 0x55b677e96530 2023-10-06T22:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.185+0000 7f3db7fff700 1 -- 172.21.15.57:0/951262460 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc401f3c0 con 0x55b677e96530 2023-10-06T22:21:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.186+0000 7f3de73f1b80 1 -- 172.21.15.57:0/951262460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 msgr2=0x55b677e8ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.186+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/951262460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e8ead0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4004bc0 tx=0x7f3dc4011000 comp rx=0 tx=0).stop 2023-10-06T22:21:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.186+0000 7f3de73f1b80 1 -- 172.21.15.57:0/951262460 shutdown_connections 2023-10-06T22:21:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.186+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/951262460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677e8ead0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.187+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/951262460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e95b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.187+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/951262460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e8e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.187+0000 7f3de73f1b80 1 -- 172.21.15.57:0/951262460 >> 172.21.15.57:0/951262460 conn(0x55b677e00200 msgr2=0x55b677ea2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.189+0000 7f3de73f1b80 1 -- 172.21.15.57:0/951262460 shutdown_connections 2023-10-06T22:21:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.190+0000 7f3de73f1b80 1 -- 172.21.15.57:0/951262460 wait complete. 2023-10-06T22:21:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.191+0000 7f3de73f1b80 1 -- start start 2023-10-06T22:21:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.191+0000 7f3de73f1b80 1 Processor -- start 2023-10-06T22:21:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e9b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e9b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3de73f1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677ef8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677e9bf80 con 0x55b677e96530 2023-10-06T22:21:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677ef9420 con 0x55b677e8b680 2023-10-06T22:21:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3de73f1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b677ef95e0 con 0x55b677e8ad60 2023-10-06T22:21:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3de73f1b80 10 client.0 _ll_register_callbacks cb 0x7f3dc8009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.192+0000 7f3dd179b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e9b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677ef8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677ef8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32930/0 (socket says 172.21.15.57:32930) 2023-10-06T22:21:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/673287538 learned_addr learned my addr 172.21.15.57:0/673287538 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd0f9a700 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e9b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/673287538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 msgr2=0x55b677e9b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e9b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/673287538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 msgr2=0x55b677e9b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e9b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.193+0000 7f3dd1f9c700 1 -- 172.21.15.57:0/673287538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc401a040 con 0x55b677e96530 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.194+0000 7f3dd1f9c700 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677ef8ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009b30 tx=0x7f3dc4020700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42011230209f1c2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.194+0000 7f3db4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.195+0000 7f3db4ff9700 1 -- 172.21.15.57:0/673287538 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4020c30 con 0x55b677e96530 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.195+0000 7f3db4ff9700 1 -- 172.21.15.57:0/673287538 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f3dc4020d90 con 0x55b677e96530 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.195+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dc8009e60 con 0x55b677e96530 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.195+0000 7f3db4ff9700 1 -- 172.21.15.57:0/673287538 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc40333f0 con 0x55b677e96530 2023-10-06T22:21:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.195+0000 7f3de73f1b80 10 client.16602 fetch_fsmap learned FSMap version 44 2023-10-06T22:21:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.195+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b677c08250 con 0x55b677e96530 2023-10-06T22:21:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3db4ff9700 1 -- 172.21.15.57:0/673287538 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dc8009e60 con 0x55b677e96530 2023-10-06T22:21:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3db4ff9700 1 -- 172.21.15.57:0/673287538 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 44) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f3dc4029610 con 0x55b677e96530 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3de73f1b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3de73f1b80 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3de73f1b80 20 client.16602 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677efada0 0x55b677efd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3de73f1b80 4 client.16602 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b677dd7ad0 con 0x55b677efada0 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.196+0000 7f3dd179b700 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677efada0 0x55b677efd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.197+0000 7f3db4ff9700 10 client.16602 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.198+0000 7f3dd179b700 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677efada0 0x55b677efd1c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42b1f37857428b9 server_cookie=bca3c5f8d5a873aa in_seq=0 out_seq=0 2023-10-06T22:21:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.597+0000 7f3db4ff9700 1 -- 172.21.15.57:0/673287538 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b677dd7ad0 con 0x55b677efada0 2023-10-06T22:21:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.597+0000 7f3db4ff9700 10 client.16602 handle_command_reply: tid=0 2023-10-06T22:21:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.597+0000 7f3de73f1b80 1 client.16602 shutdown 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 msgr2=0x55b677ef8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677ef8ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009b30 tx=0x7f3dc4020700 comp rx=0 tx=0).stop 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 shutdown_connections 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55b677efada0 0x55b677efd1c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b677e96530 0x55b677ef8ee0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b677e8ad60 0x55b677e9b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 --2- 172.21.15.57:0/673287538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b677e8b680 0x55b677e9b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 >> 172.21.15.57:0/673287538 conn(0x55b677e00200 msgr2=0x55b677e8e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3db4ff9700 0 client.16602 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.598+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 shutdown_connections 2023-10-06T22:21:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.599+0000 7f3de73f1b80 1 -- 172.21.15.57:0/673287538 wait complete. 2023-10-06T22:21:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.599+0000 7f3de73f1b80 20 client.16602 trim_cache size 0 max 16384 2023-10-06T22:21:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:21:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:21:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:21:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:21:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.602+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3db80a1800 msgr2=0x7f3db80a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.602+0000 7f3dd279d700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3db80a1800 0x7f3db80a3cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e190 tx=0x7f3dc400b040 comp rx=0 tx=0).stop 2023-10-06T22:21:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.602+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dcc10d8b0 msgr2=0x7f3dcc110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.602+0000 7f3dd279d700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dcc10d8b0 0x7f3dcc110610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc000c00 tx=0x7f3dbc005720 comp rx=0 tx=0).stop 2023-10-06T22:21:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.603+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 shutdown_connections 2023-10-06T22:21:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.603+0000 7f3dd279d700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3db80a1800 0x7f3db80a3cc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.603+0000 7f3dd279d700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dcc11a340 0x7f3dcc1ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.603+0000 7f3dd279d700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dcc10cf90 0x7f3dcc1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.603+0000 7f3dd279d700 1 --2- 172.21.15.57:0/2101645400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dcc10d8b0 0x7f3dcc110610 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.603+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 >> 172.21.15.57:0/2101645400 conn(0x7f3dcc0721d0 msgr2=0x7f3dcc06b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.604+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 shutdown_connections 2023-10-06T22:21:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:12.604+0000 7f3dd279d700 1 -- 172.21.15.57:0/2101645400 wait complete. 2023-10-06T22:21:13.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:13 smithi057 ceph-mon[98492]: pgmap v198: 97 pgs: 21 active+clean+snaptrim, 37 active+clean, 39 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 408 op/s 2023-10-06T22:21:14.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:13 smithi156 ceph-mon[134358]: pgmap v198: 97 pgs: 21 active+clean+snaptrim, 37 active+clean, 39 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 408 op/s 2023-10-06T22:21:14.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:13 smithi190 ceph-mon[134891]: pgmap v198: 97 pgs: 21 active+clean+snaptrim, 37 active+clean, 39 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 408 op/s 2023-10-06T22:21:15.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:15 smithi057 ceph-mon[98492]: pgmap v199: 97 pgs: 19 active+clean+snaptrim, 41 active+clean, 37 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 406 op/s 2023-10-06T22:21:16.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:15 smithi156 ceph-mon[134358]: pgmap v199: 97 pgs: 19 active+clean+snaptrim, 41 active+clean, 37 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 406 op/s 2023-10-06T22:21:16.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:15 smithi190 ceph-mon[134891]: pgmap v199: 97 pgs: 19 active+clean+snaptrim, 41 active+clean, 37 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 406 op/s 2023-10-06T22:21:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:17 smithi190 ceph-mon[134891]: pgmap v200: 97 pgs: 13 active+clean+snaptrim, 62 active+clean, 22 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2023-10-06T22:21:17.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:17 smithi057 ceph-mon[98492]: pgmap v200: 97 pgs: 13 active+clean+snaptrim, 62 active+clean, 22 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2023-10-06T22:21:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:17 smithi156 ceph-mon[134358]: pgmap v200: 97 pgs: 13 active+clean+snaptrim, 62 active+clean, 22 active+clean+snaptrim_wait; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2023-10-06T22:21:19.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:19 smithi156 ceph-mon[134358]: pgmap v201: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 562 op/s 2023-10-06T22:21:19.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:19 smithi190 ceph-mon[134891]: pgmap v201: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 562 op/s 2023-10-06T22:21:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:19 smithi057 ceph-mon[98492]: pgmap v201: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 562 op/s 2023-10-06T22:21:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:21 smithi156 ceph-mon[134358]: pgmap v202: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 488 op/s 2023-10-06T22:21:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:21 smithi156 ceph-mon[134358]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T22:21:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:21 smithi190 ceph-mon[134891]: pgmap v202: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 488 op/s 2023-10-06T22:21:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:21 smithi190 ceph-mon[134891]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T22:21:21.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:21 smithi057 ceph-mon[98492]: pgmap v202: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 488 op/s 2023-10-06T22:21:21.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:21 smithi057 ceph-mon[98492]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T22:21:23.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:23 smithi156 ceph-mon[134358]: pgmap v204: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 531 op/s 2023-10-06T22:21:23.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:23 smithi190 ceph-mon[134891]: pgmap v204: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 531 op/s 2023-10-06T22:21:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:23 smithi057 ceph-mon[98492]: pgmap v204: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 531 op/s 2023-10-06T22:21:24.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:24 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:24.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:24 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:24.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:24 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:25.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:25 smithi156 ceph-mon[134358]: pgmap v205: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 518 op/s 2023-10-06T22:21:25.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:25 smithi190 ceph-mon[134891]: pgmap v205: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 518 op/s 2023-10-06T22:21:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:25 smithi057 ceph-mon[98492]: pgmap v205: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 518 op/s 2023-10-06T22:21:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.779+0000 7f675a3ff700 1 -- 172.21.15.57:0/3303543516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754101100 msgr2=0x7f67541014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.779+0000 7f675a3ff700 1 --2- 172.21.15.57:0/3303543516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754101100 0x7f67541014e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6744009990 tx=0x7f6744009ca0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.779+0000 7f675a3ff700 1 -- 172.21.15.57:0/3303543516 shutdown_connections 2023-10-06T22:21:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.779+0000 7f675a3ff700 1 --2- 172.21.15.57:0/3303543516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754106220 0x7f675410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.779+0000 7f675a3ff700 1 --2- 172.21.15.57:0/3303543516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754101100 0x7f67541014e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.779+0000 7f675a3ff700 1 --2- 172.21.15.57:0/3303543516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6754101ab0 0x7f6754105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.780+0000 7f675a3ff700 1 -- 172.21.15.57:0/3303543516 >> 172.21.15.57:0/3303543516 conn(0x7f67540fc9a0 msgr2=0x7f67540fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.780+0000 7f675a3ff700 1 -- 172.21.15.57:0/3303543516 shutdown_connections 2023-10-06T22:21:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.780+0000 7f675a3ff700 1 -- 172.21.15.57:0/3303543516 wait complete. 2023-10-06T22:21:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.781+0000 7f675a3ff700 1 Processor -- start 2023-10-06T22:21:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.781+0000 7f675a3ff700 1 -- start start 2023-10-06T22:21:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f675a3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754101100 0x7f6754196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f675a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754106220 0x7f67541974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f675a3ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f675419bb20 0x7f675419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f675a3ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419c5a0 con 0x7f675419bb20 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f675a3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419c710 con 0x7f6754101100 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f675a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419c8d0 con 0x7f6754106220 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f6753fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754101100 0x7f6754196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f6753fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754101100 0x7f6754196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55056/0 (socket says 172.21.15.57:55056) 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f6753fff700 1 -- 172.21.15.57:0/1653716217 learned_addr learned my addr 172.21.15.57:0/1653716217 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f67537fe700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754106220 0x7f67541974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.782+0000 7f675899c700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f675419bb20 0x7f675419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f6753fff700 1 -- 172.21.15.57:0/1653716217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754106220 msgr2=0x7f67541974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f6753fff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754106220 0x7f67541974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f6753fff700 1 -- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f675419bb20 msgr2=0x7f675419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f6753fff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f675419bb20 0x7f675419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f6753fff700 1 -- 172.21.15.57:0/1653716217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440095f0 con 0x7f6754101100 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f6753fff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754101100 0x7f6754196f70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6744004730 tx=0x7f6744004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f67517fa700 1 -- 172.21.15.57:0/1653716217 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674401d070 con 0x7f6754101100 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67541a3dd0 con 0x7f6754101100 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f67517fa700 1 -- 172.21.15.57:0/1653716217 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6744005a00 con 0x7f6754101100 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f67517fa700 1 -- 172.21.15.57:0/1653716217 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744005450 con 0x7f6754101100 2023-10-06T22:21:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.783+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67541a41e0 con 0x7f6754101100 2023-10-06T22:21:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.787+0000 7f67517fa700 1 -- 172.21.15.57:0/1653716217 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f67440047d0 con 0x7f6754101100 2023-10-06T22:21:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.788+0000 7f676dc7cb80 1 Processor -- start 2023-10-06T22:21:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.788+0000 7f676dc7cb80 1 -- start start 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62bdbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62b45400 con 0x55ce62bdc520 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62b36bc0 con 0x55ce62bdbc00 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62b344e0 con 0x55ce62be73d0 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f675899c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62bdbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f675899c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62bdbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55072/0 (socket says 172.21.15.57:55072) 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f675899c700 1 -- 172.21.15.57:0/4111067217 learned_addr learned my addr 172.21.15.57:0/4111067217 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f67537fe700 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f6753fff700 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f67517fa700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f673c0a15f0 0x7f673c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f675899c700 1 -- 172.21.15.57:0/4111067217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 msgr2=0x55ce62be97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f675899c700 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f675899c700 1 -- 172.21.15.57:0/4111067217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 msgr2=0x55ce62be6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.789+0000 7f675899c700 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f675899c700 1 -- 172.21.15.57:0/4111067217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce6295ad80 con 0x55ce62bdbc00 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f67537fe700 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f67517fa700 1 -- 172.21.15.57:0/1653716217 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f67440ce240 con 0x7f6754101100 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f675899c700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f673c0a15f0 0x7f673c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f675899c700 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62bdbfe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f674c009440 tx=0x7f674c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4a2a6cad2884ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f673a7fc700 1 -- 172.21.15.57:0/4111067217 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c01b030 con 0x55ce62bdbc00 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f673a7fc700 1 -- 172.21.15.57:0/4111067217 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f674c00f040 con 0x55ce62bdbc00 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.790+0000 7f673a7fc700 1 -- 172.21.15.57:0/4111067217 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00e070 con 0x55ce62bdbc00 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 -- 172.21.15.57:0/4111067217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 msgr2=0x55ce62bdbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f675899c700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f673c0a15f0 0x7f673c0a3ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f674c009550 tx=0x7f674c014ac0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62bdbfe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f674c009440 tx=0x7f674c009920 comp rx=0 tx=0).stop 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 -- 172.21.15.57:0/4111067217 shutdown_connections 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62bdbfe0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/4111067217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 -- 172.21.15.57:0/4111067217 >> 172.21.15.57:0/4111067217 conn(0x55ce62b51200 msgr2=0x55ce62bd8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 -- 172.21.15.57:0/4111067217 shutdown_connections 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.791+0000 7f676dc7cb80 1 -- 172.21.15.57:0/4111067217 wait complete. 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.792+0000 7f676dc7cb80 1 -- start start 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.792+0000 7f676dc7cb80 1 Processor -- start 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62be4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62b2fdd0 con 0x55ce62bdc520 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62acc6b0 con 0x55ce62bdbc00 2023-10-06T22:21:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62acc350 con 0x55ce62be73d0 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f676dc7cb80 10 client.0 _ll_register_callbacks cb 0x55ce629297a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62be4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62be4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55078/0 (socket says 172.21.15.57:55078) 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 -- 172.21.15.57:0/46989101 learned_addr learned my addr 172.21.15.57:0/46989101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f675899c700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f67537fe700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 -- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 msgr2=0x55ce62be4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 -- 172.21.15.57:0/46989101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 msgr2=0x55ce62be4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 -- 172.21.15.57:0/46989101 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67480098e0 con 0x55ce62bdbc00 2023-10-06T22:21:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f67537fe700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f6753fff700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62be4130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f67440a3040 tx=0x7f67440975b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ccae639e6ac11b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.793+0000 7f67337fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f67337fe700 1 -- 172.21.15.57:0/46989101 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674409b6d0 con 0x55ce62bdbc00 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f67337fe700 1 -- 172.21.15.57:0/46989101 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f674409b970 con 0x55ce62bdbc00 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce62bcfd50 con 0x55ce62bdbc00 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f67337fe700 1 -- 172.21.15.57:0/46989101 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674409abd0 con 0x55ce62bdbc00 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f676dc7cb80 10 client.26654 fetch_fsmap learned FSMap version 45 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f674c0090f0 con 0x55ce62bdbc00 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f67337fe700 1 -- 172.21.15.57:0/46989101 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce62bcfd50 con 0x55ce62bdbc00 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f67337fe700 1 -- 172.21.15.57:0/46989101 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 45) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f674409a760 con 0x55ce62bdbc00 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f676dc7cb80 10 client.26654 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T22:21:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f676dc7cb80 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.794+0000 7f676dc7cb80 20 client.26654 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.795+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62bf6ba0 0x55ce62bf8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.795+0000 7f676dc7cb80 4 client.26654 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:21:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.795+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce62b28db0 con 0x55ce62bf6ba0 2023-10-06T22:21:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.795+0000 7f67537fe700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62bf6ba0 0x55ce62bf8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.795+0000 7f67537fe700 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62bf6ba0 0x55ce62bf8fc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c00c59cbd5a3dd3 server_cookie=a63d54cf70236a31 in_seq=0 out_seq=0 2023-10-06T22:21:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.795+0000 7f67337fe700 10 client.26654 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.796+0000 7f67337fe700 1 -- 172.21.15.57:0/46989101 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ce62b28db0 con 0x55ce62bf6ba0 2023-10-06T22:21:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.796+0000 7f67337fe700 10 client.26654 handle_command_reply: tid=0 2023-10-06T22:21:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.796+0000 7f676dc7cb80 1 client.26654 shutdown 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 msgr2=0x55ce62be4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62be4130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f67440a3040 tx=0x7f67440975b0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 shutdown_connections 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62bf6ba0 0x55ce62bf8fc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdbc00 0x55ce62be4130 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f67337fe700 0 client.26654 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/46989101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdc520 0x55ce62be4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 >> 172.21.15.57:0/46989101 conn(0x55ce62b51200 msgr2=0x55ce62bd7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 shutdown_connections 2023-10-06T22:21:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 1 -- 172.21.15.57:0/46989101 wait complete. 2023-10-06T22:21:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.797+0000 7f676dc7cb80 20 client.26654 trim_cache size 0 max 16384 2023-10-06T22:21:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.812+0000 7f676dc7cb80 1 Processor -- start 2023-10-06T22:21:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f676dc7cb80 1 -- start start 2023-10-06T22:21:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62be13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62be18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce629b08c0 con 0x55ce62bdbc00 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62be5590 con 0x55ce62bdc520 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62bd8dc0 con 0x55ce62be73d0 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f67537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62be18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f67537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62be18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55080/0 (socket says 172.21.15.57:55080) 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f675899c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f67537fe700 1 -- 172.21.15.57:0/48819501 learned_addr learned my addr 172.21.15.57:0/48819501 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f6753fff700 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62be13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f67537fe700 1 -- 172.21.15.57:0/48819501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 msgr2=0x55ce62be2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.813+0000 7f67537fe700 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.814+0000 7f67537fe700 1 -- 172.21.15.57:0/48819501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 msgr2=0x55ce62be13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.814+0000 7f67537fe700 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62be13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.814+0000 7f67537fe700 1 -- 172.21.15.57:0/48819501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce62be9a30 con 0x55ce62bdc520 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.814+0000 7f67537fe700 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62be18e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55ce62be5770 tx=0x7f6748009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09297922df2dc39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.814+0000 7f673affd700 1 -- 172.21.15.57:0/48819501 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748011b80 con 0x55ce62bdc520 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.814+0000 7f673affd700 1 -- 172.21.15.57:0/48819501 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6748018040 con 0x55ce62bdc520 2023-10-06T22:21:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.814+0000 7f673affd700 1 -- 172.21.15.57:0/48819501 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748004830 con 0x55ce62bdc520 2023-10-06T22:21:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 -- 172.21.15.57:0/48819501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 msgr2=0x55ce62be18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62be18e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55ce62be5770 tx=0x7f6748009260 comp rx=0 tx=0).stop 2023-10-06T22:21:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 -- 172.21.15.57:0/48819501 shutdown_connections 2023-10-06T22:21:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62be2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62be13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/48819501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62be18e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 -- 172.21.15.57:0/48819501 >> 172.21.15.57:0/48819501 conn(0x55ce62b51200 msgr2=0x55ce62bf31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 -- 172.21.15.57:0/48819501 shutdown_connections 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 -- 172.21.15.57:0/48819501 wait complete. 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.815+0000 7f676dc7cb80 1 -- start start 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 1 Processor -- start 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62c47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62c48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62c469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62c46f00 con 0x55ce62bdbc00 2023-10-06T22:21:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62c47070 con 0x55ce62bdc520 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce62c47230 con 0x55ce62be73d0 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f676dc7cb80 10 client.0 _ll_register_callbacks cb 0x7f67440d19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f6753fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62c47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f6753fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62c47c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38906/0 (socket says 172.21.15.57:38906) 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f6753fff700 1 -- 172.21.15.57:0/2633786755 learned_addr learned my addr 172.21.15.57:0/2633786755 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f675899c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62c469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.816+0000 7f67537fe700 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62c48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f6753fff700 1 -- 172.21.15.57:0/2633786755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 msgr2=0x55ce62c469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f6753fff700 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62c469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f6753fff700 1 -- 172.21.15.57:0/2633786755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 msgr2=0x55ce62c48170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f6753fff700 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62c48170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f6753fff700 1 -- 172.21.15.57:0/2633786755 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674801f9f0 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f6753fff700 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62c47c30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f674400f740 tx=0x7f674409a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd975a11e0f0b06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f67337fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f67337fe700 1 -- 172.21.15.57:0/2633786755 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744095e80 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f674c00f040 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f67337fe700 1 -- 172.21.15.57:0/2633786755 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f674409b270 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f67337fe700 1 -- 172.21.15.57:0/2633786755 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674409abd0 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f676dc7cb80 10 client.16620 fetch_fsmap learned FSMap version 45 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.817+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ce62943c00 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f67337fe700 1 -- 172.21.15.57:0/2633786755 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f674c00f040 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f67337fe700 1 -- 172.21.15.57:0/2633786755 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f67440a1180 con 0x55ce62bdbc00 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f676dc7cb80 10 client.16620 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T22:21:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f676dc7cb80 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f676dc7cb80 20 client.16620 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62c4f780 0x55ce62c51b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f676dc7cb80 4 client.16620 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:21:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ce62b28ad0 con 0x55ce62c4f780 2023-10-06T22:21:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.818+0000 7f67537fe700 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62c4f780 0x55ce62c51b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.819+0000 7f67537fe700 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62c4f780 0x55ce62c51b70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835165229c4a35b7 server_cookie=8fc80d0dee844f13 in_seq=0 out_seq=0 2023-10-06T22:21:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:25.819+0000 7f67337fe700 10 client.16620 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f67337fe700 1 -- 172.21.15.57:0/2633786755 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ce62b28ad0 con 0x55ce62c4f780 2023-10-06T22:21:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f67337fe700 10 client.16620 handle_command_reply: tid=0 2023-10-06T22:21:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 client.16620 shutdown 2023-10-06T22:21:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 msgr2=0x55ce62c47c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62c47c30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f674400f740 tx=0x7f674409a990 comp rx=0 tx=0).stop 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 shutdown_connections 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55ce62c4f780 0x55ce62c51b70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce62be73d0 0x55ce62c469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ce62bdbc00 0x55ce62c47c30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 --2- 172.21.15.57:0/2633786755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce62bdc520 0x55ce62c48170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 >> 172.21.15.57:0/2633786755 conn(0x55ce62b51200 msgr2=0x55ce62be40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.023+0000 7f67337fe700 0 client.16620 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.024+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 shutdown_connections 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.024+0000 7f676dc7cb80 1 -- 172.21.15.57:0/2633786755 wait complete. 2023-10-06T22:21:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.024+0000 7f676dc7cb80 20 client.16620 trim_cache size 0 max 16384 2023-10-06T22:21:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:21:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:21:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:21:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:21:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f673c0a15f0 msgr2=0x7f673c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f673c0a15f0 0x7f673c0a3ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f674c009550 tx=0x7f674c014ac0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754101100 msgr2=0x7f6754196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754101100 0x7f6754196f70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6744004730 tx=0x7f6744004760 comp rx=0 tx=0).stop 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 shutdown_connections 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f673c0a15f0 0x7f673c0a3ab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f675419bb20 0x7f675419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6754101100 0x7f6754196f70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 --2- 172.21.15.57:0/1653716217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6754106220 0x7f67541974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 >> 172.21.15.57:0/1653716217 conn(0x7f67540fc9a0 msgr2=0x7f6754104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 shutdown_connections 2023-10-06T22:21:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:26.025+0000 7f675a3ff700 1 -- 172.21.15.57:0/1653716217 wait complete. 2023-10-06T22:21:27.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:27 smithi190 ceph-mon[134891]: pgmap v206: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 5.0 MiB/s wr, 444 op/s 2023-10-06T22:21:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:27 smithi057 ceph-mon[98492]: pgmap v206: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 5.0 MiB/s wr, 444 op/s 2023-10-06T22:21:27.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:27 smithi156 ceph-mon[134358]: pgmap v206: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 5.0 MiB/s wr, 444 op/s 2023-10-06T22:21:29.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:29 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:29.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:29 smithi057 ceph-mon[98492]: pgmap v207: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 345 op/s 2023-10-06T22:21:29.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:29 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:29.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:29 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:29.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:29 smithi156 ceph-mon[134358]: pgmap v207: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 345 op/s 2023-10-06T22:21:29.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:29 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:29 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:29 smithi190 ceph-mon[134891]: pgmap v207: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 345 op/s 2023-10-06T22:21:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:29 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:31.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:31 smithi156 ceph-mon[134358]: pgmap v208: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 351 op/s 2023-10-06T22:21:31.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:31 smithi190 ceph-mon[134891]: pgmap v208: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 351 op/s 2023-10-06T22:21:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:31 smithi057 ceph-mon[98492]: pgmap v208: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 351 op/s 2023-10-06T22:21:32.584 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:21:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.799+0000 7f63685ff700 1 -- 172.21.15.57:0/3113394455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601061d0 msgr2=0x7f636010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.799+0000 7f63685ff700 1 --2- 172.21.15.57:0/3113394455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601061d0 0x7f636010dd50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f635800b0a0 tx=0x7f635800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.800+0000 7f63685ff700 1 -- 172.21.15.57:0/3113394455 shutdown_connections 2023-10-06T22:21:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.800+0000 7f63685ff700 1 --2- 172.21.15.57:0/3113394455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601061d0 0x7f636010dd50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.800+0000 7f63685ff700 1 --2- 172.21.15.57:0/3113394455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63601010b0 0x7f6360101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.800+0000 7f63685ff700 1 --2- 172.21.15.57:0/3113394455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6360101a60 0x7f6360105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.800+0000 7f63685ff700 1 -- 172.21.15.57:0/3113394455 >> 172.21.15.57:0/3113394455 conn(0x7f63600fc950 msgr2=0x7f63600fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:32.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.800+0000 7f63685ff700 1 -- 172.21.15.57:0/3113394455 shutdown_connections 2023-10-06T22:21:32.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.800+0000 7f63685ff700 1 -- 172.21.15.57:0/3113394455 wait complete. 2023-10-06T22:21:32.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.801+0000 7f63685ff700 1 Processor -- start 2023-10-06T22:21:32.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.801+0000 7f63685ff700 1 -- start start 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f63685ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63601010b0 0x7f63601970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f63685ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63601061d0 0x7f6360197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f63685ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636019ba60 0x7f636019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f63685ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636019c4e0 con 0x7f63601061d0 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f63685ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636019c650 con 0x7f63601010b0 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f63685ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636019c810 con 0x7f636019ba60 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f636639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63601010b0 0x7f63601970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f636639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63601010b0 0x7f63601970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55154/0 (socket says 172.21.15.57:55154) 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f636639b700 1 -- 172.21.15.57:0/2198355199 learned_addr learned my addr 172.21.15.57:0/2198355199 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:32.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6366b9c700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636019ba60 0x7f636019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6365b9a700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63601061d0 0x7f6360197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6366b9c700 1 -- 172.21.15.57:0/2198355199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63601010b0 msgr2=0x7f63601970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6366b9c700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63601010b0 0x7f63601970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6366b9c700 1 -- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63601061d0 msgr2=0x7f6360197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6366b9c700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63601061d0 0x7f6360197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6366b9c700 1 -- 172.21.15.57:0/2198355199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358009d00 con 0x7f636019ba60 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.802+0000 7f6365b9a700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63601061d0 0x7f6360197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.803+0000 7f636639b700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63601010b0 0x7f63601970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:32.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.803+0000 7f6366b9c700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636019ba60 0x7f636019bee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6358009fd0 tx=0x7f6358007910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:32.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.803+0000 7f63577fe700 1 -- 172.21.15.57:0/2198355199 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800e040 con 0x7f636019ba60 2023-10-06T22:21:32.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.803+0000 7f63577fe700 1 -- 172.21.15.57:0/2198355199 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635800bde0 con 0x7f636019ba60 2023-10-06T22:21:32.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.803+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63601a3d10 con 0x7f636019ba60 2023-10-06T22:21:32.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.803+0000 7f63577fe700 1 -- 172.21.15.57:0/2198355199 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63580052a0 con 0x7f636019ba60 2023-10-06T22:21:32.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.803+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63601a4190 con 0x7f636019ba60 2023-10-06T22:21:32.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.806+0000 7f63577fe700 1 -- 172.21.15.57:0/2198355199 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f635801d480 con 0x7f636019ba60 2023-10-06T22:21:32.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 Processor -- start 2023-10-06T22:21:32.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 -- start start 2023-10-06T22:21:32.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f63577fe700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f634c0a1670 0x7f634c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0850c0 0x55569d0854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f63577fe700 1 -- 172.21.15.57:0/2198355199 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f63580cdf40 con 0x7f636019ba60 2023-10-06T22:21:32.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0859e0 0x55569d090350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d092c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569cfee460 con 0x55569d0850c0 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569cfdfc20 con 0x55569d090890 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569cfdd540 con 0x55569d0859e0 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.808+0000 7f6365b9a700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f634c0a1670 0x7f634c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d092c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d092c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55168/0 (socket says 172.21.15.57:55168) 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 -- 172.21.15.57:0/4141993832 learned_addr learned my addr 172.21.15.57:0/4141993832 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f636639b700 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0850c0 0x55569d0854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6365b9a700 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0859e0 0x55569d090350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 -- 172.21.15.57:0/4141993832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0859e0 msgr2=0x55569d090350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0859e0 0x55569d090350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 -- 172.21.15.57:0/4141993832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0850c0 msgr2=0x55569d0854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0850c0 0x55569d0854a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 -- 172.21.15.57:0/4141993832 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569ce03de0 con 0x55569d090890 2023-10-06T22:21:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f636639b700 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0850c0 0x55569d0854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:21:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6366b9c700 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d092c80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6358097470 tx=0x7f635800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60ed62217e109f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6365b9a700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f634c0a1670 0x7f634c0a3b30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63601986d0 tx=0x7f6350014940 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.809+0000 7f6354bfe700 1 -- 172.21.15.57:0/4141993832 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635809b670 con 0x55569d090890 2023-10-06T22:21:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f6354bfe700 1 -- 172.21.15.57:0/4141993832 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635809af70 con 0x55569d090890 2023-10-06T22:21:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f6354bfe700 1 -- 172.21.15.57:0/4141993832 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63580993f0 con 0x55569d090890 2023-10-06T22:21:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 -- 172.21.15.57:0/4141993832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 msgr2=0x55569d092c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d092c80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6358097470 tx=0x7f635800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 -- 172.21.15.57:0/4141993832 shutdown_connections 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d092c80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0850c0 0x55569d0854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/4141993832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0859e0 0x55569d090350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 -- 172.21.15.57:0/4141993832 >> 172.21.15.57:0/4141993832 conn(0x55569cffa260 msgr2=0x55569d081450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 -- 172.21.15.57:0/4141993832 shutdown_connections 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.810+0000 7f637bfd5b80 1 -- 172.21.15.57:0/4141993832 wait complete. 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.811+0000 7f637bfd5b80 1 -- start start 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.811+0000 7f637bfd5b80 1 Processor -- start 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.811+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d090220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.811+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0859e0 0x55569d08d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d090890 0x55569d08da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569cfd8e30 con 0x55569d090890 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569cf75710 con 0x55569d0859e0 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569cf753b0 con 0x55569d0850c0 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f637bfd5b80 10 client.0 _ll_register_callbacks cb 0x55569cdd2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d090220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d090220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48754/0 (socket says 172.21.15.57:48754) 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 -- 172.21.15.57:0/1484736613 learned_addr learned my addr 172.21.15.57:0/1484736613 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f6366b9c700 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d090890 0x55569d08da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f6365b9a700 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0859e0 0x55569d08d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 -- 172.21.15.57:0/1484736613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0859e0 msgr2=0x55569d08d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0859e0 0x55569d08d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 -- 172.21.15.57:0/1484736613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d090890 msgr2=0x55569d08da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d090890 0x55569d08da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 -- 172.21.15.57:0/1484736613 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63601a4190 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f636639b700 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d090220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f635c012d20 tx=0x7f635c00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8ff534a0187854 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.812+0000 7f6349ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f6349ffb700 1 -- 172.21.15.57:0/1484736613 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c017070 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55569d078f10 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f6349ffb700 1 -- 172.21.15.57:0/1484736613 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f635c00fc00 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f6349ffb700 1 -- 172.21.15.57:0/1484736613 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635c0122e0 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f637bfd5b80 10 client.26677 fetch_fsmap learned FSMap version 46 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55569cdecc60 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f6349ffb700 1 -- 172.21.15.57:0/1484736613 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55569d078f10 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f6349ffb700 1 -- 172.21.15.57:0/1484736613 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 46) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f635c01c420 con 0x55569d0850c0 2023-10-06T22:21:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f637bfd5b80 10 client.26677 fetch_fsmap finished waiting for FSMap version 46 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.813+0000 7f637bfd5b80 10 client.26677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.814+0000 7f637bfd5b80 20 client.26677 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.814+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0a01f0 0x55569d0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.814+0000 7f637bfd5b80 4 client.26677 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.814+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55569cfd1e10 con 0x55569d0a01f0 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.814+0000 7f6365b9a700 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0a01f0 0x55569d0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.814+0000 7f6365b9a700 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0a01f0 0x55569d0a2610 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1394b97043e281c2 server_cookie=1d81b1cde57a2f77 in_seq=0 out_seq=0 2023-10-06T22:21:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.814+0000 7f6349ffb700 10 client.26677 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f6349ffb700 1 -- 172.21.15.57:0/1484736613 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55569cfd1e10 con 0x55569d0a01f0 2023-10-06T22:21:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f6349ffb700 10 client.26677 handle_command_reply: tid=0 2023-10-06T22:21:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 client.26677 shutdown 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 msgr2=0x55569d090220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d090220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f635c012d20 tx=0x7f635c00f6a0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 shutdown_connections 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0a01f0 0x55569d0a2610 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d090890 0x55569d08da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d090220 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f6349ffb700 0 client.26677 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/1484736613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0859e0 0x55569d08d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 >> 172.21.15.57:0/1484736613 conn(0x55569cffa260 msgr2=0x55569d081040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 shutdown_connections 2023-10-06T22:21:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.815+0000 7f637bfd5b80 1 -- 172.21.15.57:0/1484736613 wait complete. 2023-10-06T22:21:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.816+0000 7f637bfd5b80 20 client.26677 trim_cache size 0 max 16384 2023-10-06T22:21:32.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.835+0000 7f637bfd5b80 1 Processor -- start 2023-10-06T22:21:32.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.835+0000 7f637bfd5b80 1 -- start start 2023-10-06T22:21:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d097270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d090220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569ce59920 con 0x55569d0859e0 2023-10-06T22:21:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d097820 con 0x55569d090890 2023-10-06T22:21:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d082d70 con 0x55569d0850c0 2023-10-06T22:21:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f636639b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d097270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f636639b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d097270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48762/0 (socket says 172.21.15.57:48762) 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f636639b700 1 -- 172.21.15.57:0/3379734231 learned_addr learned my addr 172.21.15.57:0/3379734231 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f6365b9a700 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.836+0000 7f6366b9c700 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d090220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f636639b700 1 -- 172.21.15.57:0/3379734231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 msgr2=0x55569d090220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f636639b700 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d090220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f636639b700 1 -- 172.21.15.57:0/3379734231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 msgr2=0x55569d08fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f636639b700 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f636639b700 1 -- 172.21.15.57:0/3379734231 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569ce2a250 con 0x55569d0850c0 2023-10-06T22:21:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f636639b700 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d097270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f635c006010 tx=0x7f635c009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c7d2532e64a080 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f63553ff700 1 -- 172.21.15.57:0/3379734231 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c017040 con 0x55569d0850c0 2023-10-06T22:21:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f63553ff700 1 -- 172.21.15.57:0/3379734231 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635c00f9f0 con 0x55569d0850c0 2023-10-06T22:21:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.837+0000 7f63553ff700 1 -- 172.21.15.57:0/3379734231 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c005370 con 0x55569d0850c0 2023-10-06T22:21:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 -- 172.21.15.57:0/3379734231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 msgr2=0x55569d097270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d097270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f635c006010 tx=0x7f635c009640 comp rx=0 tx=0).stop 2023-10-06T22:21:32.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 -- 172.21.15.57:0/3379734231 shutdown_connections 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d090890 0x55569d090220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d0850c0 0x55569d097270 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/3379734231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 -- 172.21.15.57:0/3379734231 >> 172.21.15.57:0/3379734231 conn(0x55569cffa260 msgr2=0x55569d088850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 -- 172.21.15.57:0/3379734231 shutdown_connections 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.838+0000 7f637bfd5b80 1 -- 172.21.15.57:0/3379734231 wait complete. 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 -- start start 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 Processor -- start 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0850c0 0x55569d08cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d090890 0x55569d08ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d08c140 con 0x55569d0859e0 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d08c2b0 con 0x55569d0850c0 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d08c470 con 0x55569d090890 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.839+0000 7f637bfd5b80 10 client.0 _ll_register_callbacks cb 0x7f635c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0850c0 0x55569d08cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0850c0 0x55569d08cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55202/0 (socket says 172.21.15.57:55202) 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 -- 172.21.15.57:0/79730086 learned_addr learned my addr 172.21.15.57:0/79730086 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f6365b9a700 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f6366b9c700 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d090890 0x55569d08ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 -- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d090890 msgr2=0x55569d08ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d090890 0x55569d08ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 -- 172.21.15.57:0/79730086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 msgr2=0x55569d08d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 -- 172.21.15.57:0/79730086 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55569cdecc60 con 0x55569d0850c0 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f636639b700 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0850c0 0x55569d08cee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f635c00f6e0 tx=0x7f635c009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a156d9411449c279 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.840+0000 7f6349ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f6349ffb700 1 -- 172.21.15.57:0/79730086 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c01cc00 con 0x55569d0850c0 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f635809af70 con 0x55569d0850c0 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f6349ffb700 1 -- 172.21.15.57:0/79730086 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f635c01cd60 con 0x55569d0850c0 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f6349ffb700 1 -- 172.21.15.57:0/79730086 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635c005110 con 0x55569d0850c0 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f637bfd5b80 10 client.26678 fetch_fsmap learned FSMap version 46 2023-10-06T22:21:32.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55569ce022b0 con 0x55569d0850c0 2023-10-06T22:21:32.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f6349ffb700 1 -- 172.21.15.57:0/79730086 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635809af70 con 0x55569d0850c0 2023-10-06T22:21:32.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f6349ffb700 1 -- 172.21.15.57:0/79730086 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 46) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f635c003c50 con 0x55569d0850c0 2023-10-06T22:21:32.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f637bfd5b80 10 client.26678 fetch_fsmap finished waiting for FSMap version 46 2023-10-06T22:21:32.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f637bfd5b80 10 client.26678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:32.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f637bfd5b80 20 client.26678 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:32.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.841+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0f9220 0x55569d0fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f637bfd5b80 4 client.26678 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55569cfd1b30 con 0x55569d0f9220 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f6365b9a700 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0f9220 0x55569d0fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f6365b9a700 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0f9220 0x55569d0fb640 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a196f89af922f9 server_cookie=277a6d8f0f748421 in_seq=0 out_seq=0 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f6349ffb700 10 client.26678 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f6349ffb700 1 -- 172.21.15.57:0/79730086 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55569cfd1b30 con 0x55569d0f9220 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f6349ffb700 10 client.26678 handle_command_reply: tid=0 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.842+0000 7f637bfd5b80 1 client.26678 shutdown 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0850c0 msgr2=0x55569d08cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0850c0 0x55569d08cee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f635c00f6e0 tx=0x7f635c009350 comp rx=0 tx=0).stop 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 shutdown_connections 2023-10-06T22:21:32.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55569d0f9220 0x55569d0fb640 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55569d090890 0x55569d08ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55569d0850c0 0x55569d08cee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f6349ffb700 0 client.26678 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 --2- 172.21.15.57:0/79730086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55569d0859e0 0x55569d08d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 >> 172.21.15.57:0/79730086 conn(0x55569cffa260 msgr2=0x55569d087070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 shutdown_connections 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 1 -- 172.21.15.57:0/79730086 wait complete. 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.843+0000 7f637bfd5b80 20 client.26678 trim_cache size 0 max 16384 2023-10-06T22:21:32.850 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:21:32.851 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f634c0a1670 msgr2=0x7f634c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f634c0a1670 0x7f634c0a3b30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63601986d0 tx=0x7f6350014940 comp rx=0 tx=0).stop 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636019ba60 msgr2=0x7f636019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636019ba60 0x7f636019bee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6358009fd0 tx=0x7f6358007910 comp rx=0 tx=0).stop 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 shutdown_connections 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f634c0a1670 0x7f634c0a3b30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636019ba60 0x7f636019bee0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63601010b0 0x7f63601970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 --2- 172.21.15.57:0/2198355199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63601061d0 0x7f6360197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 >> 172.21.15.57:0/2198355199 conn(0x7f63600fc950 msgr2=0x7f63600fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 shutdown_connections 2023-10-06T22:21:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:21:32.844+0000 7f63685ff700 1 -- 172.21.15.57:0/2198355199 wait complete. 2023-10-06T22:21:32.866 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:21:33.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:33 smithi156 ceph-mon[134358]: pgmap v209: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 421 op/s 2023-10-06T22:21:33.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:33 smithi190 ceph-mon[134891]: pgmap v209: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 421 op/s 2023-10-06T22:21:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:33 smithi057 ceph-mon[98492]: pgmap v209: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 421 op/s 2023-10-06T22:21:34.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:34 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:34.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:34 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:34.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:34 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:35.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:35 smithi156 ceph-mon[134358]: pgmap v210: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2023-10-06T22:21:35.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:35 smithi190 ceph-mon[134891]: pgmap v210: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2023-10-06T22:21:35.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:35 smithi057 ceph-mon[98492]: pgmap v210: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2023-10-06T22:21:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:37 smithi190 ceph-mon[134891]: pgmap v211: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 316 op/s 2023-10-06T22:21:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:37 smithi057 ceph-mon[98492]: pgmap v211: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 316 op/s 2023-10-06T22:21:37.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:37 smithi156 ceph-mon[134358]: pgmap v211: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 316 op/s 2023-10-06T22:21:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.274+0000 7f81db5ff700 1 -- 172.21.15.57:0/2998689226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410f620 msgr2=0x7f81d4107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.274+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2998689226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410f620 0x7f81d4107eb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f81d00098f0 tx=0x7f81d0009c00 comp rx=0 tx=0).stop 2023-10-06T22:21:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.275+0000 7f81db5ff700 1 -- 172.21.15.57:0/2998689226 shutdown_connections 2023-10-06T22:21:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.275+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2998689226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d411a690 0x7f81d411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.275+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2998689226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410f620 0x7f81d4107eb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.275+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2998689226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d410fa90 0x7f81d41083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.275+0000 7f81db5ff700 1 -- 172.21.15.57:0/2998689226 >> 172.21.15.57:0/2998689226 conn(0x7f81d406cfa0 msgr2=0x7f81d406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.279+0000 7f81db5ff700 1 -- 172.21.15.57:0/2998689226 shutdown_connections 2023-10-06T22:21:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.284+0000 7f81db5ff700 1 -- 172.21.15.57:0/2998689226 wait complete. 2023-10-06T22:21:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.285+0000 7f81db5ff700 1 Processor -- start 2023-10-06T22:21:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.285+0000 7f81db5ff700 1 -- start start 2023-10-06T22:21:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.288+0000 7f81db5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d410f620 0x7f81d41a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.288+0000 7f81db5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410fa90 0x7f81d41a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.288+0000 7f81db5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d411a690 0x7f81d41aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.288+0000 7f81db5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41ab080 con 0x7f81d410fa90 2023-10-06T22:21:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.288+0000 7f81db5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41ab1f0 con 0x7f81d410f620 2023-10-06T22:21:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.288+0000 7f81db5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41ab3b0 con 0x7f81d411a690 2023-10-06T22:21:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.289+0000 7f81d8b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410fa90 0x7f81d41a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.289+0000 7f81d8b9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410fa90 0x7f81d41a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39596/0 (socket says 172.21.15.57:39596) 2023-10-06T22:21:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.289+0000 7f81d8b9a700 1 -- 172.21.15.57:0/2712903575 learned_addr learned my addr 172.21.15.57:0/2712903575 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.289+0000 7f81d9b9c700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d411a690 0x7f81d41aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.290+0000 7f81d939b700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d410f620 0x7f81d41a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.294+0000 7f81d8b9a700 1 -- 172.21.15.57:0/2712903575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d411a690 msgr2=0x7f81d41aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.294+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d411a690 0x7f81d41aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.294+0000 7f81d8b9a700 1 -- 172.21.15.57:0/2712903575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d410f620 msgr2=0x7f81d41a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.294+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d410f620 0x7f81d41a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.294+0000 7f81d8b9a700 1 -- 172.21.15.57:0/2712903575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0009520 con 0x7f81d410fa90 2023-10-06T22:21:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.295+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410fa90 0x7f81d41a63d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e9a0 tx=0x7f81c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.297+0000 7f81ca7fc700 1 -- 172.21.15.57:0/2712903575 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c400cbd0 con 0x7f81d410fa90 2023-10-06T22:21:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.297+0000 7f81ca7fc700 1 -- 172.21.15.57:0/2712903575 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81c400cd30 con 0x7f81d410fa90 2023-10-06T22:21:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.297+0000 7f81ca7fc700 1 -- 172.21.15.57:0/2712903575 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4018680 con 0x7f81d410fa90 2023-10-06T22:21:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.297+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d41ab690 con 0x7f81d410fa90 2023-10-06T22:21:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.297+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d40792f0 con 0x7f81d410fa90 2023-10-06T22:21:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.306+0000 7f81eefa5b80 1 Processor -- start 2023-10-06T22:21:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.306+0000 7f81eefa5b80 1 -- start start 2023-10-06T22:21:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.307+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be3c00 0x561599be3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.307+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599beee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.307+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bf17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.307+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599b4d400 con 0x561599be3c00 2023-10-06T22:21:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.307+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599b3ebc0 con 0x561599be4520 2023-10-06T22:21:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.307+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599b3c4e0 con 0x561599bef3d0 2023-10-06T22:21:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.312+0000 7f81ca7fc700 1 -- 172.21.15.57:0/2712903575 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f81c40187e0 con 0x7f81d410fa90 2023-10-06T22:21:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.312+0000 7f81d939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599beee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.312+0000 7f81d939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599beee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55228/0 (socket says 172.21.15.57:55228) 2023-10-06T22:21:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.312+0000 7f81d939b700 1 -- 172.21.15.57:0/263819405 learned_addr learned my addr 172.21.15.57:0/263819405 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.314+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bf17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.315+0000 7f81ca7fc700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f81c00a1800 0x7f81c00a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.315+0000 7f81ca7fc700 1 -- 172.21.15.57:0/2712903575 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f81c4014070 con 0x7f81d410fa90 2023-10-06T22:21:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.322+0000 7f81d939b700 1 -- 172.21.15.57:0/263819405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 msgr2=0x561599bf17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.322+0000 7f81d939b700 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bf17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.322+0000 7f81d939b700 1 -- 172.21.15.57:0/263819405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be3c00 msgr2=0x561599be3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.322+0000 7f81d939b700 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be3c00 0x561599be3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.322+0000 7f81d939b700 1 -- 172.21.15.57:0/263819405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561599962d80 con 0x561599be4520 2023-10-06T22:21:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.322+0000 7f81d9b9c700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f81c00a1800 0x7f81c00a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.324+0000 7f81d9b9c700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f81c00a1800 0x7f81c00a3cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f81d406a250 tx=0x7f81cc0099e0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.327+0000 7f81d939b700 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599beee90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81d0016000 tx=0x7f81d0009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e111bb8222177294 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.329+0000 7f81bf7fe700 1 -- 172.21.15.57:0/263819405 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d001d040 con 0x561599be4520 2023-10-06T22:21:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.329+0000 7f81bf7fe700 1 -- 172.21.15.57:0/263819405 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d000f920 con 0x561599be4520 2023-10-06T22:21:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.329+0000 7f81bf7fe700 1 -- 172.21.15.57:0/263819405 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d000b7c0 con 0x561599be4520 2023-10-06T22:21:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.330+0000 7f81eefa5b80 1 -- 172.21.15.57:0/263819405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 msgr2=0x561599beee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.330+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599beee90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81d0016000 tx=0x7f81d0009f50 comp rx=0 tx=0).stop 2023-10-06T22:21:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.331+0000 7f81eefa5b80 1 -- 172.21.15.57:0/263819405 shutdown_connections 2023-10-06T22:21:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.331+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bf17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.331+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be3c00 0x561599be3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.331+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/263819405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599beee90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.331+0000 7f81eefa5b80 1 -- 172.21.15.57:0/263819405 >> 172.21.15.57:0/263819405 conn(0x561599b59200 msgr2=0x561599be0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.334+0000 7f81eefa5b80 1 -- 172.21.15.57:0/263819405 shutdown_connections 2023-10-06T22:21:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.334+0000 7f81eefa5b80 1 -- 172.21.15.57:0/263819405 wait complete. 2023-10-06T22:21:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.334+0000 7f81eefa5b80 1 -- start start 2023-10-06T22:21:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.335+0000 7f81eefa5b80 1 Processor -- start 2023-10-06T22:21:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.336+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599beee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.336+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be4520 0x561599bec010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.336+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bec750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.336+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599b37dd0 con 0x561599be4520 2023-10-06T22:21:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.336+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599ad46b0 con 0x561599be3c00 2023-10-06T22:21:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.336+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599ad4350 con 0x561599bef3d0 2023-10-06T22:21:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.336+0000 7f81eefa5b80 10 client.0 _ll_register_callbacks cb 0x5615999317a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.337+0000 7f81d9b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.337+0000 7f81d9b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bec750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48796/0 (socket says 172.21.15.57:48796) 2023-10-06T22:21:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.337+0000 7f81d9b9c700 1 -- 172.21.15.57:0/4209655800 learned_addr learned my addr 172.21.15.57:0/4209655800 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.338+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be4520 0x561599bec010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.338+0000 7f81d8b9a700 1 -- 172.21.15.57:0/4209655800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 msgr2=0x561599bec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.338+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bec750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.338+0000 7f81d8b9a700 1 -- 172.21.15.57:0/4209655800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 msgr2=0x561599beee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.338+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599beee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.338+0000 7f81d8b9a700 1 -- 172.21.15.57:0/4209655800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d411d3b0 con 0x561599be4520 2023-10-06T22:21:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.339+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be4520 0x561599bec010 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81c40cab00 tx=0x7f81c4099f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688128a8cd25ada5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.339+0000 7f81bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:21:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.341+0000 7f81bcff9700 1 -- 172.21.15.57:0/4209655800 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c409e320 con 0x561599be4520 2023-10-06T22:21:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.341+0000 7f81bcff9700 1 -- 172.21.15.57:0/4209655800 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f81c409a390 con 0x561599be4520 2023-10-06T22:21:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.341+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561599bd7d50 con 0x561599be4520 2023-10-06T22:21:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.342+0000 7f81bcff9700 1 -- 172.21.15.57:0/4209655800 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81c409e4e0 con 0x561599be4520 2023-10-06T22:21:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.342+0000 7f81eefa5b80 10 client.16647 fetch_fsmap learned FSMap version 47 2023-10-06T22:21:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.342+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56159994bc00 con 0x561599be4520 2023-10-06T22:21:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81bcff9700 1 -- 172.21.15.57:0/4209655800 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561599bd7d50 con 0x561599be4520 2023-10-06T22:21:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81bcff9700 1 -- 172.21.15.57:0/4209655800 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 47) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f81c40cd070 con 0x561599be4520 2023-10-06T22:21:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81eefa5b80 10 client.16647 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T22:21:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81eefa5b80 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81eefa5b80 20 client.16647 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599bfef50 0x561599c01370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81eefa5b80 4 client.16647 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:21:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.344+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561599b30db0 con 0x561599bfef50 2023-10-06T22:21:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.345+0000 7f81d939b700 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599bfef50 0x561599c01370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.349+0000 7f81d939b700 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599bfef50 0x561599c01370 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e76e5ee463ed49 server_cookie=282817931fa7b5fb in_seq=0 out_seq=0 2023-10-06T22:21:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.349+0000 7f81bcff9700 10 client.16647 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.357+0000 7f81bcff9700 1 -- 172.21.15.57:0/4209655800 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561599b30db0 con 0x561599bfef50 2023-10-06T22:21:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.357+0000 7f81bcff9700 10 client.16647 handle_command_reply: tid=0 2023-10-06T22:21:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.357+0000 7f81eefa5b80 1 client.16647 shutdown 2023-10-06T22:21:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be4520 msgr2=0x561599bec010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be4520 0x561599bec010 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81c40cab00 tx=0x7f81c4099f50 comp rx=0 tx=0).stop 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 shutdown_connections 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599bfef50 0x561599c01370 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599bef3d0 0x561599bec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599beee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4209655800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599be4520 0x561599bec010 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 >> 172.21.15.57:0/4209655800 conn(0x561599b59200 msgr2=0x561599bdfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.359+0000 7f81bcff9700 0 client.16647 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.360+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 shutdown_connections 2023-10-06T22:21:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.360+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4209655800 wait complete. 2023-10-06T22:21:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.360+0000 7f81eefa5b80 20 client.16647 trim_cache size 0 max 16384 2023-10-06T22:21:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.382+0000 7f81eefa5b80 1 Processor -- start 2023-10-06T22:21:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.382+0000 7f81eefa5b80 1 -- start start 2023-10-06T22:21:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be3c00 0x561599bee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599be72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599be7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615999b88c0 con 0x561599bef3d0 2023-10-06T22:21:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599be7e60 con 0x561599be4520 2023-10-06T22:21:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599bfad80 con 0x561599be3c00 2023-10-06T22:21:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81d939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be3c00 0x561599bee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81d8b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599be72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81d939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be3c00 0x561599bee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48800/0 (socket says 172.21.15.57:48800) 2023-10-06T22:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81d8b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599be72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55246/0 (socket says 172.21.15.57:55246) 2023-10-06T22:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81d939b700 1 -- 172.21.15.57:0/4060990802 learned_addr learned my addr 172.21.15.57:0/4060990802 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.383+0000 7f81d9b9c700 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599be7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81d8b9a700 1 -- 172.21.15.57:0/4060990802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be3c00 msgr2=0x561599bee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be3c00 0x561599bee930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81d8b9a700 1 -- 172.21.15.57:0/4060990802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 msgr2=0x561599be7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599be7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81d8b9a700 1 -- 172.21.15.57:0/4060990802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615999891f0 con 0x561599be4520 2023-10-06T22:21:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599be72f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f81c400cf50 tx=0x7f81c4098940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606d3d27c5e08f12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81bffff700 1 -- 172.21.15.57:0/4060990802 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c409e320 con 0x561599be4520 2023-10-06T22:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81bffff700 1 -- 172.21.15.57:0/4060990802 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81c409a390 con 0x561599be4520 2023-10-06T22:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.384+0000 7f81bffff700 1 -- 172.21.15.57:0/4060990802 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4099ae0 con 0x561599be4520 2023-10-06T22:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4060990802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 msgr2=0x561599be72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599be72f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f81c400cf50 tx=0x7f81c4098940 comp rx=0 tx=0).stop 2023-10-06T22:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4060990802 shutdown_connections 2023-10-06T22:21:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599be7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be3c00 0x561599bee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/4060990802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be4520 0x561599be72f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4060990802 >> 172.21.15.57:0/4060990802 conn(0x561599b59200 msgr2=0x561599bfa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4060990802 shutdown_connections 2023-10-06T22:21:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 -- 172.21.15.57:0/4060990802 wait complete. 2023-10-06T22:21:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.385+0000 7f81eefa5b80 1 -- start start 2023-10-06T22:21:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 1 Processor -- start 2023-10-06T22:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599c4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be4520 0x561599c4c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599bf1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599bf2430 con 0x561599bef3d0 2023-10-06T22:21:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599bf25a0 con 0x561599be3c00 2023-10-06T22:21:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599bf2760 con 0x561599be4520 2023-10-06T22:21:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81eefa5b80 10 client.0 _ll_register_callbacks cb 0x7f81c40966f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81d939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599c4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81d939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599c4bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55262/0 (socket says 172.21.15.57:55262) 2023-10-06T22:21:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.386+0000 7f81d939b700 1 -- 172.21.15.57:0/138765768 learned_addr learned my addr 172.21.15.57:0/138765768 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be4520 0x561599c4c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d9b9c700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599bf1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d939b700 1 -- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be4520 msgr2=0x561599c4c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d939b700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be4520 0x561599c4c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d939b700 1 -- 172.21.15.57:0/138765768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 msgr2=0x561599bf1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d939b700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599bf1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d939b700 1 -- 172.21.15.57:0/138765768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159994bc00 con 0x561599be3c00 2023-10-06T22:21:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d9b9c700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599bf1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:21:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81d939b700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599c4bfd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f81d0016000 tx=0x7f81d0005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20066f920cf49a5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:21:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81bcff9700 1 -- 172.21.15.57:0/138765768 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d001d040 con 0x561599be3c00 2023-10-06T22:21:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.387+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81d000f920 con 0x561599be3c00 2023-10-06T22:21:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.388+0000 7f81bcff9700 1 -- 172.21.15.57:0/138765768 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f81d000f660 con 0x561599be3c00 2023-10-06T22:21:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.388+0000 7f81bcff9700 1 -- 172.21.15.57:0/138765768 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81d002c9e0 con 0x561599be3c00 2023-10-06T22:21:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.388+0000 7f81eefa5b80 10 client.26693 fetch_fsmap learned FSMap version 47 2023-10-06T22:21:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.388+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x561599961250 con 0x561599be3c00 2023-10-06T22:21:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.388+0000 7f81bcff9700 1 -- 172.21.15.57:0/138765768 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d000f920 con 0x561599be3c00 2023-10-06T22:21:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.388+0000 7f81bcff9700 1 -- 172.21.15.57:0/138765768 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 47) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f81d0015020 con 0x561599be3c00 2023-10-06T22:21:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.389+0000 7f81eefa5b80 10 client.26693 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T22:21:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.389+0000 7f81eefa5b80 10 client.26693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.389+0000 7f81eefa5b80 20 client.26693 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.389+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599c53a30 0x561599c55e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.389+0000 7f81eefa5b80 4 client.26693 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:21:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.389+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561599b30ad0 con 0x561599c53a30 2023-10-06T22:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.389+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599c53a30 0x561599c55e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.390+0000 7f81bcff9700 10 client.26693 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.390+0000 7f81d8b9a700 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599c53a30 0x561599c55e50 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7869c1b6ecdfbe4 server_cookie=b2f71da9af9f1442 in_seq=0 out_seq=0 2023-10-06T22:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.545+0000 7f81bcff9700 1 -- 172.21.15.57:0/138765768 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561599b30ad0 con 0x561599c53a30 2023-10-06T22:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.545+0000 7f81bcff9700 10 client.26693 handle_command_reply: tid=0 2023-10-06T22:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.545+0000 7f81eefa5b80 1 client.26693 shutdown 2023-10-06T22:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 msgr2=0x561599c4bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599c4bfd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f81d0016000 tx=0x7f81d0005a00 comp rx=0 tx=0).stop 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 shutdown_connections 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x561599c53a30 0x561599c55e50 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561599bef3d0 0x561599bf1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81bcff9700 0 client.26693 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561599be3c00 0x561599c4bfd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 --2- 172.21.15.57:0/138765768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561599be4520 0x561599c4c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 >> 172.21.15.57:0/138765768 conn(0x561599b59200 msgr2=0x561599be72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 shutdown_connections 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 1 -- 172.21.15.57:0/138765768 wait complete. 2023-10-06T22:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.546+0000 7f81eefa5b80 20 client.26693 trim_cache size 0 max 16384 2023-10-06T22:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f81c00a1800 msgr2=0x7f81c00a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f81c00a1800 0x7f81c00a3cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f81d406a250 tx=0x7f81cc0099e0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410fa90 msgr2=0x7f81d41a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410fa90 0x7f81d41a63d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e9a0 tx=0x7f81c400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 shutdown_connections 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f81c00a1800 0x7f81c00a3cc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d411a690 0x7f81d41aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81d410f620 0x7f81d41a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 --2- 172.21.15.57:0/2712903575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81d410fa90 0x7f81d41a63d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 >> 172.21.15.57:0/2712903575 conn(0x7f81d406cfa0 msgr2=0x7f81d4118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 shutdown_connections 2023-10-06T22:21:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:39.547+0000 7f81db5ff700 1 -- 172.21.15.57:0/2712903575 wait complete. 2023-10-06T22:21:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:39 smithi190 ceph-mon[134891]: pgmap v212: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 320 op/s 2023-10-06T22:21:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:39 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:21:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:39 smithi057 ceph-mon[98492]: pgmap v212: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 320 op/s 2023-10-06T22:21:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:39 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:21:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:39 smithi156 ceph-mon[134358]: pgmap v212: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 320 op/s 2023-10-06T22:21:39.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:39 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:21:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:41 smithi190 ceph-mon[134891]: pgmap v213: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2023-10-06T22:21:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:41 smithi057 ceph-mon[98492]: pgmap v213: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2023-10-06T22:21:41.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:41 smithi156 ceph-mon[134358]: pgmap v213: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.4 MiB/s wr, 239 op/s 2023-10-06T22:21:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:43 smithi190 ceph-mon[134891]: pgmap v214: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2023-10-06T22:21:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:43 smithi057 ceph-mon[98492]: pgmap v214: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2023-10-06T22:21:43.835 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:43 smithi156 ceph-mon[134358]: pgmap v214: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2023-10-06T22:21:44.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:44 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:21:44.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:44 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:21:44.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:44 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:21:44.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:44 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:21:44.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:44 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:21:44.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:44 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:21:44.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:44 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:21:44.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:44 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:21:44.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:44 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:21:45.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:45 smithi057 ceph-mon[98492]: pgmap v215: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-10-06T22:21:46.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:45 smithi156 ceph-mon[134358]: pgmap v215: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-10-06T22:21:46.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:45 smithi190 ceph-mon[134891]: pgmap v215: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-10-06T22:21:47.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:47 smithi190 ceph-mon[134891]: pgmap v216: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.7 MiB/s wr, 172 op/s 2023-10-06T22:21:47.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:47 smithi057 ceph-mon[98492]: pgmap v216: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.7 MiB/s wr, 172 op/s 2023-10-06T22:21:47.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:47 smithi156 ceph-mon[134358]: pgmap v216: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.7 MiB/s wr, 172 op/s 2023-10-06T22:21:49.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:49 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:49.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:49 smithi190 ceph-mon[134891]: pgmap v217: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 198 op/s 2023-10-06T22:21:49.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:49 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:49.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:49 smithi057 ceph-mon[98492]: pgmap v217: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 198 op/s 2023-10-06T22:21:49.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:49 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:49.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:49 smithi156 ceph-mon[134358]: pgmap v217: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 198 op/s 2023-10-06T22:21:50.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:50 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:50.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:50 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:50.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:50 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:51.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:51 smithi057 ceph-mon[98492]: pgmap v218: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 163 op/s 2023-10-06T22:21:51.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:51 smithi156 ceph-mon[134358]: pgmap v218: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 163 op/s 2023-10-06T22:21:51.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:51 smithi190 ceph-mon[134891]: pgmap v218: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 163 op/s 2023-10-06T22:21:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.708+0000 7f29d6fff700 1 -- 172.21.15.57:0/3319481432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0101ab0 msgr2=0x7f29d0105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.708+0000 7f29d6fff700 1 --2- 172.21.15.57:0/3319481432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0101ab0 0x7f29d0105af0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0098f0 tx=0x7f29bc009c00 comp rx=0 tx=0).stop 2023-10-06T22:21:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.709+0000 7f29d6fff700 1 -- 172.21.15.57:0/3319481432 shutdown_connections 2023-10-06T22:21:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.709+0000 7f29d6fff700 1 --2- 172.21.15.57:0/3319481432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29d0106220 0x7f29d010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.709+0000 7f29d6fff700 1 --2- 172.21.15.57:0/3319481432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d0101100 0x7f29d01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.709+0000 7f29d6fff700 1 --2- 172.21.15.57:0/3319481432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0101ab0 0x7f29d0105af0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.709+0000 7f29d6fff700 1 -- 172.21.15.57:0/3319481432 >> 172.21.15.57:0/3319481432 conn(0x7f29d00fc9a0 msgr2=0x7f29d00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.709+0000 7f29d6fff700 1 -- 172.21.15.57:0/3319481432 shutdown_connections 2023-10-06T22:21:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.709+0000 7f29d6fff700 1 -- 172.21.15.57:0/3319481432 wait complete. 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 Processor -- start 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 -- start start 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29d0101100 0x7f29d0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0106220 0x7f29d00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d01a31e0 0x7f29d01a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a5b10 con 0x7f29d0101100 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a5c50 con 0x7f29d01a31e0 2023-10-06T22:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29d6fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a5dc0 con 0x7f29d0106220 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.710+0000 7f29cffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0106220 0x7f29d00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29d0101100 0x7f29d0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29d559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d01a31e0 0x7f29d01a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0106220 0x7f29d00738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43366/0 (socket says 172.21.15.57:43366) 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 -- 172.21.15.57:0/790952102 learned_addr learned my addr 172.21.15.57:0/790952102 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 -- 172.21.15.57:0/790952102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d01a31e0 msgr2=0x7f29d01a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d01a31e0 0x7f29d01a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 -- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29d0101100 msgr2=0x7f29d0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29d0101100 0x7f29d0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 -- 172.21.15.57:0/790952102 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40095f0 con 0x7f29d0106220 2023-10-06T22:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29d0101100 0x7f29d0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cffff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0106220 0x7f29d00738d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009230 tx=0x7f29bc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cdffb700 1 -- 172.21.15.57:0/790952102 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc01c070 con 0x7f29d0106220 2023-10-06T22:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.711+0000 7f29cdffb700 1 -- 172.21.15.57:0/790952102 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29bc00b9c0 con 0x7f29d0106220 2023-10-06T22:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.712+0000 7f29cdffb700 1 -- 172.21.15.57:0/790952102 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc004930 con 0x7f29d0106220 2023-10-06T22:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.712+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc009520 con 0x7f29d0106220 2023-10-06T22:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.712+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d01a6280 con 0x7f29d0106220 2023-10-06T22:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.715+0000 7f29cdffb700 1 -- 172.21.15.57:0/790952102 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f29bc020070 con 0x7f29d0106220 2023-10-06T22:21:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 Processor -- start 2023-10-06T22:21:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 -- start start 2023-10-06T22:21:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b4c00 0x5629be5b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be51e400 con 0x5629be5b4c00 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be50fbc0 con 0x5629be5c03d0 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be50d4e0 con 0x5629be5b5520 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.717+0000 7f29cdffb700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29c00a1670 0x7f29c00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29cffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5bfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43368/0 (socket says 172.21.15.57:43368) 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1345212826 learned_addr learned my addr 172.21.15.57:0/1345212826 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d559c700 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b4c00 0x5629be5b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29cdffb700 1 -- 172.21.15.57:0/790952102 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f29bc0cdd80 con 0x7f29d0106220 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1345212826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 msgr2=0x5629be5c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d559c700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29c00a1670 0x7f29c00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1345212826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b4c00 msgr2=0x5629be5b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b4c00 0x5629be5b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1345212826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629be333d80 con 0x5629be5b5520 2023-10-06T22:21:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.718+0000 7f29cffff700 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5bfe90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f29c40117e0 tx=0x7f29c4011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84c929ed52c3723 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29d559c700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29c00a1670 0x7f29c00a3b30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009ec0 tx=0x7f29c800b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29baffd700 1 -- 172.21.15.57:0/1345212826 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4004020 con 0x5629be5b5520 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29baffd700 1 -- 172.21.15.57:0/1345212826 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c4003680 con 0x5629be5b5520 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29baffd700 1 -- 172.21.15.57:0/1345212826 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400f840 con 0x5629be5b5520 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1345212826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 msgr2=0x5629be5bfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5bfe90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f29c40117e0 tx=0x7f29c4011af0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1345212826 shutdown_connections 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.719+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.720+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b4c00 0x5629be5b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.720+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1345212826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5bfe90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.720+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1345212826 >> 172.21.15.57:0/1345212826 conn(0x5629be52a200 msgr2=0x5629be5b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.720+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1345212826 shutdown_connections 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.720+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1345212826 wait complete. 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.720+0000 7f29ea90bb80 1 -- start start 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.720+0000 7f29ea90bb80 1 Processor -- start 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5bd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be508dd0 con 0x5629be5b5520 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be4a56b0 con 0x5629be5c03d0 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be4a5350 con 0x5629be5b4c00 2023-10-06T22:21:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29ea90bb80 10 client.0 _ll_register_callbacks cb 0x5629be3027a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29cffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5bd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5bfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43382/0 (socket says 172.21.15.57:43382) 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d4d9b700 1 -- 172.21.15.57:0/21846947 learned_addr learned my addr 172.21.15.57:0/21846947 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d4d9b700 1 -- 172.21.15.57:0/21846947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 msgr2=0x5629be5bd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5bd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d4d9b700 1 -- 172.21.15.57:0/21846947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 msgr2=0x5629be5bd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.721+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5bd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29d4d9b700 1 -- 172.21.15.57:0/21846947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d01a6280 con 0x5629be5b4c00 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29cffff700 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5bd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29d559c700 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5bfe70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29c4005950 tx=0x7f29c4005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b59b347aeabb676 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29affff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29affff700 1 -- 172.21.15.57:0/21846947 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4004220 con 0x5629be5b4c00 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629be5a8d50 con 0x5629be5b4c00 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29affff700 1 -- 172.21.15.57:0/21846947 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f29c4004380 con 0x5629be5b4c00 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29affff700 1 -- 172.21.15.57:0/21846947 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29c4004540 con 0x5629be5b4c00 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29ea90bb80 10 client.26713 fetch_fsmap learned FSMap version 49 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.722+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f29c4011440 con 0x5629be5b4c00 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29affff700 1 -- 172.21.15.57:0/21846947 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629be5a8d50 con 0x5629be5b4c00 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29affff700 1 -- 172.21.15.57:0/21846947 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f29c4004ac0 con 0x5629be5b4c00 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29ea90bb80 10 client.26713 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T22:21:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29ea90bb80 10 client.26713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29ea90bb80 20 client.26713 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be5cfed0 0x5629be5d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29ea90bb80 4 client.26713 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629be501db0 con 0x5629be5cfed0 2023-10-06T22:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.723+0000 7f29cffff700 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be5cfed0 0x5629be5d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.724+0000 7f29cffff700 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be5cfed0 0x5629be5d22f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d55d4e946f7e95 server_cookie=6b426388fa2196de in_seq=0 out_seq=0 2023-10-06T22:21:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.724+0000 7f29affff700 10 client.26713 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.724+0000 7f29affff700 1 -- 172.21.15.57:0/21846947 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5629be501db0 con 0x5629be5cfed0 2023-10-06T22:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.724+0000 7f29affff700 10 client.26713 handle_command_reply: tid=0 2023-10-06T22:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 client.26713 shutdown 2023-10-06T22:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 msgr2=0x5629be5bfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5bfe70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29c4005950 tx=0x7f29c4005ab0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 shutdown_connections 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be5cfed0 0x5629be5d22f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29affff700 0 client.26713 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5bfe70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/21846947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5bd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 >> 172.21.15.57:0/21846947 conn(0x5629be52a200 msgr2=0x5629be5b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 shutdown_connections 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 1 -- 172.21.15.57:0/21846947 wait complete. 2023-10-06T22:21:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.725+0000 7f29ea90bb80 20 client.26713 trim_cache size 0 max 16384 2023-10-06T22:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.740+0000 7f29ea90bb80 1 Processor -- start 2023-10-06T22:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.740+0000 7f29ea90bb80 1 -- start start 2023-10-06T22:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5b4c00 0x5629be5d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5c03d0 0x5629be5b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be3898c0 con 0x5629be5c03d0 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be5c7460 con 0x5629be5b4c00 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be5b5900 con 0x5629be5b5520 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5b4c00 0x5629be5d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29cffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5b4c00 0x5629be5d22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39886/0 (socket says 172.21.15.57:39886) 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29cffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43390/0 (socket says 172.21.15.57:43390) 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29d4d9b700 1 -- 172.21.15.57:0/3969491041 learned_addr learned my addr 172.21.15.57:0/3969491041 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29d559c700 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5c03d0 0x5629be5b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29cffff700 1 -- 172.21.15.57:0/3969491041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5b4c00 msgr2=0x5629be5d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29cffff700 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5b4c00 0x5629be5d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29cffff700 1 -- 172.21.15.57:0/3969491041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5c03d0 msgr2=0x5629be5b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29cffff700 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5c03d0 0x5629be5b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29cffff700 1 -- 172.21.15.57:0/3969491041 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629be31cc00 con 0x5629be5b5520 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29d559c700 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5c03d0 0x5629be5b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.741+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5b4c00 0x5629be5d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29cffff700 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5c6f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0cb490 tx=0x7f29bc0a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9af4dc461901cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29bb7fe700 1 -- 172.21.15.57:0/3969491041 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc066620 con 0x5629be5b5520 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29bb7fe700 1 -- 172.21.15.57:0/3969491041 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29bc09adb0 con 0x5629be5b5520 2023-10-06T22:21:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29bb7fe700 1 -- 172.21.15.57:0/3969491041 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0991e0 con 0x5629be5b5520 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29ea90bb80 1 -- 172.21.15.57:0/3969491041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 msgr2=0x5629be5c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5c6f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0cb490 tx=0x7f29bc0a1040 comp rx=0 tx=0).stop 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29ea90bb80 1 -- 172.21.15.57:0/3969491041 shutdown_connections 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5c03d0 0x5629be5b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5b4c00 0x5629be5d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/3969491041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b5520 0x5629be5c6f20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.742+0000 7f29ea90bb80 1 -- 172.21.15.57:0/3969491041 >> 172.21.15.57:0/3969491041 conn(0x5629be52a200 msgr2=0x5629be5b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.743+0000 7f29ea90bb80 1 -- 172.21.15.57:0/3969491041 shutdown_connections 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.743+0000 7f29ea90bb80 1 -- 172.21.15.57:0/3969491041 wait complete. 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.743+0000 7f29ea90bb80 1 -- start start 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.743+0000 7f29ea90bb80 1 Processor -- start 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5c4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29ea90bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be622d20 con 0x5629be5b5520 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be622e60 con 0x5629be5c03d0 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29ea90bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be622fa0 con 0x5629be5b4c00 2023-10-06T22:21:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29ea90bb80 10 client.0 _ll_register_callbacks cb 0x7f29c8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29cffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:43392/0 (socket says 172.21.15.57:43392) 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1446210636 learned_addr learned my addr 172.21.15.57:0/1446210636 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1446210636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 msgr2=0x5629be5c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1446210636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 msgr2=0x5629be5c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29d4d9b700 1 -- 172.21.15.57:0/1446210636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c4011440 con 0x5629be5b4c00 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.744+0000 7f29cffff700 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29d4d9b700 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5c4330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29c4004ed0 tx=0x7f29c4009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1571f8c70540162c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:21:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29affff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29affff700 1 -- 172.21.15.57:0/1446210636 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400fbb0 con 0x5629be5b4c00 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29affff700 1 -- 172.21.15.57:0/1446210636 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f29c4000e40 con 0x5629be5b4c00 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29c4003680 con 0x5629be5b4c00 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29affff700 1 -- 172.21.15.57:0/1446210636 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29c400f850 con 0x5629be5b4c00 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29ea90bb80 10 client.26719 fetch_fsmap learned FSMap version 49 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f29bc009520 con 0x5629be5b4c00 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29affff700 1 -- 172.21.15.57:0/1446210636 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c4003680 con 0x5629be5b4c00 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.745+0000 7f29affff700 1 -- 172.21.15.57:0/1446210636 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 49) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f29c40183b0 con 0x5629be5b4c00 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29ea90bb80 10 client.26719 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29ea90bb80 10 client.26719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29ea90bb80 20 client.26719 populate_metadata read hostname 'smithi057' 2023-10-06T22:21:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be624460 0x5629be626880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:21:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29ea90bb80 4 client.26719 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:21:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629be501ad0 con 0x5629be624460 2023-10-06T22:21:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29cffff700 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be624460 0x5629be626880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:21:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29cffff700 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be624460 0x5629be626880 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b18922ee7e9b2a0 server_cookie=fec4fa98713157b in_seq=0 out_seq=0 2023-10-06T22:21:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.746+0000 7f29affff700 10 client.26719 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29affff700 1 -- 172.21.15.57:0/1446210636 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629be501ad0 con 0x5629be624460 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29affff700 10 client.26719 handle_command_reply: tid=0 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 client.26719 shutdown 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 msgr2=0x5629be5c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5c4330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29c4004ed0 tx=0x7f29c4009f30 comp rx=0 tx=0).stop 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 shutdown_connections 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5629be624460 0x5629be626880 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629be5c03d0 0x5629be5c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629be5b4c00 0x5629be5c4330 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 --2- 172.21.15.57:0/1446210636 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5629be5b5520 0x5629be5c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29affff700 0 client.26719 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 >> 172.21.15.57:0/1446210636 conn(0x5629be52a200 msgr2=0x5629be5b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 shutdown_connections 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.836+0000 7f29ea90bb80 1 -- 172.21.15.57:0/1446210636 wait complete. 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29ea90bb80 20 client.26719 trim_cache size 0 max 16384 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:21:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29c00a1670 msgr2=0x7f29c00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29c00a1670 0x7f29c00a3b30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009ec0 tx=0x7f29c800b040 comp rx=0 tx=0).stop 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0106220 msgr2=0x7f29d00738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0106220 0x7f29d00738d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009230 tx=0x7f29bc00fa00 comp rx=0 tx=0).stop 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 shutdown_connections 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29c00a1670 0x7f29c00a3b30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29d01a31e0 0x7f29d01a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29d0101100 0x7f29d0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 --2- 172.21.15.57:0/790952102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d0106220 0x7f29d00738d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 >> 172.21.15.57:0/790952102 conn(0x7f29d00fc9a0 msgr2=0x7f29d00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 shutdown_connections 2023-10-06T22:21:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:21:52.837+0000 7f29d6fff700 1 -- 172.21.15.57:0/790952102 wait complete. 2023-10-06T22:21:53.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:53 smithi156 ceph-mon[134358]: pgmap v219: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 164 op/s 2023-10-06T22:21:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:53 smithi190 ceph-mon[134891]: pgmap v219: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 164 op/s 2023-10-06T22:21:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:53 smithi057 ceph-mon[98492]: pgmap v219: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 164 op/s 2023-10-06T22:21:54.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:54 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:54.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:54 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:54.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:54 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:55.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:55 smithi156 ceph-mon[134358]: pgmap v220: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 97 op/s 2023-10-06T22:21:55.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:55 smithi190 ceph-mon[134891]: pgmap v220: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 97 op/s 2023-10-06T22:21:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:55 smithi057 ceph-mon[98492]: pgmap v220: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 97 op/s 2023-10-06T22:21:57.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:57 smithi190 ceph-mon[134891]: pgmap v221: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 95 op/s 2023-10-06T22:21:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:57 smithi057 ceph-mon[98492]: pgmap v221: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 95 op/s 2023-10-06T22:21:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:57 smithi156 ceph-mon[134358]: pgmap v221: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 95 op/s 2023-10-06T22:21:58.503 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:21:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:21:58.203+0000 7f75ce516700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x200000016ea [...a,head] /volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29/ auth{0=1,1=1,3=1} fragtree_t(*^3) v177827 ap=2 f(v13 m2023-10-06T22:21:01.314835+0000 195=195+0) n(v19 rc2023-10-06T22:21:01.314835+0000 b3443992 196=195+1) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15228=pAsLsXsFs/pAsLsXsFsxcral@394} mcw={1=AsLsXsFs,3=AsLsXsFs} | dirtyscattered=0 request=0 lock=3 importingcaps=0 importing=0 dirfrag=8 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55c41086e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x200000016ea:[<0x20000000207/blog-29 v139720>,<0x10000000206/blogtest_in v1630>,<0x10000000201/src v1823>,<0x10000000006/blogbench-1.0 v1660>,<0x10000000005/tmp v78>,<0x100000001fb/client.0 v76>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v94>,<0x10000000001/sv_1 v109>,<0x10000000000/_nogroup v96>,<0x1/volumes v93>]//[]","memoryvalue":"(2)0x200000016ea:[<0x20000000207/blog-29 v177827>,<0x10000000206/blogtest_in v2570>,<0x10000000201/src v1865>,<0x10000000006/blogbench-1.0 v1638>,<0x10000000005/tmp v56>,<0x100000001fb/client.0 v54>,<0x100000001fa/064cd3dd-8455-4835-b2cc-d301e4708adc v72>,<0x10000000001/sv_1 v151>,<0x10000000000/_nogroup v74>,<0x1/volumes v133>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-06T22:21:01.314835+0000 195=195+0)","ondisk_value.rstat":"n(v0 rc2023-10-06T22:21:01.314835+0000 b3443992 196=195+1)","memory_value.dirstat":"f(v13 m2023-10-06T22:21:01.314835+0000 195=195+0)","memory_value.rstat":"n(v19 rc2023-10-06T22:21:01.314835+0000 b3443992 196=195+1)","error_str":""},"return_code":0} 2023-10-06T22:21:59.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:59.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:21:59.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:21:59.320 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: pgmap v222: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 100 op/s 2023-10-06T22:21:59.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:21:59.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:21:59.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:21:59 smithi190 ceph-mon[134891]: Scrub error on inode 0x200000016ea (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.h log and `damage ls` output for details 2023-10-06T22:21:59.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:59.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:21:59.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:21:59.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: pgmap v222: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 100 op/s 2023-10-06T22:21:59.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:21:59.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:21:59.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:21:59 smithi057 ceph-mon[98492]: Scrub error on inode 0x200000016ea (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.h log and `damage ls` output for details 2023-10-06T22:21:59.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: pgmap v222: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 100 op/s 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:21:59.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:21:59 smithi156 ceph-mon[134358]: Scrub error on inode 0x200000016ea (/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29) see mds.h log and `damage ls` output for details 2023-10-06T22:22:01.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:01 smithi156 ceph-mon[134358]: pgmap v223: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 709 KiB/s wr, 58 op/s 2023-10-06T22:22:01.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:01 smithi190 ceph-mon[134891]: pgmap v223: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 709 KiB/s wr, 58 op/s 2023-10-06T22:22:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:01 smithi057 ceph-mon[98492]: pgmap v223: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 709 KiB/s wr, 58 op/s 2023-10-06T22:22:02.867 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 -- 172.21.15.57:0/39724917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 msgr2=0x7fc14010ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 --2- 172.21.15.57:0/39724917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 0x7fc14010ae50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc134009990 tx=0x7fc134009ca0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 -- 172.21.15.57:0/39724917 shutdown_connections 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 --2- 172.21.15.57:0/39724917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc14010b470 0x7fc14010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 --2- 172.21.15.57:0/39724917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc140102050 0x7fc140102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 --2- 172.21.15.57:0/39724917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 0x7fc14010ae50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 -- 172.21.15.57:0/39724917 >> 172.21.15.57:0/39724917 conn(0x7fc1400fb890 msgr2=0x7fc1400fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:03.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 -- 172.21.15.57:0/39724917 shutdown_connections 2023-10-06T22:22:03.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.082+0000 7fc144bff700 1 -- 172.21.15.57:0/39724917 wait complete. 2023-10-06T22:22:03.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.083+0000 7fc144bff700 1 Processor -- start 2023-10-06T22:22:03.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc144bff700 1 -- start start 2023-10-06T22:22:03.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc144bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc140102050 0x7fc1400ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc144bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 0x7fc1401001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc144bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc14010b470 0x7fc1401a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc144bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140100a80 con 0x7fc14010b470 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc144bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401a5770 con 0x7fc140102970 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc144bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401a58b0 con 0x7fc140102050 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc13e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc140102050 0x7fc1400ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 0x7fc1401001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc13e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc140102050 0x7fc1400ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33286/0 (socket says 172.21.15.57:33286) 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc13e59c700 1 -- 172.21.15.57:0/3203213639 learned_addr learned my addr 172.21.15.57:0/3203213639 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.084+0000 7fc13ed9d700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc14010b470 0x7fc1401a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc13e59c700 1 -- 172.21.15.57:0/3203213639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 msgr2=0x7fc1401001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 0x7fc1401001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc13e59c700 1 -- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc14010b470 msgr2=0x7fc1401a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc14010b470 0x7fc1401a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc13e59c700 1 -- 172.21.15.57:0/3203213639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340095f0 con 0x7fc140102050 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc140102050 0x7fc1400ffca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc13000e9a0 tx=0x7fc13000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc12f7fe700 1 -- 172.21.15.57:0/3203213639 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130009790 con 0x7fc140102050 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc12f7fe700 1 -- 172.21.15.57:0/3203213639 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc130010430 con 0x7fc140102050 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1401a5b60 con 0x7fc140102050 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc12f7fe700 1 -- 172.21.15.57:0/3203213639 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130018680 con 0x7fc140102050 2023-10-06T22:22:03.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.085+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1401a6070 con 0x7fc140102050 2023-10-06T22:22:03.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.089+0000 7fc12f7fe700 1 -- 172.21.15.57:0/3203213639 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc1300105a0 con 0x7fc140102050 2023-10-06T22:22:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.090+0000 7fc1585c1b80 1 Processor -- start 2023-10-06T22:22:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.090+0000 7fc1585c1b80 1 -- start start 2023-10-06T22:22:03.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e233b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e233c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e23322460 con 0x562e233b99e0 2023-10-06T22:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e23313c20 con 0x562e233c4890 2023-10-06T22:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e23311540 con 0x562e233b90c0 2023-10-06T22:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e233b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e233b94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33290/0 (socket says 172.21.15.57:33290) 2023-10-06T22:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc13ed9d700 1 -- 172.21.15.57:0/3569109268 learned_addr learned my addr 172.21.15.57:0/3569109268 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e233c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc12f7fe700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc1280a15f0 0x7fc1280a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc13dd9b700 1 -- 172.21.15.57:0/3569109268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 msgr2=0x562e233b94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.091+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e233b94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc13dd9b700 1 -- 172.21.15.57:0/3569109268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 msgr2=0x562e233c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e233c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc13dd9b700 1 -- 172.21.15.57:0/3569109268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e23137de0 con 0x562e233c4890 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e233c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc12f7fe700 1 -- 172.21.15.57:0/3203213639 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fc130014070 con 0x7fc140102050 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc13ed9d700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc1280a15f0 0x7fc1280a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c6c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b2e0 tx=0x7fc134005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535dd1e49b87f5a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc12cff9700 1 -- 172.21.15.57:0/3569109268 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13401d070 con 0x562e233c4890 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc12cff9700 1 -- 172.21.15.57:0/3569109268 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13401e040 con 0x562e233c4890 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.092+0000 7fc12cff9700 1 -- 172.21.15.57:0/3569109268 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134004920 con 0x562e233c4890 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3569109268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 msgr2=0x562e233c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc13ed9d700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc1280a15f0 0x7fc1280a3ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc1380075c0 tx=0x7fc138009040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c6c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b2e0 tx=0x7fc134005ea0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3569109268 shutdown_connections 2023-10-06T22:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c6c80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e233b94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3569109268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e233c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3569109268 >> 172.21.15.57:0/3569109268 conn(0x562e2332e260 msgr2=0x562e233b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3569109268 shutdown_connections 2023-10-06T22:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.093+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3569109268 wait complete. 2023-10-06T22:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 -- start start 2023-10-06T22:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 Processor -- start 2023-10-06T22:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b99e0 0x562e233c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2330ce30 con 0x562e233b90c0 2023-10-06T22:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e232a9710 con 0x562e233c4890 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e232a93b0 con 0x562e233b99e0 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.094+0000 7fc1585c1b80 10 client.0 _ll_register_callbacks cb 0x562e23106800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b99e0 0x562e233c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b99e0 0x562e233c14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33292/0 (socket says 172.21.15.57:33292) 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:40462/0 (socket says 172.21.15.57:40462) 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 -- 172.21.15.57:0/2425804470 learned_addr learned my addr 172.21.15.57:0/2425804470 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 -- 172.21.15.57:0/2425804470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 msgr2=0x562e233c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 -- 172.21.15.57:0/2425804470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 msgr2=0x562e233c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 -- 172.21.15.57:0/2425804470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14010e5f0 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b99e0 0x562e233c14c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc1340057c0 tx=0x7fc13400f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae401290a8080f32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc125ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.095+0000 7fc125ffb700 1 -- 172.21.15.57:0/2425804470 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13401d040 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e233acf10 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc125ffb700 1 -- 172.21.15.57:0/2425804470 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fc13400fda0 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc125ffb700 1 -- 172.21.15.57:0/2425804470 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc134005030 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 10 client.26731 fetch_fsmap learned FSMap version 51 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x562e23120c60 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc125ffb700 1 -- 172.21.15.57:0/2425804470 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e233acf10 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc125ffb700 1 -- 172.21.15.57:0/2425804470 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 51) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fc134004b70 con 0x562e233b99e0 2023-10-06T22:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 10 client.26731 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 20 client.26731 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e233d3f40 0x562e233d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 4 client.26731 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.096+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e23305e10 con 0x562e233d3f40 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.097+0000 7fc13e59c700 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e233d3f40 0x562e233d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.097+0000 7fc13e59c700 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e233d3f40 0x562e233d6360 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df2d4beab1105a6 server_cookie=363d4339071f684 in_seq=0 out_seq=0 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.097+0000 7fc125ffb700 10 client.26731 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc125ffb700 1 -- 172.21.15.57:0/2425804470 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e23305e10 con 0x562e233d3f40 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc125ffb700 10 client.26731 handle_command_reply: tid=0 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 client.26731 shutdown 2023-10-06T22:22:03.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b99e0 msgr2=0x562e233c14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b99e0 0x562e233c14c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc1340057c0 tx=0x7fc13400f670 comp rx=0 tx=0).stop 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 shutdown_connections 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e233d3f40 0x562e233d6360 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e233c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc125ffb700 0 client.26731 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/2425804470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b99e0 0x562e233c14c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 >> 172.21.15.57:0/2425804470 conn(0x562e2332e260 msgr2=0x562e233b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 shutdown_connections 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 1 -- 172.21.15.57:0/2425804470 wait complete. 2023-10-06T22:22:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.098+0000 7fc1585c1b80 20 client.26731 trim_cache size 0 max 16384 2023-10-06T22:22:03.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 Processor -- start 2023-10-06T22:22:03.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 -- start start 2023-10-06T22:22:03.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233b99e0 0x562e233cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233c4890 0x562e233b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2318d920 con 0x562e233b90c0 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e233ba2e0 con 0x562e233b99e0 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e233c2600 con 0x562e233c4890 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc13e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233c4890 0x562e233b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233b99e0 0x562e233cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233b99e0 0x562e233cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:40470/0 (socket says 172.21.15.57:40470) 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.116+0000 7fc13dd9b700 1 -- 172.21.15.57:0/1246503867 learned_addr learned my addr 172.21.15.57:0/1246503867 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc13ed9d700 1 -- 172.21.15.57:0/1246503867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233b99e0 msgr2=0x562e233cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc13ed9d700 1 --2- 172.21.15.57:0/1246503867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233b99e0 0x562e233cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc13ed9d700 1 -- 172.21.15.57:0/1246503867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 msgr2=0x562e233c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc13ed9d700 1 --2- 172.21.15.57:0/1246503867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc13ed9d700 1 -- 172.21.15.57:0/1246503867 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e2315e250 con 0x562e233c4890 2023-10-06T22:22:03.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc13ed9d700 1 --2- 172.21.15.57:0/1246503867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233c4890 0x562e233b3bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc138017840 tx=0x7fc138004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b044042d41cf3fa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:03.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc12d7fa700 1 -- 172.21.15.57:0/1246503867 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138028070 con 0x562e233c4890 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc12d7fa700 1 -- 172.21.15.57:0/1246503867 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13800f040 con 0x562e233c4890 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.117+0000 7fc12d7fa700 1 -- 172.21.15.57:0/1246503867 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800e040 con 0x562e233c4890 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 -- 172.21.15.57:0/1246503867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233c4890 msgr2=0x562e233b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/1246503867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233c4890 0x562e233b3bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc138017840 tx=0x7fc138004b40 comp rx=0 tx=0).stop 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 -- 172.21.15.57:0/1246503867 shutdown_connections 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/1246503867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233c4890 0x562e233b3bc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/1246503867 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b90c0 0x562e233c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/1246503867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233b99e0 0x562e233cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 -- 172.21.15.57:0/1246503867 >> 172.21.15.57:0/1246503867 conn(0x562e2332e260 msgr2=0x562e233d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:03.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 -- 172.21.15.57:0/1246503867 shutdown_connections 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 -- 172.21.15.57:0/1246503867 wait complete. 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.118+0000 7fc1585c1b80 1 -- start start 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 1 Processor -- start 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e2341beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e2341c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e23420d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e234212c0 con 0x562e233b99e0 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e23421400 con 0x562e233c4890 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e234215c0 con 0x562e233b90c0 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc1585c1b80 10 client.0 _ll_register_callbacks cb 0x7fc138009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:03.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc13e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e2341beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e23420d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc13e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e2341beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33300/0 (socket says 172.21.15.57:33300) 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc13e59c700 1 -- 172.21.15.57:0/3763792781 learned_addr learned my addr 172.21.15.57:0/3763792781 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.119+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e2341c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc13e59c700 1 -- 172.21.15.57:0/3763792781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 msgr2=0x562e23420d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e23420d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc13e59c700 1 -- 172.21.15.57:0/3763792781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 msgr2=0x562e2341c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e2341c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc13e59c700 1 -- 172.21.15.57:0/3763792781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1380174f0 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc13e59c700 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e2341beb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc130096d20 tx=0x7fc13009a2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d825916bb7bc66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc125ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc125ffb700 1 -- 172.21.15.57:0/3763792781 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300b0030 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc13401e040 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc125ffb700 1 -- 172.21.15.57:0/3763792781 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fc1300b1070 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc125ffb700 1 -- 172.21.15.57:0/3763792781 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1300a2640 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc1585c1b80 10 client.26737 fetch_fsmap learned FSMap version 51 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.120+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x562e231362b0 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc125ffb700 1 -- 172.21.15.57:0/3763792781 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13401e040 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc125ffb700 1 -- 172.21.15.57:0/3763792781 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 51) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fc13009a630 con 0x562e233b90c0 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc1585c1b80 10 client.26737 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T22:22:03.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc1585c1b80 10 client.26737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc1585c1b80 20 client.26737 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e23428dd0 0x562e2342b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc1585c1b80 4 client.26737 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e23305b30 con 0x562e23428dd0 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e23428dd0 0x562e2342b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc13dd9b700 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e23428dd0 0x562e2342b1f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a16dd663de7cf8b server_cookie=e5ed8c8105945e46 in_seq=0 out_seq=0 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.121+0000 7fc125ffb700 10 client.26737 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc125ffb700 1 -- 172.21.15.57:0/3763792781 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562e23305b30 con 0x562e23428dd0 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc125ffb700 10 client.26737 handle_command_reply: tid=0 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 client.26737 shutdown 2023-10-06T22:22:03.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 msgr2=0x562e2341beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e2341beb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc130096d20 tx=0x7fc13009a2b0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 shutdown_connections 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x562e23428dd0 0x562e2342b1f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e233c4890 0x562e23420d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e233b90c0 0x562e2341beb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc125ffb700 0 client.26737 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 --2- 172.21.15.57:0/3763792781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e233b99e0 0x562e2341c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 >> 172.21.15.57:0/3763792781 conn(0x562e2332e260 msgr2=0x562e233cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 shutdown_connections 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 1 -- 172.21.15.57:0/3763792781 wait complete. 2023-10-06T22:22:03.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.122+0000 7fc1585c1b80 20 client.26737 trim_cache size 0 max 16384 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:22:03.131 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:22:03.132 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:22:03.132 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:22:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc1280a15f0 msgr2=0x7fc1280a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc1280a15f0 0x7fc1280a3ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc1380075c0 tx=0x7fc138009040 comp rx=0 tx=0).stop 2023-10-06T22:22:03.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc140102050 msgr2=0x7fc1400ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc140102050 0x7fc1400ffca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc13000e9a0 tx=0x7fc13000ecb0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 shutdown_connections 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc1280a15f0 0x7fc1280a3ab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc14010b470 0x7fc1401a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc140102050 0x7fc1400ffca0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 --2- 172.21.15.57:0/3203213639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc140102970 0x7fc1401001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 >> 172.21.15.57:0/3203213639 conn(0x7fc1400fb890 msgr2=0x7fc14010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 shutdown_connections 2023-10-06T22:22:03.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:03.123+0000 7fc144bff700 1 -- 172.21.15.57:0/3203213639 wait complete. 2023-10-06T22:22:03.143 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:22:03.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:03 smithi156 ceph-mon[134358]: pgmap v224: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 831 KiB/s wr, 74 op/s 2023-10-06T22:22:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:03 smithi190 ceph-mon[134891]: pgmap v224: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 831 KiB/s wr, 74 op/s 2023-10-06T22:22:03.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:03 smithi057 ceph-mon[98492]: pgmap v224: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 831 KiB/s wr, 74 op/s 2023-10-06T22:22:04.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:04 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:04.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:04 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:04.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:04 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:05.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:05 smithi190 ceph-mon[134891]: pgmap v225: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 462 KiB/s wr, 70 op/s 2023-10-06T22:22:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:05 smithi057 ceph-mon[98492]: pgmap v225: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 462 KiB/s wr, 70 op/s 2023-10-06T22:22:05.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:05 smithi156 ceph-mon[134358]: pgmap v225: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 462 KiB/s wr, 70 op/s 2023-10-06T22:22:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.084+0000 7f69637ff700 1 -- 172.21.15.57:0/2797257306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c0753e0 msgr2=0x7f695c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.084+0000 7f69637ff700 1 --2- 172.21.15.57:0/2797257306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c0753e0 0x7f695c0757c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6958009990 tx=0x7f6958009ca0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.084+0000 7f69637ff700 1 -- 172.21.15.57:0/2797257306 shutdown_connections 2023-10-06T22:22:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.085+0000 7f69637ff700 1 --2- 172.21.15.57:0/2797257306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f695c111d50 0x7f695c114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.085+0000 7f69637ff700 1 --2- 172.21.15.57:0/2797257306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c0753e0 0x7f695c0757c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.085+0000 7f69637ff700 1 --2- 172.21.15.57:0/2797257306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 0x7f695c111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.085+0000 7f69637ff700 1 -- 172.21.15.57:0/2797257306 >> 172.21.15.57:0/2797257306 conn(0x7f695c0feb60 msgr2=0x7f695c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.085+0000 7f69637ff700 1 -- 172.21.15.57:0/2797257306 shutdown_connections 2023-10-06T22:22:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.085+0000 7f69637ff700 1 -- 172.21.15.57:0/2797257306 wait complete. 2023-10-06T22:22:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.086+0000 7f69637ff700 1 Processor -- start 2023-10-06T22:22:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.087+0000 7f69637ff700 1 -- start start 2023-10-06T22:22:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.087+0000 7f69637ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f695c0753e0 0x7f695c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.087+0000 7f69637ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 0x7f695c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.087+0000 7f69637ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c111d50 0x7f695c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.087+0000 7f69637ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c19db10 con 0x7f695c111d50 2023-10-06T22:22:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.087+0000 7f69637ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c19dc80 con 0x7f695c0753e0 2023-10-06T22:22:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.087+0000 7f69637ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c19de40 con 0x7f695c075d00 2023-10-06T22:22:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.088+0000 7f6960d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 0x7f695c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.088+0000 7f6960d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 0x7f695c1a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33318/0 (socket says 172.21.15.57:33318) 2023-10-06T22:22:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.088+0000 7f6960d9a700 1 -- 172.21.15.57:0/3769505441 learned_addr learned my addr 172.21.15.57:0/3769505441 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.088+0000 7f696159b700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f695c0753e0 0x7f695c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.088+0000 7f6961d9c700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c111d50 0x7f695c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.089+0000 7f6960d9a700 1 -- 172.21.15.57:0/3769505441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f695c0753e0 msgr2=0x7f695c1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.089+0000 7f6960d9a700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f695c0753e0 0x7f695c1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.089+0000 7f6960d9a700 1 -- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c111d50 msgr2=0x7f695c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.089+0000 7f6960d9a700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c111d50 0x7f695c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.089+0000 7f6960d9a700 1 -- 172.21.15.57:0/3769505441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580095f0 con 0x7f695c075d00 2023-10-06T22:22:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.089+0000 7f6960d9a700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 0x7f695c1a3b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f695400ba50 tx=0x7f695400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.090+0000 7f69527fc700 1 -- 172.21.15.57:0/3769505441 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695400c9b0 con 0x7f695c075d00 2023-10-06T22:22:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.090+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c19e120 con 0x7f695c075d00 2023-10-06T22:22:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.090+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c1aa630 con 0x7f695c075d00 2023-10-06T22:22:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.090+0000 7f69527fc700 1 -- 172.21.15.57:0/3769505441 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6954014440 con 0x7f695c075d00 2023-10-06T22:22:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.090+0000 7f69527fc700 1 -- 172.21.15.57:0/3769505441 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540075d0 con 0x7f695c075d00 2023-10-06T22:22:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.098+0000 7f6977075b80 1 Processor -- start 2023-10-06T22:22:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.098+0000 7f69527fc700 1 -- 172.21.15.57:0/3769505441 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f695400cb10 con 0x7f695c075d00 2023-10-06T22:22:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.098+0000 7f6977075b80 1 -- start start 2023-10-06T22:22:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.099+0000 7f6977075b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7020fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.099+0000 7f6977075b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a702be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.099+0000 7f6977075b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a702e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.099+0000 7f6977075b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6f8a400 con 0x5619a7020c00 2023-10-06T22:22:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.099+0000 7f6977075b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6f7bbc0 con 0x5619a7021520 2023-10-06T22:22:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.099+0000 7f6977075b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6f794e0 con 0x5619a702c3d0 2023-10-06T22:22:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.099+0000 7f696159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a702be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.100+0000 7f696159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a702be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:40514/0 (socket says 172.21.15.57:40514) 2023-10-06T22:22:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.100+0000 7f696159b700 1 -- 172.21.15.57:0/4062903246 learned_addr learned my addr 172.21.15.57:0/4062903246 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.100+0000 7f6961d9c700 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7020fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.100+0000 7f6960d9a700 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a702e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.101+0000 7f6961d9c700 1 -- 172.21.15.57:0/4062903246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 msgr2=0x5619a702e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.101+0000 7f6961d9c700 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a702e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.101+0000 7f6961d9c700 1 -- 172.21.15.57:0/4062903246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 msgr2=0x5619a702be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.101+0000 7f6961d9c700 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a702be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.101+0000 7f6961d9c700 1 -- 172.21.15.57:0/4062903246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619a6d9fd80 con 0x5619a7020c00 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.101+0000 7f69527fc700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f69480a1730 0x7f69480a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.101+0000 7f69527fc700 1 -- 172.21.15.57:0/3769505441 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f6954006e40 con 0x7f695c075d00 2023-10-06T22:22:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.102+0000 7f696159b700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f69480a1730 0x7f69480a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.102+0000 7f6961d9c700 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7020fe0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f695c068d00 tx=0x7f694c00adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dac86e7dd0cc2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.103+0000 7f69477fe700 1 -- 172.21.15.57:0/4062903246 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00fcc0 con 0x5619a7020c00 2023-10-06T22:22:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.103+0000 7f69477fe700 1 -- 172.21.15.57:0/4062903246 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f694c0044e0 con 0x5619a7020c00 2023-10-06T22:22:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.103+0000 7f69477fe700 1 -- 172.21.15.57:0/4062903246 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c005280 con 0x5619a7020c00 2023-10-06T22:22:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.104+0000 7f6977075b80 1 -- 172.21.15.57:0/4062903246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 msgr2=0x5619a7020fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.105+0000 7f6977075b80 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7020fe0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f695c068d00 tx=0x7f694c00adf0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.105+0000 7f696159b700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f69480a1730 0x7f69480a3bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6958005cd0 tx=0x7f6958019040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.105+0000 7f6977075b80 1 -- 172.21.15.57:0/4062903246 shutdown_connections 2023-10-06T22:22:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.105+0000 7f6977075b80 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a702e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.105+0000 7f6977075b80 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7020fe0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.105+0000 7f6977075b80 1 --2- 172.21.15.57:0/4062903246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a702be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.105+0000 7f6977075b80 1 -- 172.21.15.57:0/4062903246 >> 172.21.15.57:0/4062903246 conn(0x5619a6f96200 msgr2=0x5619a701d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.106+0000 7f6977075b80 1 -- 172.21.15.57:0/4062903246 shutdown_connections 2023-10-06T22:22:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.106+0000 7f6977075b80 1 -- 172.21.15.57:0/4062903246 wait complete. 2023-10-06T22:22:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.107+0000 7f6977075b80 1 -- start start 2023-10-06T22:22:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.107+0000 7f6977075b80 1 Processor -- start 2023-10-06T22:22:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.108+0000 7f6977075b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7020c00 0x5619a702be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.108+0000 7f6977075b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7021520 0x5619a70290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.108+0000 7f6977075b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a7029820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.108+0000 7f6977075b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6f74dd0 con 0x5619a7021520 2023-10-06T22:22:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.108+0000 7f6977075b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6f116b0 con 0x5619a7020c00 2023-10-06T22:22:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.108+0000 7f6977075b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6f11350 con 0x5619a702c3d0 2023-10-06T22:22:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.108+0000 7f6977075b80 10 client.0 _ll_register_callbacks cb 0x5619a6d6e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7021520 0x5619a70290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7021520 0x5619a70290e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42870/0 (socket says 172.21.15.57:42870) 2023-10-06T22:22:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 -- 172.21.15.57:0/1471075966 learned_addr learned my addr 172.21.15.57:0/1471075966 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f696159b700 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7020c00 0x5619a702be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 -- 172.21.15.57:0/1471075966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 msgr2=0x5619a7029820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a7029820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 -- 172.21.15.57:0/1471075966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7020c00 msgr2=0x5619a702be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7020c00 0x5619a702be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.109+0000 7f6960d9a700 1 -- 172.21.15.57:0/1471075966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c00a800 con 0x5619a7021520 2023-10-06T22:22:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.110+0000 7f6960d9a700 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7021520 0x5619a70290e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6954096220 tx=0x7f6954099630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602162fedf5083ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.110+0000 7f6944ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:22:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.111+0000 7f6944ff9700 1 -- 172.21.15.57:0/1471075966 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954099eb0 con 0x5619a7021520 2023-10-06T22:22:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.111+0000 7f6944ff9700 1 -- 172.21.15.57:0/1471075966 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f69540997b0 con 0x5619a7021520 2023-10-06T22:22:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.111+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619a7014d50 con 0x5619a7021520 2023-10-06T22:22:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.112+0000 7f6944ff9700 1 -- 172.21.15.57:0/1471075966 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f695409d900 con 0x5619a7021520 2023-10-06T22:22:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.113+0000 7f6977075b80 10 client.16701 fetch_fsmap learned FSMap version 53 2023-10-06T22:22:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.113+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5619a6d88c00 con 0x5619a7021520 2023-10-06T22:22:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6944ff9700 1 -- 172.21.15.57:0/1471075966 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619a7014d50 con 0x5619a7021520 2023-10-06T22:22:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6944ff9700 1 -- 172.21.15.57:0/1471075966 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 53) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f6954098ca0 con 0x5619a7021520 2023-10-06T22:22:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6977075b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T22:22:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6977075b80 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6977075b80 20 client.16701 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6977075b80 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a703c050 0x5619a703e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6977075b80 4 client.16701 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:22:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.114+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619a6f6ddb0 con 0x5619a703c050 2023-10-06T22:22:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.115+0000 7f696159b700 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a703c050 0x5619a703e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.116+0000 7f696159b700 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a703c050 0x5619a703e470 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a38367cb821bd83d server_cookie=12a81583e79918f2 in_seq=0 out_seq=0 2023-10-06T22:22:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.116+0000 7f6944ff9700 10 client.16701 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.118+0000 7f6944ff9700 1 -- 172.21.15.57:0/1471075966 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5619a6f6ddb0 con 0x5619a703c050 2023-10-06T22:22:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.118+0000 7f6944ff9700 10 client.16701 handle_command_reply: tid=0 2023-10-06T22:22:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.118+0000 7f6977075b80 1 client.16701 shutdown 2023-10-06T22:22:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7021520 msgr2=0x5619a70290e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7021520 0x5619a70290e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6954096220 tx=0x7f6954099630 comp rx=0 tx=0).stop 2023-10-06T22:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 shutdown_connections 2023-10-06T22:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a703c050 0x5619a703e470 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a7029820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7020c00 0x5619a702be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 --2- 172.21.15.57:0/1471075966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7021520 0x5619a70290e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 >> 172.21.15.57:0/1471075966 conn(0x5619a6f96200 msgr2=0x5619a701cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.119+0000 7f6944ff9700 0 client.16701 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.120+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 shutdown_connections 2023-10-06T22:22:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.120+0000 7f6977075b80 1 -- 172.21.15.57:0/1471075966 wait complete. 2023-10-06T22:22:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.121+0000 7f6977075b80 20 client.16701 trim_cache size 0 max 16384 2023-10-06T22:22:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.151+0000 7f6977075b80 1 Processor -- start 2023-10-06T22:22:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.152+0000 7f6977075b80 1 -- start start 2023-10-06T22:22:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.152+0000 7f6977075b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a702e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.152+0000 7f6977075b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a7021520 0x5619a7028670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.152+0000 7f6977075b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a702c3d0 0x5619a702b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.152+0000 7f6977075b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6df58c0 con 0x5619a7020c00 2023-10-06T22:22:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.152+0000 7f6977075b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a7024f70 con 0x5619a702c3d0 2023-10-06T22:22:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.152+0000 7f6977075b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a7021b60 con 0x5619a7021520 2023-10-06T22:22:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.153+0000 7f696159b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a702e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.153+0000 7f696159b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a702e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42878/0 (socket says 172.21.15.57:42878) 2023-10-06T22:22:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.153+0000 7f696159b700 1 -- 172.21.15.57:0/2084169073 learned_addr learned my addr 172.21.15.57:0/2084169073 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f696159b700 1 -- 172.21.15.57:0/2084169073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a7021520 msgr2=0x5619a7028670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f696159b700 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a7021520 0x5619a7028670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f6961d9c700 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a702c3d0 0x5619a702b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f696159b700 1 -- 172.21.15.57:0/2084169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a702c3d0 msgr2=0x5619a702b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f696159b700 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a702c3d0 0x5619a702b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f696159b700 1 -- 172.21.15.57:0/2084169073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619a6dc61f0 con 0x5619a7020c00 2023-10-06T22:22:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f6961d9c700 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a702c3d0 0x5619a702b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:22:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.154+0000 7f696159b700 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a702e7c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6958006010 tx=0x7f6958020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ba4d74657daf47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.155+0000 7f6947fff700 1 -- 172.21.15.57:0/2084169073 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958003680 con 0x5619a7020c00 2023-10-06T22:22:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.155+0000 7f6947fff700 1 -- 172.21.15.57:0/2084169073 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f695802b040 con 0x5619a7020c00 2023-10-06T22:22:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.155+0000 7f6947fff700 1 -- 172.21.15.57:0/2084169073 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580054c0 con 0x5619a7020c00 2023-10-06T22:22:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.156+0000 7f6977075b80 1 -- 172.21.15.57:0/2084169073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 msgr2=0x5619a702e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.156+0000 7f6977075b80 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a702e7c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6958006010 tx=0x7f6958020000 comp rx=0 tx=0).stop 2023-10-06T22:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.156+0000 7f6977075b80 1 -- 172.21.15.57:0/2084169073 shutdown_connections 2023-10-06T22:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.156+0000 7f6977075b80 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a702c3d0 0x5619a702b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.156+0000 7f6977075b80 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a702e7c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.156+0000 7f6977075b80 1 --2- 172.21.15.57:0/2084169073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a7021520 0x5619a7028670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.156+0000 7f6977075b80 1 -- 172.21.15.57:0/2084169073 >> 172.21.15.57:0/2084169073 conn(0x5619a6f96200 msgr2=0x5619a701b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.157+0000 7f6977075b80 1 -- 172.21.15.57:0/2084169073 shutdown_connections 2023-10-06T22:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.157+0000 7f6977075b80 1 -- 172.21.15.57:0/2084169073 wait complete. 2023-10-06T22:22:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.157+0000 7f6977075b80 1 -- start start 2023-10-06T22:22:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 1 Processor -- start 2023-10-06T22:22:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7032f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a7087e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a7088370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a708bc10 con 0x5619a7020c00 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a70888b0 con 0x5619a7021520 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a7088a70 con 0x5619a702c3d0 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.158+0000 7f6977075b80 10 client.0 _ll_register_callbacks cb 0x7f6954009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.159+0000 7f696159b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7032f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.159+0000 7f696159b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7032f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42882/0 (socket says 172.21.15.57:42882) 2023-10-06T22:22:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.159+0000 7f696159b700 1 -- 172.21.15.57:0/3535343836 learned_addr learned my addr 172.21.15.57:0/3535343836 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.160+0000 7f696159b700 1 -- 172.21.15.57:0/3535343836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 msgr2=0x5619a7088370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.160+0000 7f696159b700 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a7088370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.160+0000 7f696159b700 1 -- 172.21.15.57:0/3535343836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 msgr2=0x5619a7087e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.160+0000 7f696159b700 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a7087e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.160+0000 7f696159b700 1 -- 172.21.15.57:0/3535343836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c114c30 con 0x5619a7020c00 2023-10-06T22:22:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.161+0000 7f696159b700 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7032f80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6958009350 tx=0x7f6958034da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530152ae2bb24c09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.161+0000 7f6944ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:22:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.161+0000 7f6944ff9700 1 -- 172.21.15.57:0/3535343836 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958028070 con 0x5619a7020c00 2023-10-06T22:22:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.162+0000 7f6944ff9700 1 -- 172.21.15.57:0/3535343836 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f6958032240 con 0x5619a7020c00 2023-10-06T22:22:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.162+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f694c0044e0 con 0x5619a7020c00 2023-10-06T22:22:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.162+0000 7f6944ff9700 1 -- 172.21.15.57:0/3535343836 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6958032400 con 0x5619a7020c00 2023-10-06T22:22:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.162+0000 7f6977075b80 10 client.16707 fetch_fsmap learned FSMap version 53 2023-10-06T22:22:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.162+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5619a6d9e250 con 0x5619a7020c00 2023-10-06T22:22:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.163+0000 7f6944ff9700 1 -- 172.21.15.57:0/3535343836 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f694c0044e0 con 0x5619a7020c00 2023-10-06T22:22:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.163+0000 7f6944ff9700 1 -- 172.21.15.57:0/3535343836 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 53) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f695803a070 con 0x5619a7020c00 2023-10-06T22:22:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.164+0000 7f6977075b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T22:22:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.164+0000 7f6977075b80 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.164+0000 7f6977075b80 20 client.16707 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.164+0000 7f6977075b80 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a70b2c40 0x5619a70b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.164+0000 7f6977075b80 4 client.16707 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:22:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.164+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619a6f6dad0 con 0x5619a70b2c40 2023-10-06T22:22:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.166+0000 7f6960d9a700 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a70b2c40 0x5619a70b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.169+0000 7f6960d9a700 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a70b2c40 0x5619a70b5060 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92b939f13267836 server_cookie=debf98ed6ea73220 in_seq=0 out_seq=0 2023-10-06T22:22:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.170+0000 7f6944ff9700 10 client.16707 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.490+0000 7f6944ff9700 1 -- 172.21.15.57:0/3535343836 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619a6f6dad0 con 0x5619a70b2c40 2023-10-06T22:22:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.490+0000 7f6944ff9700 10 client.16707 handle_command_reply: tid=0 2023-10-06T22:22:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.490+0000 7f6977075b80 1 client.16707 shutdown 2023-10-06T22:22:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.491+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 msgr2=0x5619a7032f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.491+0000 7f6977075b80 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7032f80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6958009350 tx=0x7f6958034da0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 shutdown_connections 2023-10-06T22:22:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5619a70b2c40 0x5619a70b5060 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619a702c3d0 0x5619a7088370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5619a7020c00 0x5619a7032f80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 --2- 172.21.15.57:0/3535343836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619a7021520 0x5619a7087e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 >> 172.21.15.57:0/3535343836 conn(0x5619a6f96200 msgr2=0x5619a7024430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6944ff9700 0 client.16707 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 shutdown_connections 2023-10-06T22:22:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.493+0000 7f6977075b80 1 -- 172.21.15.57:0/3535343836 wait complete. 2023-10-06T22:22:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.494+0000 7f6977075b80 20 client.16707 trim_cache size 0 max 16384 2023-10-06T22:22:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:22:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:22:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:22:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:22:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f69480a1730 msgr2=0x7f69480a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f69480a1730 0x7f69480a3bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6958005cd0 tx=0x7f6958019040 comp rx=0 tx=0).stop 2023-10-06T22:22:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 msgr2=0x7f695c1a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 0x7f695c1a3b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f695400ba50 tx=0x7f695400bd60 comp rx=0 tx=0).stop 2023-10-06T22:22:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 shutdown_connections 2023-10-06T22:22:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f69480a1730 0x7f69480a3bf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f695c111d50 0x7f695c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f695c0753e0 0x7f695c1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 --2- 172.21.15.57:0/3769505441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f695c075d00 0x7f695c1a3b80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.497+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 >> 172.21.15.57:0/3769505441 conn(0x7f695c0feb60 msgr2=0x7f695c100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.499+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 shutdown_connections 2023-10-06T22:22:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:06.499+0000 7f69637ff700 1 -- 172.21.15.57:0/3769505441 wait complete. 2023-10-06T22:22:06.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:06 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:22:06.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:06 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:22:06.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:06 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:06.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:06 smithi190 ceph-mon[134891]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T22:22:06.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:06 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:22:06.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:06 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:22:06.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:06 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:06.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:06 smithi057 ceph-mon[98492]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T22:22:06.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:06 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:22:06.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:06 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:22:06.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:06 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:06.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:06 smithi156 ceph-mon[134358]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T22:22:07.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:07 smithi190 ceph-mon[134891]: pgmap v227: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 462 KiB/s wr, 78 op/s 2023-10-06T22:22:07.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:07 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:22:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:07 smithi057 ceph-mon[98492]: pgmap v227: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 462 KiB/s wr, 78 op/s 2023-10-06T22:22:07.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:07 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:22:07.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:07 smithi156 ceph-mon[134358]: pgmap v227: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 462 KiB/s wr, 78 op/s 2023-10-06T22:22:07.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:07 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:22:09.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:09 smithi190 ceph-mon[134891]: pgmap v228: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-10-06T22:22:09.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:09 smithi057 ceph-mon[98492]: pgmap v228: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-10-06T22:22:09.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:09 smithi156 ceph-mon[134358]: pgmap v228: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-10-06T22:22:10.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:10 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:10.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:10 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:10.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:10 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:11.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:11 smithi190 ceph-mon[134891]: pgmap v229: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 229 op/s 2023-10-06T22:22:11.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:11 smithi057 ceph-mon[98492]: pgmap v229: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 229 op/s 2023-10-06T22:22:11.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:11 smithi156 ceph-mon[134358]: pgmap v229: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 229 op/s 2023-10-06T22:22:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:13 smithi057 ceph-mon[98492]: pgmap v230: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1.0 MiB/s wr, 233 op/s 2023-10-06T22:22:13.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:13 smithi156 ceph-mon[134358]: pgmap v230: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1.0 MiB/s wr, 233 op/s 2023-10-06T22:22:13.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:13 smithi190 ceph-mon[134891]: pgmap v230: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1.0 MiB/s wr, 233 op/s 2023-10-06T22:22:14.991 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-06T22:22:14.991 INFO:tasks.workunit.client.0.smithi057.stdout:Final score for writes: 344 2023-10-06T22:22:14.991 INFO:tasks.workunit.client.0.smithi057.stdout:Final score for reads : 393471 2023-10-06T22:22:14.991 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-06T22:22:14.992 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T22:22:14.993 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T22:22:15.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:15 smithi057 ceph-mon[98492]: pgmap v231: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 999 KiB/s wr, 218 op/s 2023-10-06T22:22:15.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:15 smithi156 ceph-mon[134358]: pgmap v231: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 999 KiB/s wr, 218 op/s 2023-10-06T22:22:15.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:15 smithi190 ceph-mon[134891]: pgmap v231: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 999 KiB/s wr, 218 op/s 2023-10-06T22:22:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:17 smithi190 ceph-mon[134891]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T22:22:17.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:17 smithi190 ceph-mon[134891]: pgmap v233: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 981 KiB/s wr, 216 op/s 2023-10-06T22:22:17.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:17 smithi057 ceph-mon[98492]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T22:22:17.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:17 smithi057 ceph-mon[98492]: pgmap v233: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 981 KiB/s wr, 216 op/s 2023-10-06T22:22:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:17 smithi156 ceph-mon[134358]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T22:22:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:17 smithi156 ceph-mon[134358]: pgmap v233: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 981 KiB/s wr, 216 op/s 2023-10-06T22:22:19.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:19 smithi156 ceph-mon[134358]: pgmap v234: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 543 KiB/s wr, 106 op/s 2023-10-06T22:22:19.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:19 smithi190 ceph-mon[134891]: pgmap v234: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 543 KiB/s wr, 106 op/s 2023-10-06T22:22:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:19 smithi057 ceph-mon[98492]: pgmap v234: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 543 KiB/s wr, 106 op/s 2023-10-06T22:22:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 -- 172.21.15.57:0/237743032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec0101a80 msgr2=0x7f5ec0105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/237743032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec0101a80 0x7f5ec0105ac0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009990 tx=0x7f5ea8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 -- 172.21.15.57:0/237743032 shutdown_connections 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/237743032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec01061f0 0x7f5ec010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/237743032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec01010d0 0x7f5ec01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/237743032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec0101a80 0x7f5ec0105ac0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 -- 172.21.15.57:0/237743032 >> 172.21.15.57:0/237743032 conn(0x7f5ec00fc930 msgr2=0x7f5ec00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 -- 172.21.15.57:0/237743032 shutdown_connections 2023-10-06T22:22:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.714+0000 7f5ec49ff700 1 -- 172.21.15.57:0/237743032 wait complete. 2023-10-06T22:22:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.715+0000 7f5ec49ff700 1 Processor -- start 2023-10-06T22:22:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ec49ff700 1 -- start start 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ec49ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec01010d0 0x7f5ec01991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ec49ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec01061f0 0x7f5ec0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ec49ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec019dd80 0x7f5ec019e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ec49ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec019e800 con 0x7f5ec01061f0 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ec49ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec019e970 con 0x7f5ec019dd80 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ec49ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec019eb30 con 0x7f5ec01010d0 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec01010d0 0x7f5ec01991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ebed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec019dd80 0x7f5ec019e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec01010d0 0x7f5ec01991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46114/0 (socket says 172.21.15.57:46114) 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.716+0000 7f5ebe59c700 1 -- 172.21.15.57:0/3528878894 learned_addr learned my addr 172.21.15.57:0/3528878894 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec01061f0 0x7f5ec0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ebe59c700 1 -- 172.21.15.57:0/3528878894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec019dd80 msgr2=0x7f5ec019e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec019dd80 0x7f5ec019e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ebe59c700 1 -- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec01061f0 msgr2=0x7f5ec0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec01061f0 0x7f5ec0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ebe59c700 1 -- 172.21.15.57:0/3528878894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009520 con 0x7f5ec01010d0 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec01010d0 0x7f5ec01991d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000eb30 tx=0x7f5eb000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5eb77fe700 1 -- 172.21.15.57:0/3528878894 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0009880 con 0x7f5ec01010d0 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5eb77fe700 1 -- 172.21.15.57:0/3528878894 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb00044e0 con 0x7f5ec01010d0 2023-10-06T22:22:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea80095f0 con 0x7f5ec01010d0 2023-10-06T22:22:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5eb77fe700 1 -- 172.21.15.57:0/3528878894 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003d80 con 0x7f5ec01010d0 2023-10-06T22:22:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.717+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec01a6430 con 0x7f5ec01010d0 2023-10-06T22:22:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.721+0000 7f5eb77fe700 1 -- 172.21.15.57:0/3528878894 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5eb0003f20 con 0x7f5ec01010d0 2023-10-06T22:22:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.722+0000 7f5ed82a4b80 1 Processor -- start 2023-10-06T22:22:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.722+0000 7f5ed82a4b80 1 -- start start 2023-10-06T22:22:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.722+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169757c00 0x563169757fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169762e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631696c1400 con 0x5631697633d0 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631696b2bc0 con 0x563169757c00 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631696b04e0 con 0x563169758520 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb77fe700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5eac0a1670 0x7f5eac0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ebed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169757c00 0x563169757fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ebed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169757c00 0x563169757fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:37644/0 (socket says 172.21.15.57:37644) 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ebed9d700 1 -- 172.21.15.57:0/1418089784 learned_addr learned my addr 172.21.15.57:0/1418089784 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169762e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb77fe700 1 -- 172.21.15.57:0/3528878894 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f5eb0014070 con 0x7f5ec01010d0 2023-10-06T22:22:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ebed9d700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5eac0a1670 0x7f5eac0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb7fff700 1 -- 172.21.15.57:0/1418089784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 msgr2=0x563169762e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169762e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb7fff700 1 -- 172.21.15.57:0/1418089784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169757c00 msgr2=0x563169757fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169757c00 0x563169757fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5eb7fff700 1 -- 172.21.15.57:0/1418089784 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631694d6d80 con 0x5631697633d0 2023-10-06T22:22:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.723+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169762e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.724+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697657c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8000c00 tx=0x7f5ea8005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa302018f291077 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.724+0000 7f5ebed9d700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5eac0a1670 0x7f5eac0a3b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800fb80 tx=0x7f5eb8004b40 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.724+0000 7f5eb4ff9700 1 -- 172.21.15.57:0/1418089784 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea801d070 con 0x5631697633d0 2023-10-06T22:22:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.724+0000 7f5eb4ff9700 1 -- 172.21.15.57:0/1418089784 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea801e040 con 0x5631697633d0 2023-10-06T22:22:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.724+0000 7f5eb4ff9700 1 -- 172.21.15.57:0/1418089784 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004760 con 0x5631697633d0 2023-10-06T22:22:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1418089784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 msgr2=0x5631697657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697657c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8000c00 tx=0x7f5ea8005ea0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1418089784 shutdown_connections 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697657c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169757c00 0x563169757fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1418089784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169762e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1418089784 >> 172.21.15.57:0/1418089784 conn(0x5631696cd200 msgr2=0x563169754080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1418089784 shutdown_connections 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1418089784 wait complete. 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.725+0000 7f5ed82a4b80 1 -- start start 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 1 Processor -- start 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x563169762e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169758520 0x5631697600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631696abdd0 con 0x5631697633d0 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631696486b0 con 0x563169758520 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563169648350 con 0x563169757c00 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.726+0000 7f5ed82a4b80 10 client.0 _ll_register_callbacks cb 0x5631694a57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169758520 0x5631697600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x563169762e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x563169762e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46140/0 (socket says 172.21.15.57:46140) 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169758520 0x5631697600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:37660/0 (socket says 172.21.15.57:37660) 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 -- 172.21.15.57:0/918041412 learned_addr learned my addr 172.21.15.57:0/918041412 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebed9d700 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 -- 172.21.15.57:0/918041412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169758520 msgr2=0x5631697600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169758520 0x5631697600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 -- 172.21.15.57:0/918041412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 msgr2=0x5631697607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 -- 172.21.15.57:0/918041412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec01a6430 con 0x563169757c00 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x563169762e40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000eb00 tx=0x7f5eb006a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a755ad05b6508cec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ea5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/918041412 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0099f30 con 0x563169757c00 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/918041412 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f5eb0065e40 con 0x563169757c00 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.727+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56316974bd50 con 0x563169757c00 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/918041412 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb0068070 con 0x563169757c00 2023-10-06T22:22:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 10 client.26755 fetch_fsmap learned FSMap version 54 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f5ec010e5e0 con 0x563169757c00 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/918041412 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56316974bd50 con 0x563169757c00 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/918041412 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 54) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f5eb0096ce0 con 0x563169757c00 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 10 client.26755 fetch_fsmap finished waiting for FSMap version 54 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 10 client.26755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 20 client.26755 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563169772f30 0x563169775350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 4 client.26755 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631696a4db0 con 0x563169772f30 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.728+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563169772f30 0x563169775350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.729+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563169772f30 0x563169775350 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e2f261a3a4a6e7 server_cookie=3ab7d29609e660e3 in_seq=0 out_seq=0 2023-10-06T22:22:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.729+0000 7f5ea5ffb700 10 client.26755 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/918041412 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5631696a4db0 con 0x563169772f30 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ea5ffb700 10 client.26755 handle_command_reply: tid=0 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 client.26755 shutdown 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 msgr2=0x563169762e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x563169762e40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000eb00 tx=0x7f5eb006a560 comp rx=0 tx=0).stop 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 shutdown_connections 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563169772f30 0x563169775350 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5631697633d0 0x5631697607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x563169762e40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ea5ffb700 0 client.26755 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/918041412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563169758520 0x5631697600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 >> 172.21.15.57:0/918041412 conn(0x5631696cd200 msgr2=0x563169753c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 shutdown_connections 2023-10-06T22:22:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.730+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/918041412 wait complete. 2023-10-06T22:22:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.731+0000 7f5ed82a4b80 20 client.26755 trim_cache size 0 max 16384 2023-10-06T22:22:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 Processor -- start 2023-10-06T22:22:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 -- start start 2023-10-06T22:22:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x5631697629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169758520 0x56316975b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x56316975b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316952c8c0 con 0x563169758520 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563169760db0 con 0x5631697633d0 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.746+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563169754dc0 con 0x563169757c00 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x5631697629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x5631697629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46154/0 (socket says 172.21.15.57:46154) 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 -- 172.21.15.57:0/1365428686 learned_addr learned my addr 172.21.15.57:0/1365428686 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebed9d700 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x56316975b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169758520 0x56316975b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 -- 172.21.15.57:0/1365428686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 msgr2=0x56316975b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x56316975b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 -- 172.21.15.57:0/1365428686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169758520 msgr2=0x56316975b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169758520 0x56316975b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 -- 172.21.15.57:0/1365428686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631694bfc00 con 0x563169757c00 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169758520 0x56316975b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x5631697629e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5eb009a2d0 tx=0x7f5eb0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=762f81847903ff46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5eb57fa700 1 -- 172.21.15.57:0/1365428686 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0068030 con 0x563169757c00 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5eb57fa700 1 -- 172.21.15.57:0/1365428686 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb009a3a0 con 0x563169757c00 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.747+0000 7f5eb57fa700 1 -- 172.21.15.57:0/1365428686 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0098970 con 0x563169757c00 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1365428686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 msgr2=0x5631697629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x5631697629e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5eb009a2d0 tx=0x7f5eb0003680 comp rx=0 tx=0).stop 2023-10-06T22:22:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1365428686 shutdown_connections 2023-10-06T22:22:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x56316975b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169757c00 0x5631697629e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/1365428686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169758520 0x56316975b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1365428686 >> 172.21.15.57:0/1365428686 conn(0x5631696cd200 msgr2=0x5631697527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1365428686 shutdown_connections 2023-10-06T22:22:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.748+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/1365428686 wait complete. 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 -- start start 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 Processor -- start 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169757c00 0x563169767ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169768420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x5631697c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563169768d30 con 0x563169757c00 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631697c6280 con 0x5631697633d0 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.749+0000 7f5ed82a4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631697c63c0 con 0x563169758520 2023-10-06T22:22:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5ed82a4b80 10 client.0 _ll_register_callbacks cb 0x7f5eb0096970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169768420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169757c00 0x563169767ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169768420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46166/0 (socket says 172.21.15.57:46166) 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 -- 172.21.15.57:0/4144930345 learned_addr learned my addr 172.21.15.57:0/4144930345 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5ebed9d700 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x5631697c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 -- 172.21.15.57:0/4144930345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 msgr2=0x5631697c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x5631697c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 -- 172.21.15.57:0/4144930345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169757c00 msgr2=0x563169767ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169757c00 0x563169767ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 -- 172.21.15.57:0/4144930345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec010e5e0 con 0x563169758520 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169757c00 0x563169767ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5eb7fff700 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169768420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009ce0 tx=0x7f5ea800b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f5310ee8fafa10f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.750+0000 7f5ea5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/4144930345 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8032040 con 0x563169758520 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea801e040 con 0x563169758520 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/4144930345 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f5ea80053a0 con 0x563169758520 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/4144930345 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea801d040 con 0x563169758520 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 10 client.26761 fetch_fsmap learned FSMap version 54 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5631694fd1f0 con 0x563169758520 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/4144930345 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea801e040 con 0x563169758520 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/4144930345 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 54) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f5ea8034be0 con 0x563169758520 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 10 client.26761 fetch_fsmap finished waiting for FSMap version 54 2023-10-06T22:22:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 10 client.26761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 20 client.26761 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5631697c7860 0x5631697c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 4 client.26761 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:22:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.751+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631696a4ad0 con 0x5631697c7860 2023-10-06T22:22:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.752+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5631697c7860 0x5631697c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.752+0000 7f5ea5ffb700 10 client.26761 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.752+0000 7f5ebe59c700 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5631697c7860 0x5631697c9c80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b074e003d06d8f59 server_cookie=dd9e160374f99b53 in_seq=0 out_seq=0 2023-10-06T22:22:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.932+0000 7f5ea5ffb700 1 -- 172.21.15.57:0/4144930345 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631696a4ad0 con 0x5631697c7860 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.932+0000 7f5ea5ffb700 10 client.26761 handle_command_reply: tid=0 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.932+0000 7f5ed82a4b80 1 client.26761 shutdown 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 msgr2=0x563169768420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169768420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009ce0 tx=0x7f5ea800b5b0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 shutdown_connections 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5631697c7860 0x5631697c9c80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631697633d0 0x5631697c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563169757c00 0x563169767ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 --2- 172.21.15.57:0/4144930345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563169758520 0x563169768420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ea5ffb700 0 client.26761 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 >> 172.21.15.57:0/4144930345 conn(0x5631696cd200 msgr2=0x56316975b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 shutdown_connections 2023-10-06T22:22:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 1 -- 172.21.15.57:0/4144930345 wait complete. 2023-10-06T22:22:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.933+0000 7f5ed82a4b80 20 client.26761 trim_cache size 0 max 16384 2023-10-06T22:22:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:22:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:22:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:22:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:22:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5eac0a1670 msgr2=0x7f5eac0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5eac0a1670 0x7f5eac0a3b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800fb80 tx=0x7f5eb8004b40 comp rx=0 tx=0).stop 2023-10-06T22:22:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec01010d0 msgr2=0x7f5ec01991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec01010d0 0x7f5ec01991d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000eb30 tx=0x7f5eb000ee40 comp rx=0 tx=0).stop 2023-10-06T22:22:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 shutdown_connections 2023-10-06T22:22:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5eac0a1670 0x7f5eac0a3b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ec019dd80 0x7f5ec019e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec01010d0 0x7f5ec01991d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 --2- 172.21.15.57:0/3528878894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec01061f0 0x7f5ec0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 >> 172.21.15.57:0/3528878894 conn(0x7f5ec00fc930 msgr2=0x7f5ec00fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 shutdown_connections 2023-10-06T22:22:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:19.934+0000 7f5ec49ff700 1 -- 172.21.15.57:0/3528878894 wait complete. 2023-10-06T22:22:21.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:21 smithi156 ceph-mon[134358]: pgmap v235: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 530 KiB/s wr, 86 op/s 2023-10-06T22:22:21.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:21 smithi190 ceph-mon[134891]: pgmap v235: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 530 KiB/s wr, 86 op/s 2023-10-06T22:22:21.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:21 smithi057 ceph-mon[98492]: pgmap v235: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 530 KiB/s wr, 86 op/s 2023-10-06T22:22:23.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:23 smithi156 ceph-mon[134358]: pgmap v236: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-10-06T22:22:23.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:23 smithi190 ceph-mon[134891]: pgmap v236: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-10-06T22:22:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:23 smithi057 ceph-mon[98492]: pgmap v236: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-10-06T22:22:25.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:25 smithi156 ceph-mon[134358]: pgmap v237: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 277 op/s 2023-10-06T22:22:25.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:25 smithi190 ceph-mon[134891]: pgmap v237: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 277 op/s 2023-10-06T22:22:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:25 smithi057 ceph-mon[98492]: pgmap v237: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 277 op/s 2023-10-06T22:22:27.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:27 smithi190 ceph-mon[134891]: pgmap v238: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 333 op/s 2023-10-06T22:22:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:27 smithi057 ceph-mon[98492]: pgmap v238: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 333 op/s 2023-10-06T22:22:27.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:27 smithi156 ceph-mon[134358]: pgmap v238: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 333 op/s 2023-10-06T22:22:29.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:29 smithi156 ceph-mon[134358]: pgmap v239: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 368 op/s 2023-10-06T22:22:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:29 smithi190 ceph-mon[134891]: pgmap v239: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 368 op/s 2023-10-06T22:22:29.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:29 smithi057 ceph-mon[98492]: pgmap v239: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 368 op/s 2023-10-06T22:22:30.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:30 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:30.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:30 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:30.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:30 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:31.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:31 smithi190 ceph-mon[134891]: pgmap v240: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 383 op/s 2023-10-06T22:22:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:31 smithi057 ceph-mon[98492]: pgmap v240: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 383 op/s 2023-10-06T22:22:31.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:31 smithi156 ceph-mon[134358]: pgmap v240: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 383 op/s 2023-10-06T22:22:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 -- 172.21.15.57:0/795905561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 msgr2=0x7f427810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 --2- 172.21.15.57:0/795905561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 0x7f427810d770 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f427000b0a0 tx=0x7f427000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 -- 172.21.15.57:0/795905561 shutdown_connections 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 --2- 172.21.15.57:0/795905561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 0x7f427810d770 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 --2- 172.21.15.57:0/795905561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4278101e90 0x7f4278102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 --2- 172.21.15.57:0/795905561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 0x7f427810ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 -- 172.21.15.57:0/795905561 >> 172.21.15.57:0/795905561 conn(0x7f42780fb690 msgr2=0x7f42780fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 -- 172.21.15.57:0/795905561 shutdown_connections 2023-10-06T22:22:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.085+0000 7f427db9d700 1 -- 172.21.15.57:0/795905561 wait complete. 2023-10-06T22:22:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.086+0000 7f427db9d700 1 Processor -- start 2023-10-06T22:22:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.086+0000 7f427db9d700 1 -- start start 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.086+0000 7f427db9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4278101e90 0x7f4278199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427db9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 0x7f4278199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427db9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 0x7f427819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427db9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819e110 con 0x7f4278101e90 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427db9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819e280 con 0x7f42781027b0 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427db9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819e440 con 0x7f427810b2b0 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f4277fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 0x7f4278199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427d39c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 0x7f427819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f4277fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 0x7f4278199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57450/0 (socket says 172.21.15.57:57450) 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f4277fff700 1 -- 172.21.15.57:0/2267261212 learned_addr learned my addr 172.21.15.57:0/2267261212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427cb9b700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4278101e90 0x7f4278199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427d39c700 1 -- 172.21.15.57:0/2267261212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 msgr2=0x7f4278199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427d39c700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 0x7f4278199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427d39c700 1 -- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4278101e90 msgr2=0x7f4278199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427d39c700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4278101e90 0x7f4278199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f427d39c700 1 -- 172.21.15.57:0/2267261212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c009520 con 0x7f427810b2b0 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.087+0000 7f4277fff700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 0x7f4278199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.088+0000 7f427d39c700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 0x7f427819dad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4270009fd0 tx=0x7f4270007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.088+0000 7f4275ffb700 1 -- 172.21.15.57:0/2267261212 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000e040 con 0x7f427810b2b0 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.088+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4264009520 con 0x7f427810b2b0 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.088+0000 7f4275ffb700 1 -- 172.21.15.57:0/2267261212 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f427001d470 con 0x7f427810b2b0 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.088+0000 7f4275ffb700 1 -- 172.21.15.57:0/2267261212 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42700052a0 con 0x7f427810b2b0 2023-10-06T22:22:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.088+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4270009d00 con 0x7f427810b2b0 2023-10-06T22:22:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.091+0000 7f4275ffb700 1 -- 172.21.15.57:0/2267261212 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f427001d5e0 con 0x7f427810b2b0 2023-10-06T22:22:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.093+0000 7f4292806b80 1 Processor -- start 2023-10-06T22:22:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.093+0000 7f4292806b80 1 -- start start 2023-10-06T22:22:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.093+0000 7f4292806b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856ed60 0x5565b856f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.093+0000 7f4292806b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856f680 0x5565b8579ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.093+0000 7f4292806b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b857c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4292806b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b84d8400 con 0x5565b857a530 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4292806b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b84c9bc0 con 0x5565b856f680 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4292806b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b84c74e0 con 0x5565b856ed60 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856f680 0x5565b8579ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4275ffb700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f42680a15f0 0x7f42680a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856f680 0x5565b8579ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57452/0 (socket says 172.21.15.57:57452) 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 -- 172.21.15.57:0/1268113086 learned_addr learned my addr 172.21.15.57:0/1268113086 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4277fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856ed60 0x5565b856f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4277fff700 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b857c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4275ffb700 1 -- 172.21.15.57:0/2267261212 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f42700ce3e0 con 0x7f427810b2b0 2023-10-06T22:22:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 -- 172.21.15.57:0/1268113086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856ed60 msgr2=0x5565b856f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856ed60 0x5565b856f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 -- 172.21.15.57:0/1268113086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 msgr2=0x5565b857c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b857c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427d39c700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f42680a15f0 0x7f42680a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f427cb9b700 1 -- 172.21.15.57:0/1268113086 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b82edd80 con 0x5565b856f680 2023-10-06T22:22:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.094+0000 7f4277fff700 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b857c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.095+0000 7f427cb9b700 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856f680 0x5565b8579ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f426c009f10 tx=0x7f426c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc621f792ab2fb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.095+0000 7f427d39c700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f42680a15f0 0x7f42680a3ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42700a31e0 tx=0x7f427000b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.095+0000 7f4262ffd700 1 -- 172.21.15.57:0/1268113086 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c019630 con 0x5565b856f680 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.095+0000 7f4262ffd700 1 -- 172.21.15.57:0/1268113086 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426c019c50 con 0x5565b856f680 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.095+0000 7f4262ffd700 1 -- 172.21.15.57:0/1268113086 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00ece0 con 0x5565b856f680 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.095+0000 7f4292806b80 1 -- 172.21.15.57:0/1268113086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856f680 msgr2=0x5565b8579ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.095+0000 7f4292806b80 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856f680 0x5565b8579ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f426c009f10 tx=0x7f426c011720 comp rx=0 tx=0).stop 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 -- 172.21.15.57:0/1268113086 shutdown_connections 2023-10-06T22:22:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b857c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856ed60 0x5565b856f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 --2- 172.21.15.57:0/1268113086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856f680 0x5565b8579ff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 -- 172.21.15.57:0/1268113086 >> 172.21.15.57:0/1268113086 conn(0x5565b84e4200 msgr2=0x5565b856b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 -- 172.21.15.57:0/1268113086 shutdown_connections 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 -- 172.21.15.57:0/1268113086 wait complete. 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 -- start start 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.096+0000 7f4292806b80 1 Processor -- start 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4292806b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856ed60 0x5565b8579fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4292806b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8577260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4292806b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b85779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4292806b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b84c2dd0 con 0x5565b857a530 2023-10-06T22:22:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4292806b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b845f6b0 con 0x5565b856ed60 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4292806b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b845f350 con 0x5565b856f680 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4292806b80 10 client.0 _ll_register_callbacks cb 0x5565b82bc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8577260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8577260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47854/0 (socket says 172.21.15.57:47854) 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 -- 172.21.15.57:0/892402369 learned_addr learned my addr 172.21.15.57:0/892402369 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4277fff700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856ed60 0x5565b8579fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f4277fff700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b85779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 -- 172.21.15.57:0/892402369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856ed60 msgr2=0x5565b8579fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856ed60 0x5565b8579fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 -- 172.21.15.57:0/892402369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 msgr2=0x5565b85779a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b85779a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.097+0000 7f427cb9b700 1 -- 172.21.15.57:0/892402369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c009b70 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4277fff700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b85779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f427cb9b700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8577260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f426c005950 tx=0x7f426c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a952d4de58bdab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4257fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4257fff700 1 -- 172.21.15.57:0/892402369 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c019630 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565b8562eb0 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4257fff700 1 -- 172.21.15.57:0/892402369 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f426c019790 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4257fff700 1 -- 172.21.15.57:0/892402369 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f426c019950 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4292806b80 10 client.26773 fetch_fsmap learned FSMap version 55 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.098+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5565b82d6c00 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4257fff700 1 -- 172.21.15.57:0/892402369 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565b8562eb0 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4257fff700 1 -- 172.21.15.57:0/892402369 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 55) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f426c0048f0 con 0x5565b856f680 2023-10-06T22:22:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4292806b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4292806b80 10 client.26773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4292806b80 20 client.26773 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4292806b80 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b858a060 0x5565b858c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4292806b80 4 client.26773 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565b84bbdb0 con 0x5565b858a060 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.099+0000 7f4277fff700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b858a060 0x5565b858c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.100+0000 7f4257fff700 10 client.26773 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.100+0000 7f4277fff700 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b858a060 0x5565b858c480 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e1142fe0657281 server_cookie=29567702b1e55bec in_seq=0 out_seq=0 2023-10-06T22:22:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4257fff700 1 -- 172.21.15.57:0/892402369 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5565b84bbdb0 con 0x5565b858a060 2023-10-06T22:22:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4257fff700 10 client.26773 handle_command_reply: tid=0 2023-10-06T22:22:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 client.26773 shutdown 2023-10-06T22:22:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 msgr2=0x5565b8577260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8577260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f426c005950 tx=0x7f426c005dc0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 shutdown_connections 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b858a060 0x5565b858c480 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b857a530 0x5565b85779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b856ed60 0x5565b8579fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4257fff700 0 client.26773 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 --2- 172.21.15.57:0/892402369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8577260 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 >> 172.21.15.57:0/892402369 conn(0x5565b84e4200 msgr2=0x5565b856ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 shutdown_connections 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.101+0000 7f4292806b80 1 -- 172.21.15.57:0/892402369 wait complete. 2023-10-06T22:22:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.102+0000 7f4292806b80 20 client.26773 trim_cache size 0 max 16384 2023-10-06T22:22:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 Processor -- start 2023-10-06T22:22:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 -- start start 2023-10-06T22:22:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b8571d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8576ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 0x5565b8572630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b83438c0 con 0x5565b856ed60 2023-10-06T22:22:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b856c680 con 0x5565b857a530 2023-10-06T22:22:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4292806b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b85656b0 con 0x5565b856f680 2023-10-06T22:22:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8576ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8576ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47864/0 (socket says 172.21.15.57:47864) 2023-10-06T22:22:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f427cb9b700 1 -- 172.21.15.57:0/95875034 learned_addr learned my addr 172.21.15.57:0/95875034 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.117+0000 7f4277fff700 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b8571d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f427cb9b700 1 -- 172.21.15.57:0/95875034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 msgr2=0x5565b8572630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f427cb9b700 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 0x5565b8572630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f427cb9b700 1 -- 172.21.15.57:0/95875034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 msgr2=0x5565b8571d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f427cb9b700 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b8571d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f427cb9b700 1 -- 172.21.15.57:0/95875034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b83141f0 con 0x5565b856f680 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f4277fff700 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b8571d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f427cb9b700 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8576ac0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f426c0092a0 tx=0x7f426c00e5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e75f0f9a86a045a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f42637fe700 1 -- 172.21.15.57:0/95875034 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e730 con 0x5565b856f680 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f42637fe700 1 -- 172.21.15.57:0/95875034 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426c019430 con 0x5565b856f680 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.118+0000 7f42637fe700 1 -- 172.21.15.57:0/95875034 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0258f0 con 0x5565b856f680 2023-10-06T22:22:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 -- 172.21.15.57:0/95875034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 msgr2=0x5565b8576ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8576ac0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f426c0092a0 tx=0x7f426c00e5c0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 -- 172.21.15.57:0/95875034 shutdown_connections 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 0x5565b8572630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b8571d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 --2- 172.21.15.57:0/95875034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b8576ac0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 -- 172.21.15.57:0/95875034 >> 172.21.15.57:0/95875034 conn(0x5565b84e4200 msgr2=0x5565b8579c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 -- 172.21.15.57:0/95875034 shutdown_connections 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 -- 172.21.15.57:0/95875034 wait complete. 2023-10-06T22:22:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.119+0000 7f4292806b80 1 -- start start 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 1 Processor -- start 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b857f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b857f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 0x5565b85dce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b85dd390 con 0x5565b856ed60 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b85dd4d0 con 0x5565b857a530 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b85dd690 con 0x5565b856f680 2023-10-06T22:22:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4292806b80 10 client.0 _ll_register_callbacks cb 0x7f4264013560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4277fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b857f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b857f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4277fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b857f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36394/0 (socket says 172.21.15.57:36394) 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4277fff700 1 -- 172.21.15.57:0/2481593162 learned_addr learned my addr 172.21.15.57:0/2481593162 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.120+0000 7f4277fff700 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 0x5565b85dce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f427cb9b700 1 -- 172.21.15.57:0/2481593162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 msgr2=0x5565b85dce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f427cb9b700 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 0x5565b85dce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f427cb9b700 1 -- 172.21.15.57:0/2481593162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 msgr2=0x5565b857f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f427cb9b700 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b857f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f427cb9b700 1 -- 172.21.15.57:0/2481593162 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565b82d6c00 con 0x5565b856f680 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f427cb9b700 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b857f790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5565b8577770 tx=0x7f426c025550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dff6e1fa131febd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f4257fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:22:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f4257fff700 1 -- 172.21.15.57:0/2481593162 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0199b0 con 0x5565b856f680 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f426c019c50 con 0x5565b856f680 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.121+0000 7f4257fff700 1 -- 172.21.15.57:0/2481593162 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f426c000e40 con 0x5565b856f680 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4257fff700 1 -- 172.21.15.57:0/2481593162 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f426c00e7b0 con 0x5565b856f680 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 10 client.26779 fetch_fsmap learned FSMap version 55 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5565b82ec250 con 0x5565b856f680 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4257fff700 1 -- 172.21.15.57:0/2481593162 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426c04d020 con 0x5565b856f680 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4257fff700 1 -- 172.21.15.57:0/2481593162 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 55) v1 ==== 7946+0+0 (secure 0 0 0) 0x7f426c003810 con 0x5565b856f680 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 10 client.26779 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 10 client.26779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 20 client.26779 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b85de930 0x5565b85e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 4 client.26779 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:22:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565b84bbad0 con 0x5565b85de930 2023-10-06T22:22:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.122+0000 7f4277fff700 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b85de930 0x5565b85e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.123+0000 7f4277fff700 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b85de930 0x5565b85e0d50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9811dc1b18cdd855 server_cookie=58ad6ddb615a7582 in_seq=0 out_seq=0 2023-10-06T22:22:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.123+0000 7f4257fff700 10 client.26779 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.144 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:22:33.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.392+0000 7ffaff5ff700 1 -- 172.21.15.57:0/3699839743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf80ffc00 msgr2=0x7ffaf80fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.392+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/3699839743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf80ffc00 0x7ffaf80fffe0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009990 tx=0x7ffaf4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.392+0000 7ffaff5ff700 1 -- 172.21.15.57:0/3699839743 shutdown_connections 2023-10-06T22:22:33.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.392+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/3699839743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf810d7b0 0x7ffaf810fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.392+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/3699839743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf80ffc00 0x7ffaf80fffe0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.392+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/3699839743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaf81005b0 0x7ffaf810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.392+0000 7ffaff5ff700 1 -- 172.21.15.57:0/3699839743 >> 172.21.15.57:0/3699839743 conn(0x7ffaf80fb830 msgr2=0x7ffaf80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.393+0000 7ffaff5ff700 1 -- 172.21.15.57:0/3699839743 shutdown_connections 2023-10-06T22:22:33.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.393+0000 7ffaff5ff700 1 -- 172.21.15.57:0/3699839743 wait complete. 2023-10-06T22:22:33.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 Processor -- start 2023-10-06T22:22:33.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 -- start start 2023-10-06T22:22:33.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf80ffc00 0x7ffaf81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf81005b0 0x7ffaf8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaf810d7b0 0x7ffaf819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf819e1a0 con 0x7ffaf810d7b0 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf819e310 con 0x7ffaf81005b0 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffaff5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf819e4d0 con 0x7ffaf80ffc00 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.394+0000 7ffafdb9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaf810d7b0 0x7ffaf819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf80ffc00 0x7ffaf81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf80ffc00 0x7ffaf81991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47896/0 (socket says 172.21.15.57:47896) 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafd39b700 1 -- 172.21.15.57:0/2182894576 learned_addr learned my addr 172.21.15.57:0/2182894576 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf81005b0 0x7ffaf8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafdb9c700 1 -- 172.21.15.57:0/2182894576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf80ffc00 msgr2=0x7ffaf81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf80ffc00 0x7ffaf81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafdb9c700 1 -- 172.21.15.57:0/2182894576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf81005b0 msgr2=0x7ffaf8199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf81005b0 0x7ffaf8199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafdb9c700 1 -- 172.21.15.57:0/2182894576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40095f0 con 0x7ffaf810d7b0 2023-10-06T22:22:33.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaf810d7b0 0x7ffaf819db60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ffaf806a380 tx=0x7ffaf0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.395+0000 7ffaee7fc700 1 -- 172.21.15.57:0/2182894576 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0004db0 con 0x7ffaf810d7b0 2023-10-06T22:22:33.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.396+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf819e7b0 con 0x7ffaf810d7b0 2023-10-06T22:22:33.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.396+0000 7ffaee7fc700 1 -- 172.21.15.57:0/2182894576 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf000f040 con 0x7ffaf810d7b0 2023-10-06T22:22:33.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.396+0000 7ffaee7fc700 1 -- 172.21.15.57:0/2182894576 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0005a20 con 0x7ffaf810d7b0 2023-10-06T22:22:33.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.396+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf81a5ff0 con 0x7ffaf810d7b0 2023-10-06T22:22:33.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.399+0000 7ffaee7fc700 1 -- 172.21.15.57:0/2182894576 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ffaf000db60 con 0x7ffaf810d7b0 2023-10-06T22:22:33.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.401+0000 7ffb12f63b80 1 Processor -- start 2023-10-06T22:22:33.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffb12f63b80 1 -- start start 2023-10-06T22:22:33.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffaee7fc700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffae40a1670 0x7ffae40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b389f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffaee7fc700 1 -- 172.21.15.57:0/2182894576 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ffaf00cc9b0 con 0x7ffaf810d7b0 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b3808460 con 0x5568b389f9e0 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b37f9c20 con 0x5568b38aa890 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.402+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b37f7540 con 0x5568b389f0c0 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38acc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57492/0 (socket says 172.21.15.57:57492) 2023-10-06T22:22:33.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 -- 172.21.15.57:0/3570647103 learned_addr learned my addr 172.21.15.57:0/3570647103 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffae40a1670 0x7ffae40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 -- 172.21.15.57:0/3570647103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 msgr2=0x5568b389f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b389f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 -- 172.21.15.57:0/3570647103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 msgr2=0x5568b38aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 -- 172.21.15.57:0/3570647103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568b361dde0 con 0x5568b38aa890 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38acc80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00963c0 tx=0x7ffaf0099580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05c5f249f65dfbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffae40a1670 0x7ffae40a3b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffaf819a7d0 tx=0x7ffae8009610 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.403+0000 7ffae37fe700 1 -- 172.21.15.57:0/3570647103 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf009daa0 con 0x5568b38aa890 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffae37fe700 1 -- 172.21.15.57:0/3570647103 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf00a1ce0 con 0x5568b38aa890 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffae37fe700 1 -- 172.21.15.57:0/3570647103 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0099bb0 con 0x5568b38aa890 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3570647103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 msgr2=0x5568b38acc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38acc80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00963c0 tx=0x7ffaf0099580 comp rx=0 tx=0).stop 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3570647103 shutdown_connections 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38acc80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b389f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3570647103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3570647103 >> 172.21.15.57:0/3570647103 conn(0x5568b3814260 msgr2=0x5568b389b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.404+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3570647103 shutdown_connections 2023-10-06T22:22:33.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.405+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3570647103 wait complete. 2023-10-06T22:22:33.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.405+0000 7ffb12f63b80 1 -- start start 2023-10-06T22:22:33.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.405+0000 7ffb12f63b80 1 Processor -- start 2023-10-06T22:22:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b38a7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b37f2e30 con 0x5568b389f9e0 2023-10-06T22:22:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b378f710 con 0x5568b38aa890 2023-10-06T22:22:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b378f3b0 con 0x5568b389f0c0 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffb12f63b80 10 client.0 _ll_register_callbacks cb 0x5568b35ec800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b38a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b38a7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47906/0 (socket says 172.21.15.57:47906) 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafd39b700 1 -- 172.21.15.57:0/3229627469 learned_addr learned my addr 172.21.15.57:0/3229627469 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafdb9c700 1 -- 172.21.15.57:0/3229627469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 msgr2=0x5568b38a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b38a7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafdb9c700 1 -- 172.21.15.57:0/3229627469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 msgr2=0x5568b38a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.406+0000 7ffafdb9c700 1 -- 172.21.15.57:0/3229627469 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf81a5ff0 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38a8100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00998e0 tx=0x7ffaf0095dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d429fdfcd1298e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffae0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3229627469 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0099050 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568b3892f10 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3229627469 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ffaf00b2160 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3229627469 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaf0099bb0 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffb12f63b80 10 client.26780 fetch_fsmap learned FSMap version 55 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.407+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5568b3606c60 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3229627469 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568b3892f10 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3229627469 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 55) v1 ==== 7946+0+0 (secure 0 0 0) 0x7ffaf00b9050 con 0x5568b38aa890 2023-10-06T22:22:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffb12f63b80 10 client.26780 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffb12f63b80 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffb12f63b80 20 client.26780 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b38ba0c0 0x5568b38bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffb12f63b80 4 client.26780 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568b37ebe10 con 0x5568b38ba0c0 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.408+0000 7ffafd39b700 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b38ba0c0 0x5568b38bc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.409+0000 7ffae0ff9700 10 client.26780 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.409+0000 7ffafd39b700 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b38ba0c0 0x5568b38bc4e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe58ee9487cf0d1b server_cookie=35b43845bee4d09f in_seq=0 out_seq=0 2023-10-06T22:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.556+0000 7f4257fff700 1 -- 172.21.15.57:0/2481593162 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565b84bbad0 con 0x5565b85de930 2023-10-06T22:22:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.556+0000 7f4257fff700 10 client.26779 handle_command_reply: tid=0 2023-10-06T22:22:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.556+0000 7f4292806b80 1 client.26779 shutdown 2023-10-06T22:22:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.556+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 msgr2=0x5565b857f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.556+0000 7f4292806b80 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b857f790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5565b8577770 tx=0x7f426c025550 comp rx=0 tx=0).stop 2023-10-06T22:22:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.557+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 shutdown_connections 2023-10-06T22:22:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.557+0000 7f4292806b80 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5565b85de930 0x5565b85e0d50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.557+0000 7f4292806b80 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b857a530 0x5565b85dce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.557+0000 7f4292806b80 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5565b856ed60 0x5565b857f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.557+0000 7f4292806b80 1 --2- 172.21.15.57:0/2481593162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565b856f680 0x5565b857f790 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5565b8577770 tx=0x7f426c025550 comp rx=0 tx=0).stop 2023-10-06T22:22:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.557+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 >> 172.21.15.57:0/2481593162 conn(0x5565b84e4200 msgr2=0x5565b856a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.557+0000 7f4257fff700 0 client.26779 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.557+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3229627469 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5568b37ebe10 con 0x5568b38ba0c0 2023-10-06T22:22:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.557+0000 7ffae0ff9700 10 client.26780 handle_command_reply: tid=0 2023-10-06T22:22:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.557+0000 7ffb12f63b80 1 client.26780 shutdown 2023-10-06T22:22:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.558+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 shutdown_connections 2023-10-06T22:22:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.558+0000 7f4292806b80 1 -- 172.21.15.57:0/2481593162 wait complete. 2023-10-06T22:22:33.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 msgr2=0x5568b38a8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38a8100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00998e0 tx=0x7ffaf0095dc0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.558+0000 7f4292806b80 20 client.26779 trim_cache size 0 max 16384 2023-10-06T22:22:33.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 shutdown_connections 2023-10-06T22:22:33.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b38ba0c0 0x5568b38bc4e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38aa890 0x5568b38a8100 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b389f0c0 0x5568b38a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3229627469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 >> 172.21.15.57:0/3229627469 conn(0x5568b3814260 msgr2=0x5568b389afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.558+0000 7ffae0ff9700 0 client.26780 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:22:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:22:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:22:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:22:33.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.559+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 shutdown_connections 2023-10-06T22:22:33.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.559+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3229627469 wait complete. 2023-10-06T22:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f42680a15f0 msgr2=0x7f42680a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f42680a15f0 0x7f42680a3ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42700a31e0 tx=0x7f427000b040 comp rx=0 tx=0).stop 2023-10-06T22:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 msgr2=0x7f427819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 0x7f427819dad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4270009fd0 tx=0x7f4270007930 comp rx=0 tx=0).stop 2023-10-06T22:22:33.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.559+0000 7ffb12f63b80 20 client.26780 trim_cache size 0 max 16384 2023-10-06T22:22:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 shutdown_connections 2023-10-06T22:22:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f42680a15f0 0x7f42680a3ab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f427810b2b0 0x7f427819dad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4278101e90 0x7f4278199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 --2- 172.21.15.57:0/2267261212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781027b0 0x7f4278199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 >> 172.21.15.57:0/2267261212 conn(0x7f42780fb690 msgr2=0x7f427810d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.559+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 shutdown_connections 2023-10-06T22:22:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:33.560+0000 7f427db9d700 1 -- 172.21.15.57:0/2267261212 wait complete. 2023-10-06T22:22:33.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:33 smithi190 ceph-mon[134891]: pgmap v241: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 449 op/s 2023-10-06T22:22:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 Processor -- start 2023-10-06T22:22:33.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 -- start start 2023-10-06T22:22:33.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b3673920 con 0x5568b389f9e0 2023-10-06T22:22:33.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b37ebe10 con 0x5568b389f0c0 2023-10-06T22:22:33.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.583+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b38a08b0 con 0x5568b38aa890 2023-10-06T22:22:33.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafcb9a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57506/0 (socket says 172.21.15.57:57506) 2023-10-06T22:22:33.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 -- 172.21.15.57:0/464738697 learned_addr learned my addr 172.21.15.57:0/464738697 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 -- 172.21.15.57:0/464738697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 msgr2=0x5568b38a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 -- 172.21.15.57:0/464738697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 msgr2=0x5568b38a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 -- 172.21.15.57:0/464738697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568b3897140 con 0x5568b389f0c0 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.584+0000 7ffafd39b700 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38a1f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5568b38bcdc0 tx=0x7ffaf400b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8b11e90869b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.585+0000 7ffae3fff700 1 -- 172.21.15.57:0/464738697 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400bd50 con 0x5568b389f0c0 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.585+0000 7ffae3fff700 1 -- 172.21.15.57:0/464738697 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf401e040 con 0x5568b389f0c0 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.585+0000 7ffae3fff700 1 -- 172.21.15.57:0/464738697 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40052a0 con 0x5568b389f0c0 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.585+0000 7ffb12f63b80 1 -- 172.21.15.57:0/464738697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 msgr2=0x5568b38a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.585+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38a1f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5568b38bcdc0 tx=0x7ffaf400b610 comp rx=0 tx=0).stop 2023-10-06T22:22:33.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 -- 172.21.15.57:0/464738697 shutdown_connections 2023-10-06T22:22:33.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38a1f60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/464738697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 -- 172.21.15.57:0/464738697 >> 172.21.15.57:0/464738697 conn(0x5568b3814260 msgr2=0x5568b38a5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 -- 172.21.15.57:0/464738697 shutdown_connections 2023-10-06T22:22:33.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 -- 172.21.15.57:0/464738697 wait complete. 2023-10-06T22:22:33.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 -- start start 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.586+0000 7ffb12f63b80 1 Processor -- start 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffb12f63b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b38afe70 con 0x5568b389f9e0 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b38affe0 con 0x5568b389f0c0 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffb12f63b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568b38b01a0 con 0x5568b38aa890 2023-10-06T22:22:33.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffb12f63b80 10 client.0 _ll_register_callbacks cb 0x7ffaf0099810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffafd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:57518/0 (socket says 172.21.15.57:57518) 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffafd39b700 1 -- 172.21.15.57:0/3421673191 learned_addr learned my addr 172.21.15.57:0/3421673191 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.587+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffafdb9c700 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffafd39b700 1 -- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 msgr2=0x5568b38af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffafd39b700 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffafd39b700 1 -- 172.21.15.57:0/3421673191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 msgr2=0x5568b38af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffafd39b700 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffafd39b700 1 -- 172.21.15.57:0/3421673191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf8110810 con 0x5568b389f0c0 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffafd39b700 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38b1130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b2e0 tx=0x7ffaf4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfeba473af4809a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:33.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffae0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3421673191 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400bd50 con 0x5568b389f0c0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffaf00a1ce0 con 0x5568b389f0c0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.588+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3421673191 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ffaf4026260 con 0x5568b389f0c0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3421673191 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaf400beb0 con 0x5568b389f0c0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 10 client.26786 fetch_fsmap learned FSMap version 55 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5568b3644250 con 0x5568b389f0c0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3421673191 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf403d020 con 0x5568b389f0c0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3421673191 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 55) v1 ==== 7946+0+0 (secure 0 0 0) 0x7ffaf4004500 con 0x5568b389f0c0 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 10 client.26786 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 20 client.26786 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:33.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b390efe0 0x5568b3911400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 4 client.26786 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5568b37ebb30 con 0x5568b390efe0 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.589+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b390efe0 0x5568b3911400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffafcb9a700 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b390efe0 0x5568b3911400 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8130401408bf11d7 server_cookie=d2ac4da3118c42c in_seq=0 out_seq=0 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffae0ff9700 10 client.26786 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffae0ff9700 1 -- 172.21.15.57:0/3421673191 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5568b37ebb30 con 0x5568b390efe0 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffae0ff9700 10 client.26786 handle_command_reply: tid=0 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 client.26786 shutdown 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 msgr2=0x5568b38b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38b1130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b2e0 tx=0x7ffaf4005f10 comp rx=0 tx=0).stop 2023-10-06T22:22:33.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 shutdown_connections 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5568b390efe0 0x5568b3911400 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5568b38aa890 0x5568b38af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389f0c0 0x5568b38b1130 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 --2- 172.21.15.57:0/3421673191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b389f9e0 0x5568b38af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffae0ff9700 0 client.26786 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.590+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 >> 172.21.15.57:0/3421673191 conn(0x5568b3814260 msgr2=0x5568b38a66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 shutdown_connections 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffb12f63b80 1 -- 172.21.15.57:0/3421673191 wait complete. 2023-10-06T22:22:33.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffb12f63b80 20 client.26786 trim_cache size 0 max 16384 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:22:33.598 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:22:33.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffae40a1670 msgr2=0x7ffae40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffae40a1670 0x7ffae40a3b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffaf819a7d0 tx=0x7ffae8009610 comp rx=0 tx=0).stop 2023-10-06T22:22:33.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaf810d7b0 msgr2=0x7ffaf819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:33.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaf810d7b0 0x7ffaf819db60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ffaf806a380 tx=0x7ffaf0007f90 comp rx=0 tx=0).stop 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 shutdown_connections 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ffae40a1670 0x7ffae40a3b30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffaf810d7b0 0x7ffaf819db60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf80ffc00 0x7ffaf81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 --2- 172.21.15.57:0/2182894576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaf81005b0 0x7ffaf8199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.591+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 >> 172.21.15.57:0/2182894576 conn(0x7ffaf80fb830 msgr2=0x7ffaf810f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.592+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 shutdown_connections 2023-10-06T22:22:33.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:22:33.592+0000 7ffaff5ff700 1 -- 172.21.15.57:0/2182894576 wait complete. 2023-10-06T22:22:33.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:33 smithi057 ceph-mon[98492]: pgmap v241: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 449 op/s 2023-10-06T22:22:33.619 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:22:33.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:33 smithi156 ceph-mon[134358]: pgmap v241: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 449 op/s 2023-10-06T22:22:35.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:35 smithi190 ceph-mon[134891]: pgmap v242: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2023-10-06T22:22:35.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:35 smithi057 ceph-mon[98492]: pgmap v242: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2023-10-06T22:22:35.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:35 smithi156 ceph-mon[134358]: pgmap v242: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2023-10-06T22:22:37.319 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:37 smithi190 ceph-mon[134891]: pgmap v243: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2023-10-06T22:22:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:37 smithi057 ceph-mon[98492]: pgmap v243: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2023-10-06T22:22:37.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:37 smithi156 ceph-mon[134358]: pgmap v243: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2023-10-06T22:22:39.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:39 smithi156 ceph-mon[134358]: pgmap v244: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 431 op/s 2023-10-06T22:22:39.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:39 smithi190 ceph-mon[134891]: pgmap v244: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 431 op/s 2023-10-06T22:22:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:39 smithi057 ceph-mon[98492]: pgmap v244: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 431 op/s 2023-10-06T22:22:40.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:40 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:40.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:40 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:40.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:40 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:22:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:41 smithi190 ceph-mon[134891]: pgmap v245: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 384 op/s 2023-10-06T22:22:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:41 smithi057 ceph-mon[98492]: pgmap v245: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 384 op/s 2023-10-06T22:22:41.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:41 smithi156 ceph-mon[134358]: pgmap v245: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 384 op/s 2023-10-06T22:22:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:43 smithi190 ceph-mon[134891]: pgmap v246: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 435 op/s 2023-10-06T22:22:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:43 smithi057 ceph-mon[98492]: pgmap v246: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 435 op/s 2023-10-06T22:22:43.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:43 smithi156 ceph-mon[134358]: pgmap v246: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 435 op/s 2023-10-06T22:22:45.585 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:45 smithi190 ceph-mon[134891]: pgmap v247: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 355 op/s 2023-10-06T22:22:45.586 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:45 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:22:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:45 smithi057 ceph-mon[98492]: pgmap v247: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 355 op/s 2023-10-06T22:22:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:45 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:22:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:45 smithi156 ceph-mon[134358]: pgmap v247: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 355 op/s 2023-10-06T22:22:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:45 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:22:46.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:22:46.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:22:46.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:46 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:22:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:22:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:22:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:46 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:22:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 -- 172.21.15.57:0/2549807947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14000732e0 msgr2=0x7f14000736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 --2- 172.21.15.57:0/2549807947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14000732e0 0x7f14000736c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f13f00098f0 tx=0x7f13f0009c00 comp rx=0 tx=0).stop 2023-10-06T22:22:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 -- 172.21.15.57:0/2549807947 shutdown_connections 2023-10-06T22:22:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 --2- 172.21.15.57:0/2549807947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f140010d6a0 0x7f140010fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 --2- 172.21.15.57:0/2549807947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14000732e0 0x7f14000736c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 --2- 172.21.15.57:0/2549807947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1400073c00 0x7f140010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 -- 172.21.15.57:0/2549807947 >> 172.21.15.57:0/2549807947 conn(0x7f14000fc950 msgr2=0x7f14000fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 -- 172.21.15.57:0/2549807947 shutdown_connections 2023-10-06T22:22:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.718+0000 7f1404bff700 1 -- 172.21.15.57:0/2549807947 wait complete. 2023-10-06T22:22:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 Processor -- start 2023-10-06T22:22:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 -- start start 2023-10-06T22:22:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14000732e0 0x7f1400199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1400073c00 0x7f1400199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f140010d6a0 0x7f140019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019e410 con 0x7f140010d6a0 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019e580 con 0x7f1400073c00 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.719+0000 7f1404bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019e740 con 0x7f14000732e0 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14000732e0 0x7f1400199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14000732e0 0x7f1400199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46132/0 (socket says 172.21.15.57:46132) 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fe59c700 1 -- 172.21.15.57:0/3757917495 learned_addr learned my addr 172.21.15.57:0/3757917495 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1400073c00 0x7f1400199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fed9d700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f140010d6a0 0x7f140019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fdd9b700 1 -- 172.21.15.57:0/3757917495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14000732e0 msgr2=0x7f1400199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14000732e0 0x7f1400199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fdd9b700 1 -- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f140010d6a0 msgr2=0x7f140019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f140010d6a0 0x7f140019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fdd9b700 1 -- 172.21.15.57:0/3757917495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f8009d00 con 0x7f1400073c00 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fe59c700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14000732e0 0x7f1400199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.720+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1400073c00 0x7f1400199760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f13f400e9f0 tx=0x7f13f400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.721+0000 7f13ef7fe700 1 -- 172.21.15.57:0/3757917495 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f400c9c0 con 0x7f1400073c00 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.721+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f0009520 con 0x7f1400073c00 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.721+0000 7f13ef7fe700 1 -- 172.21.15.57:0/3757917495 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f4010430 con 0x7f1400073c00 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.721+0000 7f13ef7fe700 1 -- 172.21.15.57:0/3757917495 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4018680 con 0x7f1400073c00 2023-10-06T22:22:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.721+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14001a6080 con 0x7f1400073c00 2023-10-06T22:22:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.724+0000 7f13ef7fe700 1 -- 172.21.15.57:0/3757917495 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f13f4010860 con 0x7f1400073c00 2023-10-06T22:22:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.725+0000 7f14185ddb80 1 Processor -- start 2023-10-06T22:22:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.725+0000 7f14185ddb80 1 -- start start 2023-10-06T22:22:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13ef7fe700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13e80a15f0 0x7f13e80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a576c00 0x555a5a576fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a577520 0x555a5a581e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a5847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f14185ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a4e0400 con 0x555a5a577520 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f14185ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a4d1bc0 con 0x555a5a5823d0 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f14185ddb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a4cf4e0 con 0x555a5a576c00 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a576c00 0x555a5a576fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13ef7fe700 1 -- 172.21.15.57:0/3757917495 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f13f4014070 con 0x7f1400073c00 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fed9d700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13e80a15f0 0x7f13e80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a576c00 0x555a5a576fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46134/0 (socket says 172.21.15.57:46134) 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fe59c700 1 -- 172.21.15.57:0/2590238855 learned_addr learned my addr 172.21.15.57:0/2590238855 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a577520 0x555a5a581e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fed9d700 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a5847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fe59c700 1 -- 172.21.15.57:0/2590238855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 msgr2=0x555a5a5847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fe59c700 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a5847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.726+0000 7f13fe59c700 1 -- 172.21.15.57:0/2590238855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a577520 msgr2=0x555a5a581e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.727+0000 7f13fe59c700 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a577520 0x555a5a581e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.727+0000 7f13fe59c700 1 -- 172.21.15.57:0/2590238855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a5a2f5d80 con 0x555a5a576c00 2023-10-06T22:22:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.727+0000 7f13fe59c700 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a576c00 0x555a5a576fe0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13f0005b60 tx=0x7f13f0005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e899929d526e791b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.727+0000 7f13fed9d700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13e80a15f0 0x7f13e80a3ab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009610 tx=0x7f13f80099e0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.727+0000 7f13ecff9700 1 -- 172.21.15.57:0/2590238855 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f001c050 con 0x555a5a576c00 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.727+0000 7f13ecff9700 1 -- 172.21.15.57:0/2590238855 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f001d040 con 0x555a5a576c00 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.727+0000 7f13ecff9700 1 -- 172.21.15.57:0/2590238855 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0005360 con 0x555a5a576c00 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 -- 172.21.15.57:0/2590238855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a576c00 msgr2=0x555a5a576fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a576c00 0x555a5a576fe0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13f0005b60 tx=0x7f13f0005e00 comp rx=0 tx=0).stop 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 -- 172.21.15.57:0/2590238855 shutdown_connections 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a5847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a576c00 0x555a5a576fe0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2590238855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a577520 0x555a5a581e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 -- 172.21.15.57:0/2590238855 >> 172.21.15.57:0/2590238855 conn(0x555a5a4ec200 msgr2=0x555a5a573080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 -- 172.21.15.57:0/2590238855 shutdown_connections 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 -- 172.21.15.57:0/2590238855 wait complete. 2023-10-06T22:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.728+0000 7f14185ddb80 1 -- start start 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 1 Processor -- start 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a581e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a57f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a57f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a4cadd0 con 0x555a5a576c00 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a4676b0 con 0x555a5a577520 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a467350 con 0x555a5a5823d0 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f14185ddb80 10 client.0 _ll_register_callbacks cb 0x555a5a2c47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f13fe59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a581e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.729+0000 7f13fdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a57f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a57f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:42960/0 (socket says 172.21.15.57:42960) 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a57f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 -- 172.21.15.57:0/2200100873 learned_addr learned my addr 172.21.15.57:0/2200100873 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 -- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 msgr2=0x555a5a57f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a57f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 -- 172.21.15.57:0/2200100873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 msgr2=0x555a5a581e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a581e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 -- 172.21.15.57:0/2200100873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14001107d0 con 0x555a5a577520 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fed9d700 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a57f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fe59c700 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a581e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a57f010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f13f400e9c0 tx=0x7f13f40caec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d90580528c5048a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13e5ffb700 1 -- 172.21.15.57:0/2200100873 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f409e120 con 0x555a5a577520 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f13e5ffb700 1 -- 172.21.15.57:0/2200100873 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f13f409a8f0 con 0x555a5a577520 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.730+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a5a56ad50 con 0x555a5a577520 2023-10-06T22:22:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f13e5ffb700 1 -- 172.21.15.57:0/2200100873 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13f409a1f0 con 0x555a5a577520 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 10 client.26798 fetch_fsmap learned FSMap version 56 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x555a5a2dec00 con 0x555a5a577520 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f13e5ffb700 1 -- 172.21.15.57:0/2200100873 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a5a56ad50 con 0x555a5a577520 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f13e5ffb700 1 -- 172.21.15.57:0/2200100873 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f13f4099770 con 0x555a5a577520 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 10 client.26798 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 10 client.26798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 20 client.26798 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a591ea0 0x555a5a5942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 4 client.26798 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a5a4c3db0 con 0x555a5a591ea0 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.731+0000 7f13fe59c700 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a591ea0 0x555a5a5942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.732+0000 7f13fe59c700 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a591ea0 0x555a5a5942c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a795fcc6b1b75df8 server_cookie=57a823c4e0e98675 in_seq=0 out_seq=0 2023-10-06T22:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.732+0000 7f13e5ffb700 10 client.26798 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.732+0000 7f13e5ffb700 1 -- 172.21.15.57:0/2200100873 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a5a4c3db0 con 0x555a5a591ea0 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.732+0000 7f13e5ffb700 10 client.26798 handle_command_reply: tid=0 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 client.26798 shutdown 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 msgr2=0x555a5a57f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a57f010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f13f400e9c0 tx=0x7f13f40caec0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 shutdown_connections 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a591ea0 0x555a5a5942c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a57f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a581e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 --2- 172.21.15.57:0/2200100873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a57f010 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f13e5ffb700 0 client.26798 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 >> 172.21.15.57:0/2200100873 conn(0x555a5a4ec200 msgr2=0x555a5a572c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 shutdown_connections 2023-10-06T22:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 1 -- 172.21.15.57:0/2200100873 wait complete. 2023-10-06T22:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.733+0000 7f14185ddb80 20 client.26798 trim_cache size 0 max 16384 2023-10-06T22:22:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 Processor -- start 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 -- start start 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a588eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a58e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a34b8c0 con 0x555a5a576c00 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a573e30 con 0x555a5a577520 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.749+0000 7f14185ddb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a589440 con 0x555a5a5823d0 2023-10-06T22:22:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a588eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a588eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:42970/0 (socket says 172.21.15.57:42970) 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a58e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 -- 172.21.15.57:0/1584677390 learned_addr learned my addr 172.21.15.57:0/1584677390 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fe59c700 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 -- 172.21.15.57:0/1584677390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 msgr2=0x555a5a58e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a58e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 -- 172.21.15.57:0/1584677390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 msgr2=0x555a5a57e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 -- 172.21.15.57:0/1584677390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a5a31c1f0 con 0x555a5a577520 2023-10-06T22:22:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a588eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13f409a180 tx=0x7f13f40987a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975c2d4eebd6eca3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13ed7fa700 1 -- 172.21.15.57:0/1584677390 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f409e120 con 0x555a5a577520 2023-10-06T22:22:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13ed7fa700 1 -- 172.21.15.57:0/1584677390 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f409a8f0 con 0x555a5a577520 2023-10-06T22:22:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.750+0000 7f13ed7fa700 1 -- 172.21.15.57:0/1584677390 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40a2260 con 0x555a5a577520 2023-10-06T22:22:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:22:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:22:46.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:46 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:22:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 -- 172.21.15.57:0/1584677390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 msgr2=0x555a5a588eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a588eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13f409a180 tx=0x7f13f40987a0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 -- 172.21.15.57:0/1584677390 shutdown_connections 2023-10-06T22:22:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a5823d0 0x555a5a58e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 --2- 172.21.15.57:0/1584677390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a577520 0x555a5a588eb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 -- 172.21.15.57:0/1584677390 >> 172.21.15.57:0/1584677390 conn(0x555a5a4ec200 msgr2=0x555a5a598c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 -- 172.21.15.57:0/1584677390 shutdown_connections 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 -- 172.21.15.57:0/1584677390 wait complete. 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.751+0000 7f14185ddb80 1 -- start start 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 1 Processor -- start 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a577520 0x555a5a57b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a57bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a57c2e0 con 0x555a5a576c00 2023-10-06T22:22:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a57c450 con 0x555a5a5823d0 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5a57c610 con 0x555a5a577520 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f14185ddb80 10 client.0 _ll_register_callbacks cb 0x7f13f40bb0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f13fdd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a577520 0x555a5a57b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f13fe59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f13fdd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a577520 0x555a5a57b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46152/0 (socket says 172.21.15.57:46152) 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.752+0000 7f13fdd9b700 1 -- 172.21.15.57:0/719243133 learned_addr learned my addr 172.21.15.57:0/719243133 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fed9d700 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a57bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fdd9b700 1 -- 172.21.15.57:0/719243133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 msgr2=0x555a5a57bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a57bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fdd9b700 1 -- 172.21.15.57:0/719243133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 msgr2=0x555a5a57b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fdd9b700 1 -- 172.21.15.57:0/719243133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a5a2dec00 con 0x555a5a577520 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fe59c700 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13fdd9b700 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a577520 0x555a5a57b7f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f13f40a2050 tx=0x7f13f40caa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39a23828abd74c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13e5ffb700 1 -- 172.21.15.57:0/719243133 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40998f0 con 0x555a5a577520 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13e5ffb700 1 -- 172.21.15.57:0/719243133 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f13f409e120 con 0x555a5a577520 2023-10-06T22:22:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13f001d040 con 0x555a5a577520 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.753+0000 7f13e5ffb700 1 -- 172.21.15.57:0/719243133 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13f40a2580 con 0x555a5a577520 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 10 client.26812 fetch_fsmap learned FSMap version 56 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x555a5a2f4250 con 0x555a5a577520 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f13e5ffb700 1 -- 172.21.15.57:0/719243133 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f4063080 con 0x555a5a577520 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f13e5ffb700 1 -- 172.21.15.57:0/719243133 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f13f40961a0 con 0x555a5a577520 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 10 client.26812 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 10 client.26812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 20 client.26812 populate_metadata read hostname 'smithi057' 2023-10-06T22:22:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a5e6250 0x555a5a5e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:22:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 4 client.26812 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:22:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a5a4c3ad0 con 0x555a5a5e6250 2023-10-06T22:22:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.754+0000 7f13fe59c700 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a5e6250 0x555a5a5e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:22:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.755+0000 7f13fe59c700 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a5e6250 0x555a5a5e8640 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b900f094f8bfa8 server_cookie=3b0ceb44ae2110d1 in_seq=0 out_seq=0 2023-10-06T22:22:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.755+0000 7f13e5ffb700 10 client.26812 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f13e5ffb700 1 -- 172.21.15.57:0/719243133 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a5a4c3ad0 con 0x555a5a5e6250 2023-10-06T22:22:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f13e5ffb700 10 client.26812 handle_command_reply: tid=0 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 client.26812 shutdown 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a577520 msgr2=0x555a5a57b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a577520 0x555a5a57b7f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f13f40a2050 tx=0x7f13f40caa60 comp rx=0 tx=0).stop 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 shutdown_connections 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x555a5a5e6250 0x555a5a5e8640 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a5a5823d0 0x555a5a57bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555a5a576c00 0x555a5a57b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f13e5ffb700 0 client.26812 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 --2- 172.21.15.57:0/719243133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555a5a577520 0x555a5a57b7f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 >> 172.21.15.57:0/719243133 conn(0x555a5a4ec200 msgr2=0x555a5a57e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 shutdown_connections 2023-10-06T22:22:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.845+0000 7f14185ddb80 1 -- 172.21.15.57:0/719243133 wait complete. 2023-10-06T22:22:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.846+0000 7f14185ddb80 20 client.26812 trim_cache size 0 max 16384 2023-10-06T22:22:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:22:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:22:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:22:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.846+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13e80a15f0 msgr2=0x7f13e80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.846+0000 7f1404bff700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13e80a15f0 0x7f13e80a3ab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009610 tx=0x7f13f80099e0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.846+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1400073c00 msgr2=0x7f1400199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:22:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.846+0000 7f1404bff700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1400073c00 0x7f1400199760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f13f400e9f0 tx=0x7f13f400ed00 comp rx=0 tx=0).stop 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 shutdown_connections 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f13e80a15f0 0x7f13e80a3ab0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f140010d6a0 0x7f140019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14000732e0 0x7f1400199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 --2- 172.21.15.57:0/3757917495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1400073c00 0x7f1400199760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 >> 172.21.15.57:0/3757917495 conn(0x7f14000fc950 msgr2=0x7f140010f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 shutdown_connections 2023-10-06T22:22:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:22:46.847+0000 7f1404bff700 1 -- 172.21.15.57:0/3757917495 wait complete. 2023-10-06T22:22:47.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:47 smithi190 ceph-mon[134891]: pgmap v248: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 381 op/s 2023-10-06T22:22:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:47 smithi057 ceph-mon[98492]: pgmap v248: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 381 op/s 2023-10-06T22:22:47.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:47 smithi156 ceph-mon[134358]: pgmap v248: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 381 op/s 2023-10-06T22:22:49.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:49 smithi190 ceph-mon[134891]: pgmap v249: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2023-10-06T22:22:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:49 smithi057 ceph-mon[98492]: pgmap v249: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2023-10-06T22:22:49.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:49 smithi156 ceph-mon[134358]: pgmap v249: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2023-10-06T22:22:51.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:51 smithi190 ceph-mon[134891]: pgmap v250: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 315 op/s 2023-10-06T22:22:51.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:51 smithi057 ceph-mon[98492]: pgmap v250: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 315 op/s 2023-10-06T22:22:51.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:51 smithi156 ceph-mon[134358]: pgmap v250: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 315 op/s 2023-10-06T22:22:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:53 smithi190 ceph-mon[134891]: pgmap v251: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 390 op/s 2023-10-06T22:22:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:53 smithi057 ceph-mon[98492]: pgmap v251: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 390 op/s 2023-10-06T22:22:53.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:53 smithi156 ceph-mon[134358]: pgmap v251: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 390 op/s 2023-10-06T22:22:55.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:55 smithi190 ceph-mon[134891]: pgmap v252: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2023-10-06T22:22:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:55 smithi057 ceph-mon[98492]: pgmap v252: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2023-10-06T22:22:55.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:55 smithi156 ceph-mon[134358]: pgmap v252: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2023-10-06T22:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:57 smithi057 ceph-mon[98492]: pgmap v253: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2023-10-06T22:22:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:57 smithi156 ceph-mon[134358]: pgmap v253: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2023-10-06T22:22:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:57 smithi190 ceph-mon[134891]: pgmap v253: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2023-10-06T22:22:59.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:22:59 smithi156 ceph-mon[134358]: pgmap v254: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 358 op/s 2023-10-06T22:22:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:22:59 smithi190 ceph-mon[134891]: pgmap v254: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 358 op/s 2023-10-06T22:22:59.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:22:59 smithi057 ceph-mon[98492]: pgmap v254: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 358 op/s 2023-10-06T22:23:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 -- 172.21.15.57:0/1371203944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6c0ff850 msgr2=0x7f3e6c111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 --2- 172.21.15.57:0/1371203944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6c0ff850 0x7f3e6c111680 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009990 tx=0x7f3e60009ca0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 -- 172.21.15.57:0/1371203944 shutdown_connections 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 --2- 172.21.15.57:0/1371203944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e6c111bc0 0x7f3e6c113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 --2- 172.21.15.57:0/1371203944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e6c0fef30 0x7f3e6c0ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 --2- 172.21.15.57:0/1371203944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6c0ff850 0x7f3e6c111680 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 -- 172.21.15.57:0/1371203944 >> 172.21.15.57:0/1371203944 conn(0x7f3e6c076e70 msgr2=0x7f3e6c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 -- 172.21.15.57:0/1371203944 shutdown_connections 2023-10-06T22:23:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.019+0000 7f3e727ff700 1 -- 172.21.15.57:0/1371203944 wait complete. 2023-10-06T22:23:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.021+0000 7f3e727ff700 1 Processor -- start 2023-10-06T22:23:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.021+0000 7f3e727ff700 1 -- start start 2023-10-06T22:23:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.022+0000 7f3e727ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e6c0fef30 0x7f3e6c1a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.022+0000 7f3e727ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e6c0ff850 0x7f3e6c1a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.022+0000 7f3e727ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6c111bc0 0x7f3e6c19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.022+0000 7f3e727ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c19da30 con 0x7f3e6c0fef30 2023-10-06T22:23:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.022+0000 7f3e727ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c19dba0 con 0x7f3e6c111bc0 2023-10-06T22:23:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.022+0000 7f3e727ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c19dd60 con 0x7f3e6c0ff850 2023-10-06T22:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e6c0fef30 0x7f3e6c1a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e6c0ff850 0x7f3e6c1a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e6c0ff850 0x7f3e6c1a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36568/0 (socket says 172.21.15.57:36568) 2023-10-06T22:23:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6b7fe700 1 -- 172.21.15.57:0/312705782 learned_addr learned my addr 172.21.15.57:0/312705782 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6bfff700 1 -- 172.21.15.57:0/312705782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e6c0ff850 msgr2=0x7f3e6c1a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e6c0ff850 0x7f3e6c1a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6bfff700 1 -- 172.21.15.57:0/312705782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6c111bc0 msgr2=0x7f3e6c19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6c111bc0 0x7f3e6c19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.023+0000 7f3e6bfff700 1 -- 172.21.15.57:0/312705782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600095f0 con 0x7f3e6c0fef30 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.024+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e6c0fef30 0x7f3e6c1a3560 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00d950 tx=0x7f3e5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.024+0000 7f3e697fa700 1 -- 172.21.15.57:0/312705782 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c009930 con 0x7f3e6c0fef30 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.024+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c19e040 con 0x7f3e6c0fef30 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.024+0000 7f3e697fa700 1 -- 172.21.15.57:0/312705782 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c010460 con 0x7f3e6c0fef30 2023-10-06T22:23:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.024+0000 7f3e697fa700 1 -- 172.21.15.57:0/312705782 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00b5d0 con 0x7f3e6c0fef30 2023-10-06T22:23:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.025+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c1aa550 con 0x7f3e6c0fef30 2023-10-06T22:23:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.029+0000 7f3e697fa700 1 -- 172.21.15.57:0/312705782 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3e5c0107d0 con 0x7f3e6c0fef30 2023-10-06T22:23:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.031+0000 7f3e861d0b80 1 Processor -- start 2023-10-06T22:23:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e861d0b80 1 -- start start 2023-10-06T22:23:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a40e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42699f400 con 0x55c426a36520 2023-10-06T22:23:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426990bc0 con 0x55c426a413d0 2023-10-06T22:23:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42698e4e0 con 0x55c426a35c00 2023-10-06T22:23:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e70d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e697fa700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3e540a16b0 0x7f3e540a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e70d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a35fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36582/0 (socket says 172.21.15.57:36582) 2023-10-06T22:23:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.032+0000 7f3e70d9c700 1 -- 172.21.15.57:0/676497402 learned_addr learned my addr 172.21.15.57:0/676497402 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a40e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e697fa700 1 -- 172.21.15.57:0/312705782 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f3e5c0ccd20 con 0x7f3e6c0fef30 2023-10-06T22:23:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e6bfff700 1 -- 172.21.15.57:0/676497402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 msgr2=0x55c426a35fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a35fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e6bfff700 1 -- 172.21.15.57:0/676497402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 msgr2=0x55c426a437c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:23:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a437c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e6bfff700 1 -- 172.21.15.57:0/676497402 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4267b4d80 con 0x55c426a36520 2023-10-06T22:23:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.033+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a40e90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0ca430 tx=0x7f3e5c06cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d521a8d1d538c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.034+0000 7f3e527fc700 1 -- 172.21.15.57:0/676497402 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c099620 con 0x55c426a36520 2023-10-06T22:23:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.034+0000 7f3e527fc700 1 -- 172.21.15.57:0/676497402 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c065530 con 0x55c426a36520 2023-10-06T22:23:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.034+0000 7f3e527fc700 1 -- 172.21.15.57:0/676497402 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c09de10 con 0x55c426a36520 2023-10-06T22:23:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.034+0000 7f3e70d9c700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3e540a16b0 0x7f3e540a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.035+0000 7f3e861d0b80 1 -- 172.21.15.57:0/676497402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 msgr2=0x55c426a40e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.035+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a40e90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0ca430 tx=0x7f3e5c06cc30 comp rx=0 tx=0).stop 2023-10-06T22:23:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e861d0b80 1 -- 172.21.15.57:0/676497402 shutdown_connections 2023-10-06T22:23:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a35fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/676497402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a40e90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e861d0b80 1 -- 172.21.15.57:0/676497402 >> 172.21.15.57:0/676497402 conn(0x55c4269ab200 msgr2=0x55c426a32080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e70d9c700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3e540a16b0 0x7f3e540a3b70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e640075c0 tx=0x7f3e640097e0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e861d0b80 1 -- 172.21.15.57:0/676497402 shutdown_connections 2023-10-06T22:23:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.036+0000 7f3e861d0b80 1 -- 172.21.15.57:0/676497402 wait complete. 2023-10-06T22:23:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.037+0000 7f3e861d0b80 1 -- start start 2023-10-06T22:23:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.037+0000 7f3e861d0b80 1 Processor -- start 2023-10-06T22:23:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a3e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a3e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a3ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426989dd0 con 0x55c426a36520 2023-10-06T22:23:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4269266b0 con 0x55c426a413d0 2023-10-06T22:23:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426926350 con 0x55c426a35c00 2023-10-06T22:23:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e861d0b80 10 client.0 _ll_register_callbacks cb 0x55c4267837a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a3e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a3e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36590/0 (socket says 172.21.15.57:36590) 2023-10-06T22:23:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e6bfff700 1 -- 172.21.15.57:0/242828144 learned_addr learned my addr 172.21.15.57:0/242828144 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a3e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.038+0000 7f3e70d9c700 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a3ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e6bfff700 1 -- 172.21.15.57:0/242828144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 msgr2=0x55c426a3ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a3ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e6bfff700 1 -- 172.21.15.57:0/242828144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 msgr2=0x55c426a3e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a3e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e6bfff700 1 -- 172.21.15.57:0/242828144 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c1aa550 con 0x55c426a35c00 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a3e310 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0a1b20 tx=0x7f3e5c010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf9ad1f59c69bbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/242828144 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c013250 con 0x55c426a35c00 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/242828144 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f3e5c094070 con 0x55c426a35c00 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.039+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c426a29d50 con 0x55c426a35c00 2023-10-06T22:23:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/242828144 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e5c0105d0 con 0x55c426a35c00 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 10 client.26824 fetch_fsmap learned FSMap version 56 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55c42679dc00 con 0x55c426a35c00 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/242828144 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c0b6020 con 0x55c426a35c00 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/242828144 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f3e5c0991b0 con 0x55c426a35c00 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 10 client.26824 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 10 client.26824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 20 client.26824 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426a50d00 0x55c426a53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 4 client.26824 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:23:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.040+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c426982db0 con 0x55c426a50d00 2023-10-06T22:23:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.041+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426a50d00 0x55c426a53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.041+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426a50d00 0x55c426a53120 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6d670b9d5dee4 server_cookie=57aee2fdd6b6f7f6 in_seq=0 out_seq=0 2023-10-06T22:23:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.041+0000 7f3e4b7fe700 10 client.26824 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.111+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/242828144 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c426982db0 con 0x55c426a50d00 2023-10-06T22:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.111+0000 7f3e4b7fe700 10 client.26824 handle_command_reply: tid=0 2023-10-06T22:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.111+0000 7f3e861d0b80 1 client.26824 shutdown 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 msgr2=0x55c426a3e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a3e310 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0a1b20 tx=0x7f3e5c010bd0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 shutdown_connections 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426a50d00 0x55c426a53120 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a3ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a35c00 0x55c426a3e310 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/242828144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a36520 0x55c426a3e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 >> 172.21.15.57:0/242828144 conn(0x55c4269ab200 msgr2=0x55c426a31c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e4b7fe700 0 client.26824 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 shutdown_connections 2023-10-06T22:23:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 1 -- 172.21.15.57:0/242828144 wait complete. 2023-10-06T22:23:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.112+0000 7f3e861d0b80 20 client.26824 trim_cache size 0 max 16384 2023-10-06T22:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.138+0000 7f3e861d0b80 1 Processor -- start 2023-10-06T22:23:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e861d0b80 1 -- start start 2023-10-06T22:23:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a35c00 0x55c426a4d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42680a8c0 con 0x55c426a35c00 2023-10-06T22:23:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426a29920 con 0x55c426a413d0 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426a3a160 con 0x55c426a36520 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e6b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a35c00 0x55c426a4d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e6b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a4c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:36604/0 (socket says 172.21.15.57:36604) 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.139+0000 7f3e6b7fe700 1 -- 172.21.15.57:0/348478053 learned_addr learned my addr 172.21.15.57:0/348478053 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e70d9c700 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e6b7fe700 1 -- 172.21.15.57:0/348478053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 msgr2=0x55c426a395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e6b7fe700 1 -- 172.21.15.57:0/348478053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a35c00 msgr2=0x55c426a4d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a35c00 0x55c426a4d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e6b7fe700 1 -- 172.21.15.57:0/348478053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4267db1f0 con 0x55c426a36520 2023-10-06T22:23:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a4c670 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000b5c0 tx=0x7f3e60005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=873b031849a68331 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.140+0000 7f3e52ffd700 1 -- 172.21.15.57:0/348478053 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6001d040 con 0x55c426a36520 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e52ffd700 1 -- 172.21.15.57:0/348478053 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e6001e040 con 0x55c426a36520 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e52ffd700 1 -- 172.21.15.57:0/348478053 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60003af0 con 0x55c426a36520 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e861d0b80 1 -- 172.21.15.57:0/348478053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 msgr2=0x55c426a4c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a4c670 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000b5c0 tx=0x7f3e60005fb0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e861d0b80 1 -- 172.21.15.57:0/348478053 shutdown_connections 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a413d0 0x55c426a395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a35c00 0x55c426a4d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.141+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/348478053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a4c670 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.142+0000 7f3e861d0b80 1 -- 172.21.15.57:0/348478053 >> 172.21.15.57:0/348478053 conn(0x55c4269ab200 msgr2=0x55c426a31b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.142+0000 7f3e861d0b80 1 -- 172.21.15.57:0/348478053 shutdown_connections 2023-10-06T22:23:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.142+0000 7f3e861d0b80 1 -- 172.21.15.57:0/348478053 wait complete. 2023-10-06T22:23:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.142+0000 7f3e861d0b80 1 -- start start 2023-10-06T22:23:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.142+0000 7f3e861d0b80 1 Processor -- start 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.143+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a35c00 0x55c426a45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.143+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a45b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.143+0000 7f3e861d0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a413d0 0x55c426a46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.143+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426aa3af0 con 0x55c426a413d0 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.143+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426aa3c30 con 0x55c426a35c00 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.143+0000 7f3e861d0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426aa3dc0 con 0x55c426a36520 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.143+0000 7f3e861d0b80 10 client.0 _ll_register_callbacks cb 0x7f3e6c06a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a35c00 0x55c426a45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a35c00 0x55c426a45600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:34268/0 (socket says 172.21.15.57:34268) 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 -- 172.21.15.57:0/2107392144 learned_addr learned my addr 172.21.15.57:0/2107392144 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 -- 172.21.15.57:0/2107392144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 msgr2=0x55c426a45b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a45b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 -- 172.21.15.57:0/2107392144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a413d0 msgr2=0x55c426a46080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a413d0 0x55c426a46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.144+0000 7f3e6bfff700 1 -- 172.21.15.57:0/2107392144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c114c20 con 0x55c426a35c00 2023-10-06T22:23:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.145+0000 7f3e6bfff700 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a35c00 0x55c426a45600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c005950 tx=0x7f3e5c0996d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0be31b5aa7077c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.145+0000 7f3e4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:23:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.145+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/2107392144 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c09a440 con 0x55c426a35c00 2023-10-06T22:23:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.145+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/2107392144 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f3e5c094040 con 0x55c426a35c00 2023-10-06T22:23:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.145+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e5c065530 con 0x55c426a35c00 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.146+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/2107392144 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e5c013250 con 0x55c426a35c00 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.146+0000 7f3e861d0b80 10 client.26816 fetch_fsmap learned FSMap version 56 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.146+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55c4267b3250 con 0x55c426a35c00 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.146+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/2107392144 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c065530 con 0x55c426a35c00 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.146+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/2107392144 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f3e5c065140 con 0x55c426a35c00 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.147+0000 7f3e861d0b80 10 client.26816 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.147+0000 7f3e861d0b80 10 client.26816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.147+0000 7f3e861d0b80 20 client.26816 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.147+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426aa50c0 0x55c426aa7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.147+0000 7f3e861d0b80 4 client.26816 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:23:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.147+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c426982ad0 con 0x55c426aa50c0 2023-10-06T22:23:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.147+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426aa50c0 0x55c426aa7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.148+0000 7f3e6b7fe700 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426aa50c0 0x55c426aa7520 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f2eb9c66caf1f8 server_cookie=7099894f9da71b39 in_seq=0 out_seq=0 2023-10-06T22:23:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.148+0000 7f3e4b7fe700 10 client.26816 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.860+0000 7f3e4b7fe700 1 -- 172.21.15.57:0/2107392144 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c426982ad0 con 0x55c426aa50c0 2023-10-06T22:23:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.860+0000 7f3e4b7fe700 10 client.26816 handle_command_reply: tid=0 2023-10-06T22:23:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 client.26816 shutdown 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a35c00 msgr2=0x55c426a45600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a35c00 0x55c426a45600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c005950 tx=0x7f3e5c0996d0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 shutdown_connections 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55c426aa50c0 0x55c426aa7520 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c426a413d0 0x55c426a46080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c426a35c00 0x55c426a45600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e4b7fe700 0 client.26816 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 --2- 172.21.15.57:0/2107392144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c426a36520 0x55c426a45b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 >> 172.21.15.57:0/2107392144 conn(0x55c4269ab200 msgr2=0x55c426a395d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 shutdown_connections 2023-10-06T22:23:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.861+0000 7f3e861d0b80 1 -- 172.21.15.57:0/2107392144 wait complete. 2023-10-06T22:23:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e861d0b80 20 client.26816 trim_cache size 0 max 16384 2023-10-06T22:23:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:23:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:23:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:23:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:23:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3e540a16b0 msgr2=0x7f3e540a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3e540a16b0 0x7f3e540a3b70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e640075c0 tx=0x7f3e640097e0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e6c0fef30 msgr2=0x7f3e6c1a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e6c0fef30 0x7f3e6c1a3560 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00d950 tx=0x7f3e5c00dc60 comp rx=0 tx=0).stop 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 shutdown_connections 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3e540a16b0 0x7f3e540a3b70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e6c111bc0 0x7f3e6c19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e6c0fef30 0x7f3e6c1a3560 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 --2- 172.21.15.57:0/312705782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e6c0ff850 0x7f3e6c1a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.862+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 >> 172.21.15.57:0/312705782 conn(0x7f3e6c076e70 msgr2=0x7f3e6c10fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.863+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 shutdown_connections 2023-10-06T22:23:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:00.863+0000 7f3e727ff700 1 -- 172.21.15.57:0/312705782 wait complete. 2023-10-06T22:23:01.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:01 smithi190 ceph-mon[134891]: pgmap v255: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2023-10-06T22:23:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:01 smithi057 ceph-mon[98492]: pgmap v255: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2023-10-06T22:23:01.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:01 smithi156 ceph-mon[134358]: pgmap v255: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2023-10-06T22:23:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:03 smithi190 ceph-mon[134891]: pgmap v256: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 465 op/s 2023-10-06T22:23:03.621 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:23:03.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:03 smithi057 ceph-mon[98492]: pgmap v256: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 465 op/s 2023-10-06T22:23:03.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:03 smithi156 ceph-mon[134358]: pgmap v256: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 465 op/s 2023-10-06T22:23:03.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 -- 172.21.15.57:0/3689470308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 msgr2=0x7fc814111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/3689470308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 0x7fc814111910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc804009990 tx=0x7fc804009ca0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 -- 172.21.15.57:0/3689470308 shutdown_connections 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/3689470308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc814111e50 0x7fc814114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/3689470308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 0x7fc814073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/3689470308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 0x7fc814111910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 -- 172.21.15.57:0/3689470308 >> 172.21.15.57:0/3689470308 conn(0x7fc8140fc930 msgr2=0x7fc8140fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 -- 172.21.15.57:0/3689470308 shutdown_connections 2023-10-06T22:23:03.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.835+0000 7fc81b7ff700 1 -- 172.21.15.57:0/3689470308 wait complete. 2023-10-06T22:23:03.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.836+0000 7fc81b7ff700 1 Processor -- start 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.836+0000 7fc81b7ff700 1 -- start start 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81b7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 0x7fc81419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 0x7fc81419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81b7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc814111e50 0x7fc8141a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81b7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141a15a0 con 0x7fc814111e50 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141a1710 con 0x7fc814073b70 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81b7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141a18d0 con 0x7fc814073250 2023-10-06T22:23:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 0x7fc81419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 0x7fc81419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45044/0 (socket says 172.21.15.57:45044) 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 -- 172.21.15.57:0/667398561 learned_addr learned my addr 172.21.15.57:0/667398561 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc818d9a700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 0x7fc81419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc819d9c700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc814111e50 0x7fc8141a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 -- 172.21.15.57:0/667398561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 msgr2=0x7fc81419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 0x7fc81419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 -- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc814111e50 msgr2=0x7fc8141a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc814111e50 0x7fc8141a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.837+0000 7fc81959b700 1 -- 172.21.15.57:0/667398561 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810009520 con 0x7fc814073250 2023-10-06T22:23:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.838+0000 7fc81959b700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 0x7fc81419d6b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc81000eb30 tx=0x7fc81000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.838+0000 7fc80a7fc700 1 -- 172.21.15.57:0/667398561 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810009880 con 0x7fc814073250 2023-10-06T22:23:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.838+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8040095f0 con 0x7fc814073250 2023-10-06T22:23:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.838+0000 7fc80a7fc700 1 -- 172.21.15.57:0/667398561 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8100044e0 con 0x7fc814073250 2023-10-06T22:23:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.838+0000 7fc80a7fc700 1 -- 172.21.15.57:0/667398561 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810003d80 con 0x7fc814073250 2023-10-06T22:23:03.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.838+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8141aa330 con 0x7fc814073250 2023-10-06T22:23:03.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.841+0000 7fc80a7fc700 1 -- 172.21.15.57:0/667398561 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc810003f20 con 0x7fc814073250 2023-10-06T22:23:03.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.842+0000 7fc82f19fb80 1 Processor -- start 2023-10-06T22:23:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc82f19fb80 1 -- start start 2023-10-06T22:23:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc80a7fc700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc8000a15f0 0x7fc8000a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb20c0 0x559d6bcb24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcb29e0 0x559d6bcbd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcbd890 0x559d6bcbfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bc1b460 con 0x559d6bcbd890 2023-10-06T22:23:03.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bc0cc20 con 0x559d6bcb29e0 2023-10-06T22:23:03.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bc0a540 con 0x559d6bcb20c0 2023-10-06T22:23:03.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc80a7fc700 1 -- 172.21.15.57:0/667398561 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fc810014070 con 0x7fc814073250 2023-10-06T22:23:03.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcb29e0 0x559d6bcbd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcb29e0 0x559d6bcbd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53312/0 (socket says 172.21.15.57:53312) 2023-10-06T22:23:03.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc818d9a700 1 -- 172.21.15.57:0/2939105691 learned_addr learned my addr 172.21.15.57:0/2939105691 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:03.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc81959b700 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb20c0 0x559d6bcb24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc819d9c700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc8000a15f0 0x7fc8000a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc819d9c700 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcbd890 0x559d6bcbfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc818d9a700 1 -- 172.21.15.57:0/2939105691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb20c0 msgr2=0x559d6bcb24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.843+0000 7fc818d9a700 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb20c0 0x559d6bcb24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc818d9a700 1 -- 172.21.15.57:0/2939105691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcbd890 msgr2=0x559d6bcbfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc818d9a700 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcbd890 0x559d6bcbfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc818d9a700 1 -- 172.21.15.57:0/2939105691 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6ba30de0 con 0x559d6bcb29e0 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc819d9c700 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcbd890 0x559d6bcbfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:23:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc818d9a700 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcb29e0 0x559d6bcbd350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc804009990 tx=0x7fc804005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a38b0e316e9b45a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:03.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc819d9c700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc8000a15f0 0x7fc8000a3ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009ec0 tx=0x7fc80c009920 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc7ff7fe700 1 -- 172.21.15.57:0/2939105691 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80401d070 con 0x559d6bcb29e0 2023-10-06T22:23:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc7ff7fe700 1 -- 172.21.15.57:0/2939105691 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80401e040 con 0x559d6bcb29e0 2023-10-06T22:23:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.844+0000 7fc7ff7fe700 1 -- 172.21.15.57:0/2939105691 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8040048f0 con 0x559d6bcb29e0 2023-10-06T22:23:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 -- 172.21.15.57:0/2939105691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcb29e0 msgr2=0x559d6bcbd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcb29e0 0x559d6bcbd350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc804009990 tx=0x7fc804005ea0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 -- 172.21.15.57:0/2939105691 shutdown_connections 2023-10-06T22:23:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcbd890 0x559d6bcbfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb20c0 0x559d6bcb24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/2939105691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcb29e0 0x559d6bcbd350 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 -- 172.21.15.57:0/2939105691 >> 172.21.15.57:0/2939105691 conn(0x559d6bc27260 msgr2=0x559d6bcae450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 -- 172.21.15.57:0/2939105691 shutdown_connections 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.845+0000 7fc82f19fb80 1 -- 172.21.15.57:0/2939105691 wait complete. 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.846+0000 7fc82f19fb80 1 -- start start 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.846+0000 7fc82f19fb80 1 Processor -- start 2023-10-06T22:23:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcbd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcbac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bc05e30 con 0x559d6bcb20c0 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bba2710 con 0x559d6bcbd890 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bba23b0 con 0x559d6bcb29e0 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc82f19fb80 10 client.0 _ll_register_callbacks cb 0x559d6b9ff800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc819d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcbac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcba4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45068/0 (socket says 172.21.15.57:45068) 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 -- 172.21.15.57:0/1220776160 learned_addr learned my addr 172.21.15.57:0/1220776160 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc81959b700 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcbd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 -- 172.21.15.57:0/1220776160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 msgr2=0x559d6bcbac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcbac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 -- 172.21.15.57:0/1220776160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 msgr2=0x559d6bcbd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcbd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.847+0000 7fc818d9a700 1 -- 172.21.15.57:0/1220776160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc814114bf0 con 0x559d6bcb29e0 2023-10-06T22:23:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc818d9a700 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcba4d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc804009ce0 tx=0x7fc80400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f21219c53c71de9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc7fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:23:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc7fcff9700 1 -- 172.21.15.57:0/1220776160 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80401d040 con 0x559d6bcb29e0 2023-10-06T22:23:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d6bca5f10 con 0x559d6bcb29e0 2023-10-06T22:23:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc7fcff9700 1 -- 172.21.15.57:0/1220776160 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fc80400fc90 con 0x559d6bcb29e0 2023-10-06T22:23:03.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc7fcff9700 1 -- 172.21.15.57:0/1220776160 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc804005bc0 con 0x559d6bcb29e0 2023-10-06T22:23:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc82f19fb80 10 client.26839 fetch_fsmap learned FSMap version 56 2023-10-06T22:23:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.848+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x559d6ba19c60 con 0x559d6bcb29e0 2023-10-06T22:23:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc7fcff9700 1 -- 172.21.15.57:0/1220776160 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d6bca5f10 con 0x559d6bcb29e0 2023-10-06T22:23:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc7fcff9700 1 -- 172.21.15.57:0/1220776160 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fc80402b3f0 con 0x559d6bcb29e0 2023-10-06T22:23:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc82f19fb80 10 client.26839 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T22:23:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc82f19fb80 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc82f19fb80 20 client.26839 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bccceb0 0x559d6bccf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc82f19fb80 4 client.26839 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:23:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d6bbfee10 con 0x559d6bccceb0 2023-10-06T22:23:03.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.849+0000 7fc81959b700 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bccceb0 0x559d6bccf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.850+0000 7fc81959b700 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bccceb0 0x559d6bccf2d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e110496de44bfb25 server_cookie=191b390eb01186be in_seq=0 out_seq=0 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.850+0000 7fc7fcff9700 10 client.26839 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.850+0000 7fc7fcff9700 1 -- 172.21.15.57:0/1220776160 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d6bbfee10 con 0x559d6bccceb0 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.850+0000 7fc7fcff9700 10 client.26839 handle_command_reply: tid=0 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.850+0000 7fc82f19fb80 1 client.26839 shutdown 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 msgr2=0x559d6bcba4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcba4d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc804009ce0 tx=0x7fc80400f690 comp rx=0 tx=0).stop 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 shutdown_connections 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bccceb0 0x559d6bccf2d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcbac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcbd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1220776160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcba4d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc7fcff9700 0 client.26839 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:03.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 >> 172.21.15.57:0/1220776160 conn(0x559d6bc27260 msgr2=0x559d6bcae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:03.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 shutdown_connections 2023-10-06T22:23:03.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1220776160 wait complete. 2023-10-06T22:23:03.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.851+0000 7fc82f19fb80 20 client.26839 trim_cache size 0 max 16384 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.868+0000 7fc82f19fb80 1 Processor -- start 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.868+0000 7fc82f19fb80 1 -- start start 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcc43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcbcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6ba86920 con 0x559d6bcb20c0 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bcc4910 con 0x559d6bcbd890 2023-10-06T22:23:03.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bcbb210 con 0x559d6bcb29e0 2023-10-06T22:23:03.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc819d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc81959b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcc43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcbcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc819d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53326/0 (socket says 172.21.15.57:53326) 2023-10-06T22:23:03.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcbcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45070/0 (socket says 172.21.15.57:45070) 2023-10-06T22:23:03.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc819d9c700 1 -- 172.21.15.57:0/1893965154 learned_addr learned my addr 172.21.15.57:0/1893965154 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc818d9a700 1 -- 172.21.15.57:0/1893965154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 msgr2=0x559d6bcb9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc818d9a700 1 --2- 172.21.15.57:0/1893965154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc818d9a700 1 -- 172.21.15.57:0/1893965154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 msgr2=0x559d6bcc43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc818d9a700 1 --2- 172.21.15.57:0/1893965154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcc43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.869+0000 7fc818d9a700 1 -- 172.21.15.57:0/1893965154 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6ba57250 con 0x559d6bcb29e0 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc818d9a700 1 --2- 172.21.15.57:0/1893965154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcbcdf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc80402b780 tx=0x7fc804000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=180036bf0c142c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc7fffff700 1 -- 172.21.15.57:0/1893965154 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804005bc0 con 0x559d6bcb29e0 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc7fffff700 1 -- 172.21.15.57:0/1893965154 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80400f700 con 0x559d6bcb29e0 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc7fffff700 1 -- 172.21.15.57:0/1893965154 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804022640 con 0x559d6bcb29e0 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1893965154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 msgr2=0x559d6bcbcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1893965154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcbcdf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc80402b780 tx=0x7fc804000bc0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1893965154 shutdown_connections 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1893965154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1893965154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/1893965154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcbcdf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.870+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1893965154 >> 172.21.15.57:0/1893965154 conn(0x559d6bc27260 msgr2=0x559d6bcc95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.871+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1893965154 shutdown_connections 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.871+0000 7fc82f19fb80 1 -- 172.21.15.57:0/1893965154 wait complete. 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.871+0000 7fc82f19fb80 1 -- start start 2023-10-06T22:23:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.871+0000 7fc82f19fb80 1 Processor -- start 2023-10-06T22:23:03.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcb8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcb68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc82f19fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bcb7500 con 0x559d6bcb20c0 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bcb7670 con 0x559d6bcbd890 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc82f19fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6bd20ca0 con 0x559d6bcb29e0 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc82f19fb80 10 client.0 _ll_register_callbacks cb 0x7fc810096230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc819d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcb68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcb68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45082/0 (socket says 172.21.15.57:45082) 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 -- 172.21.15.57:0/3942481950 learned_addr learned my addr 172.21.15.57:0/3942481950 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc81959b700 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcb8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 -- 172.21.15.57:0/3942481950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 msgr2=0x559d6bcb6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 -- 172.21.15.57:0/3942481950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 msgr2=0x559d6bcb8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcb8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.872+0000 7fc818d9a700 1 -- 172.21.15.57:0/3942481950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d6ba19c60 con 0x559d6bcb29e0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc818d9a700 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcb68b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc80403a040 tx=0x7fc804009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c8b9ab6f0ad135 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc7fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc7fcff9700 1 -- 172.21.15.57:0/3942481950 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80401d030 con 0x559d6bcb29e0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc7fcff9700 1 -- 172.21.15.57:0/3942481950 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fc804000e40 con 0x559d6bcb29e0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d6bbfeb30 con 0x559d6bcb29e0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc7fcff9700 1 -- 172.21.15.57:0/3942481950 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc80400fa90 con 0x559d6bcb29e0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc82f19fb80 10 client.26845 fetch_fsmap learned FSMap version 56 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x559d6ba2f2b0 con 0x559d6bcb29e0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.873+0000 7fc7fcff9700 1 -- 172.21.15.57:0/3942481950 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc804055020 con 0x559d6bcb29e0 2023-10-06T22:23:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc7fcff9700 1 -- 172.21.15.57:0/3942481950 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fc804051b30 con 0x559d6bcb29e0 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc82f19fb80 10 client.26845 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc82f19fb80 10 client.26845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc82f19fb80 20 client.26845 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bd21ff0 0x559d6bd24410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc82f19fb80 4 client.26845 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559d6ba51f70 con 0x559d6bd21ff0 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc81959b700 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bd21ff0 0x559d6bd24410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc81959b700 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bd21ff0 0x559d6bd24410 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cadf75b0ff5ac3 server_cookie=cf4c518ff9d132b in_seq=0 out_seq=0 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.874+0000 7fc7fcff9700 10 client.26845 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc7fcff9700 1 -- 172.21.15.57:0/3942481950 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559d6ba51f70 con 0x559d6bd21ff0 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc7fcff9700 10 client.26845 handle_command_reply: tid=0 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 client.26845 shutdown 2023-10-06T22:23:03.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 msgr2=0x559d6bcb68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcb68b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc80403a040 tx=0x7fc804009950 comp rx=0 tx=0).stop 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 shutdown_connections 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x559d6bd21ff0 0x559d6bd24410 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d6bcbd890 0x559d6bcb6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d6bcb20c0 0x559d6bcb8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 --2- 172.21.15.57:0/3942481950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559d6bcb29e0 0x559d6bcb68b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc7fcff9700 0 client.26845 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 >> 172.21.15.57:0/3942481950 conn(0x559d6bc27260 msgr2=0x559d6bcb9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 shutdown_connections 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 1 -- 172.21.15.57:0/3942481950 wait complete. 2023-10-06T22:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.875+0000 7fc82f19fb80 20 client.26845 trim_cache size 0 max 16384 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:23:03.882 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:23:03.883 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:23:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc8000a15f0 msgr2=0x7fc8000a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc8000a15f0 0x7fc8000a3ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009ec0 tx=0x7fc80c009920 comp rx=0 tx=0).stop 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 msgr2=0x7fc81419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 0x7fc81419d6b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc81000eb30 tx=0x7fc81000ee40 comp rx=0 tx=0).stop 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 shutdown_connections 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc8000a15f0 0x7fc8000a3ab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc814111e50 0x7fc8141a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc814073250 0x7fc81419d6b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 --2- 172.21.15.57:0/667398561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc814073b70 0x7fc81419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 >> 172.21.15.57:0/667398561 conn(0x7fc8140fc930 msgr2=0x7fc8140fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 shutdown_connections 2023-10-06T22:23:03.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:03.876+0000 7fc81b7ff700 1 -- 172.21.15.57:0/667398561 wait complete. 2023-10-06T22:23:03.895 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:23:05.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:05 smithi190 ceph-mon[134891]: pgmap v257: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 373 op/s 2023-10-06T22:23:05.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:05 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:23:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:05 smithi057 ceph-mon[98492]: pgmap v257: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 373 op/s 2023-10-06T22:23:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:05 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:23:05.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:05 smithi156 ceph-mon[134358]: pgmap v257: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 373 op/s 2023-10-06T22:23:05.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:05 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:23:07.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:07 smithi057 ceph-mon[98492]: pgmap v258: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 437 op/s 2023-10-06T22:23:07.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:07 smithi156 ceph-mon[134358]: pgmap v258: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 437 op/s 2023-10-06T22:23:07.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:07 smithi190 ceph-mon[134891]: pgmap v258: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 437 op/s 2023-10-06T22:23:09.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:09 smithi156 ceph-mon[134358]: pgmap v259: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 419 op/s 2023-10-06T22:23:09.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:09 smithi190 ceph-mon[134891]: pgmap v259: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 419 op/s 2023-10-06T22:23:09.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:09 smithi057 ceph-mon[98492]: pgmap v259: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 419 op/s 2023-10-06T22:23:11.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:11 smithi190 ceph-mon[134891]: pgmap v260: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2023-10-06T22:23:11.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:11 smithi190 ceph-mon[134891]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T22:23:11.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:11 smithi057 ceph-mon[98492]: pgmap v260: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2023-10-06T22:23:11.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:11 smithi057 ceph-mon[98492]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T22:23:11.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:11 smithi156 ceph-mon[134358]: pgmap v260: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2023-10-06T22:23:11.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:11 smithi156 ceph-mon[134358]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T22:23:12.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:12 smithi190 ceph-mon[134891]: scrub summary: active paths [/] 2023-10-06T22:23:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:12 smithi057 ceph-mon[98492]: scrub summary: active paths [/] 2023-10-06T22:23:12.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:12 smithi156 ceph-mon[134358]: scrub summary: active paths [/] 2023-10-06T22:23:13.574 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:13 smithi190 ceph-mon[134891]: pgmap v262: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2023-10-06T22:23:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:13 smithi057 ceph-mon[98492]: pgmap v262: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2023-10-06T22:23:13.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:13 smithi156 ceph-mon[134358]: pgmap v262: 97 pgs: 17 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 51 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2023-10-06T22:23:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.099+0000 7f248afff700 1 -- 172.21.15.57:0/4135253276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102980 msgr2=0x7f248410ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.099+0000 7f248afff700 1 --2- 172.21.15.57:0/4135253276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102980 0x7f248410ae60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f246c009990 tx=0x7f246c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.099+0000 7f248afff700 1 -- 172.21.15.57:0/4135253276 shutdown_connections 2023-10-06T22:23:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.099+0000 7f248afff700 1 --2- 172.21.15.57:0/4135253276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248410b480 0x7f248410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.099+0000 7f248afff700 1 --2- 172.21.15.57:0/4135253276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2484102060 0x7f2484102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.099+0000 7f248afff700 1 --2- 172.21.15.57:0/4135253276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102980 0x7f248410ae60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.099+0000 7f248afff700 1 -- 172.21.15.57:0/4135253276 >> 172.21.15.57:0/4135253276 conn(0x7f24840fb860 msgr2=0x7f24840fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.100+0000 7f248afff700 1 -- 172.21.15.57:0/4135253276 shutdown_connections 2023-10-06T22:23:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.100+0000 7f248afff700 1 -- 172.21.15.57:0/4135253276 wait complete. 2023-10-06T22:23:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.101+0000 7f248afff700 1 Processor -- start 2023-10-06T22:23:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f248afff700 1 -- start start 2023-10-06T22:23:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f248afff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102060 0x7f24841991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f248afff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2484102980 0x7f2484199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f248afff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248410b480 0x7f248419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f248afff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e3b0 con 0x7f2484102980 2023-10-06T22:23:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f248afff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e520 con 0x7f2484102060 2023-10-06T22:23:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f248afff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e6e0 con 0x7f248410b480 2023-10-06T22:23:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f2488d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102060 0x7f24841991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f2488d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102060 0x7f24841991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53780/0 (socket says 172.21.15.57:53780) 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.102+0000 7f2488d9b700 1 -- 172.21.15.57:0/4038045593 learned_addr learned my addr 172.21.15.57:0/4038045593 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f248959c700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248410b480 0x7f248419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f2488d9b700 1 -- 172.21.15.57:0/4038045593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248410b480 msgr2=0x7f248419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f2488d9b700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248410b480 0x7f248419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f2488d9b700 1 -- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2484102980 msgr2=0x7f2484199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f2488d9b700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2484102980 0x7f2484199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f2488d9b700 1 -- 172.21.15.57:0/4038045593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c0095f0 con 0x7f2484102060 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f2488d9b700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102060 0x7f24841991c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f247400ef10 tx=0x7f247400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f24827fc700 1 -- 172.21.15.57:0/4038045593 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247400cad0 con 0x7f2484102060 2023-10-06T22:23:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f24827fc700 1 -- 172.21.15.57:0/4038045593 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f247400f040 con 0x7f2484102060 2023-10-06T22:23:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f24827fc700 1 -- 172.21.15.57:0/4038045593 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24740146e0 con 0x7f2484102060 2023-10-06T22:23:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.103+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248419e960 con 0x7f2484102060 2023-10-06T22:23:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.104+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24841a6200 con 0x7f2484102060 2023-10-06T22:23:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.109+0000 7f24827fc700 1 -- 172.21.15.57:0/4038045593 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2474007500 con 0x7f2484102060 2023-10-06T22:23:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.110+0000 7f249e9c9b80 1 Processor -- start 2023-10-06T22:23:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.110+0000 7f249e9c9b80 1 -- start start 2023-10-06T22:23:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f8bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f96e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322ef5400 con 0x557322f8c520 2023-10-06T22:23:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322ee6bc0 con 0x557322f8bc00 2023-10-06T22:23:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322ee44e0 con 0x557322f973d0 2023-10-06T22:23:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f248959c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f8bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f248959c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f8bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53792/0 (socket says 172.21.15.57:53792) 2023-10-06T22:23:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.111+0000 7f248959c700 1 -- 172.21.15.57:0/2764933132 learned_addr learned my addr 172.21.15.57:0/2764933132 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f247bfff700 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f2488d9b700 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f96e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f248959c700 1 -- 172.21.15.57:0/2764933132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 msgr2=0x557322f997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f248959c700 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f248959c700 1 -- 172.21.15.57:0/2764933132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 msgr2=0x557322f96e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f248959c700 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f96e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f248959c700 1 -- 172.21.15.57:0/2764933132 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557322d0ad80 con 0x557322f8bc00 2023-10-06T22:23:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f2488d9b700 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f96e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:23:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.112+0000 7f24827fc700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f24700a1740 0x7f24700a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.113+0000 7f24827fc700 1 -- 172.21.15.57:0/4038045593 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f24740cd4b0 con 0x7f2484102060 2023-10-06T22:23:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.113+0000 7f2488d9b700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f24700a1740 0x7f24700a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.114+0000 7f2488d9b700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f24700a1740 0x7f24700a3c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24740caca0 tx=0x7f2474007c40 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.114+0000 7f248959c700 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f8bfe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f247c009c80 tx=0x7f247c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e830293fc323462 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.116+0000 7f247affd700 1 -- 172.21.15.57:0/2764933132 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0097d0 con 0x557322f8bc00 2023-10-06T22:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.117+0000 7f247affd700 1 -- 172.21.15.57:0/2764933132 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f247c015070 con 0x557322f8bc00 2023-10-06T22:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.117+0000 7f247affd700 1 -- 172.21.15.57:0/2764933132 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c011680 con 0x557322f8bc00 2023-10-06T22:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2764933132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 msgr2=0x557322f8bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f8bfe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f247c009c80 tx=0x7f247c009f90 comp rx=0 tx=0).stop 2023-10-06T22:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2764933132 shutdown_connections 2023-10-06T22:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f8bfe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2764933132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f96e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2764933132 >> 172.21.15.57:0/2764933132 conn(0x557322f01200 msgr2=0x557322f88080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2764933132 shutdown_connections 2023-10-06T22:23:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.118+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2764933132 wait complete. 2023-10-06T22:23:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.119+0000 7f249e9c9b80 1 -- start start 2023-10-06T22:23:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.119+0000 7f249e9c9b80 1 Processor -- start 2023-10-06T22:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.120+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f94190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.120+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.120+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f94c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.120+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322edfdd0 con 0x557322f8c520 2023-10-06T22:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.120+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322e7c6b0 con 0x557322f8bc00 2023-10-06T22:23:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.120+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322e7c350 con 0x557322f973d0 2023-10-06T22:23:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.120+0000 7f249e9c9b80 10 client.0 _ll_register_callbacks cb 0x557322cd97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f248959c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f94c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f248959c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f94c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45268/0 (socket says 172.21.15.57:45268) 2023-10-06T22:23:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f248959c700 1 -- 172.21.15.57:0/1018138806 learned_addr learned my addr 172.21.15.57:0/1018138806 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f247bfff700 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f248959c700 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f94190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f247bfff700 1 -- 172.21.15.57:0/1018138806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 msgr2=0x557322f94c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f247bfff700 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f94c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f247bfff700 1 -- 172.21.15.57:0/1018138806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 msgr2=0x557322f94190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f247bfff700 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f94190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.121+0000 7f247bfff700 1 -- 172.21.15.57:0/1018138806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c009930 con 0x557322f8c520 2023-10-06T22:23:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.122+0000 7f247bfff700 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f946d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f246c006010 tx=0x7f246c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fdc68a85f4f916 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.122+0000 7f2463fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:23:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.122+0000 7f2463fff700 1 -- 172.21.15.57:0/1018138806 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c004500 con 0x557322f8c520 2023-10-06T22:23:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.123+0000 7f2463fff700 1 -- 172.21.15.57:0/1018138806 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f246c004660 con 0x557322f8c520 2023-10-06T22:23:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.123+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557322f7fd50 con 0x557322f8c520 2023-10-06T22:23:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.123+0000 7f2463fff700 1 -- 172.21.15.57:0/1018138806 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f246c004820 con 0x557322f8c520 2023-10-06T22:23:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.123+0000 7f249e9c9b80 10 client.16812 fetch_fsmap learned FSMap version 57 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.123+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x557322cf3c00 con 0x557322f8c520 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.124+0000 7f2463fff700 1 -- 172.21.15.57:0/1018138806 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557322f7fd50 con 0x557322f8c520 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.124+0000 7f2463fff700 1 -- 172.21.15.57:0/1018138806 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 57) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f246c02a070 con 0x557322f8c520 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.125+0000 7f249e9c9b80 10 client.16812 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.125+0000 7f249e9c9b80 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.125+0000 7f249e9c9b80 20 client.16812 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.125+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557322faf6d0 0x557322fb1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.125+0000 7f249e9c9b80 4 client.16812 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:23:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.125+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557322ed8db0 con 0x557322faf6d0 2023-10-06T22:23:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.125+0000 7f248959c700 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557322faf6d0 0x557322fb1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.127+0000 7f248959c700 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557322faf6d0 0x557322fb1af0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bba209262791200 server_cookie=2eecdfdfc7090a2f in_seq=0 out_seq=0 2023-10-06T22:23:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.128+0000 7f2463fff700 10 client.16812 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.131+0000 7f2463fff700 1 -- 172.21.15.57:0/1018138806 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557322ed8db0 con 0x557322faf6d0 2023-10-06T22:23:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.131+0000 7f2463fff700 10 client.16812 handle_command_reply: tid=0 2023-10-06T22:23:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.133+0000 7f249e9c9b80 1 client.16812 shutdown 2023-10-06T22:23:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 msgr2=0x557322f946d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f946d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f246c006010 tx=0x7f246c00beb0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 shutdown_connections 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557322faf6d0 0x557322fb1af0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557322f94c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322f94190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/1018138806 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322f946d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 >> 172.21.15.57:0/1018138806 conn(0x557322f01200 msgr2=0x557322f87c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.134+0000 7f2463fff700 0 client.16812 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.135+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 shutdown_connections 2023-10-06T22:23:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.135+0000 7f249e9c9b80 1 -- 172.21.15.57:0/1018138806 wait complete. 2023-10-06T22:23:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.137+0000 7f249e9c9b80 20 client.16812 trim_cache size 0 max 16384 2023-10-06T22:23:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.171+0000 7f249e9c9b80 1 Processor -- start 2023-10-06T22:23:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.172+0000 7f249e9c9b80 1 -- start start 2023-10-06T22:23:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.172+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8bc00 0x557322fa6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.172+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f8c520 0x557322f8eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.172+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f973d0 0x557322f94740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.172+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322d608c0 con 0x557322f8bc00 2023-10-06T22:23:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.172+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322fa6d20 con 0x557322f973d0 2023-10-06T22:23:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.172+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322f8f060 con 0x557322f8c520 2023-10-06T22:23:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.173+0000 7f247bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f8c520 0x557322f8eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.173+0000 7f247bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f8c520 0x557322f8eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45284/0 (socket says 172.21.15.57:45284) 2023-10-06T22:23:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.173+0000 7f247bfff700 1 -- 172.21.15.57:0/2815154168 learned_addr learned my addr 172.21.15.57:0/2815154168 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.173+0000 7f248959c700 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8bc00 0x557322fa6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.174+0000 7f248959c700 1 -- 172.21.15.57:0/2815154168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f8c520 msgr2=0x557322f8eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.174+0000 7f248959c700 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f8c520 0x557322f8eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.174+0000 7f248959c700 1 -- 172.21.15.57:0/2815154168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f973d0 msgr2=0x557322f94740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.174+0000 7f248959c700 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f973d0 0x557322f94740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.174+0000 7f248959c700 1 -- 172.21.15.57:0/2815154168 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557322d311f0 con 0x557322f8bc00 2023-10-06T22:23:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.174+0000 7f248959c700 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8bc00 0x557322fa6770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x557322fb23a0 tx=0x7f247c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182f1cd2995e66ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.176+0000 7f247b7fe700 1 -- 172.21.15.57:0/2815154168 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c023070 con 0x557322f8bc00 2023-10-06T22:23:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.176+0000 7f247b7fe700 1 -- 172.21.15.57:0/2815154168 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f247c02b340 con 0x557322f8bc00 2023-10-06T22:23:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.176+0000 7f247b7fe700 1 -- 172.21.15.57:0/2815154168 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c01c8f0 con 0x557322f8bc00 2023-10-06T22:23:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.178+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2815154168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8bc00 msgr2=0x557322fa6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.178+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8bc00 0x557322fa6770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x557322fb23a0 tx=0x7f247c009260 comp rx=0 tx=0).stop 2023-10-06T22:23:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.179+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2815154168 shutdown_connections 2023-10-06T22:23:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.179+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f973d0 0x557322f94740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.179+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8bc00 0x557322fa6770 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.180+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/2815154168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f8c520 0x557322f8eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.180+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2815154168 >> 172.21.15.57:0/2815154168 conn(0x557322f01200 msgr2=0x557322f8f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.183+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2815154168 shutdown_connections 2023-10-06T22:23:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.184+0000 7f249e9c9b80 1 -- 172.21.15.57:0/2815154168 wait complete. 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.184+0000 7f249e9c9b80 1 -- start start 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.184+0000 7f249e9c9b80 1 Processor -- start 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.186+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322ff7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.186+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322ff7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f249e9c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557323002510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322ff8290 con 0x557322f8c520 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323002a50 con 0x557322f8bc00 2023-10-06T22:23:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f249e9c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323002b90 con 0x557322f973d0 2023-10-06T22:23:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f249e9c9b80 10 client.0 _ll_register_callbacks cb 0x7f24740097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f247bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322ff7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f248959c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322ff7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f247bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322ff7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52938/0 (socket says 172.21.15.57:52938) 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.187+0000 7f247bfff700 1 -- 172.21.15.57:0/4023342265 learned_addr learned my addr 172.21.15.57:0/4023342265 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.188+0000 7f247bfff700 1 -- 172.21.15.57:0/4023342265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 msgr2=0x557323002510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.188+0000 7f247bfff700 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557323002510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.188+0000 7f247bfff700 1 -- 172.21.15.57:0/4023342265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 msgr2=0x557322ff7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.188+0000 7f247bfff700 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322ff7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.188+0000 7f247bfff700 1 -- 172.21.15.57:0/4023342265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c026040 con 0x557322f8c520 2023-10-06T22:23:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.188+0000 7f248959c700 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322ff7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.189+0000 7f247bfff700 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322ff7bc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f246c006010 tx=0x7f246c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5550b05af3cad0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.189+0000 7f2463fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:23:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.189+0000 7f2463fff700 1 -- 172.21.15.57:0/4023342265 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c004910 con 0x557322f8c520 2023-10-06T22:23:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.189+0000 7f2463fff700 1 -- 172.21.15.57:0/4023342265 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f246c000e40 con 0x557322f8c520 2023-10-06T22:23:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.189+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f247c015070 con 0x557322f8c520 2023-10-06T22:23:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.190+0000 7f2463fff700 1 -- 172.21.15.57:0/4023342265 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f246c0058e0 con 0x557322f8c520 2023-10-06T22:23:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.191+0000 7f249e9c9b80 10 client.16818 fetch_fsmap learned FSMap version 57 2023-10-06T22:23:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.191+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x557322d09250 con 0x557322f8c520 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.193+0000 7f2463fff700 1 -- 172.21.15.57:0/4023342265 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f246c042080 con 0x557322f8c520 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.193+0000 7f2463fff700 1 -- 172.21.15.57:0/4023342265 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 57) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f246c02a070 con 0x557322f8c520 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.194+0000 7f249e9c9b80 10 client.16818 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.194+0000 7f249e9c9b80 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.194+0000 7f249e9c9b80 20 client.16818 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.194+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5573230040b0 0x5573230064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.194+0000 7f249e9c9b80 4 client.16818 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:23:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.194+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557322ed8ad0 con 0x5573230040b0 2023-10-06T22:23:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.195+0000 7f248959c700 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5573230040b0 0x5573230064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.195+0000 7f248959c700 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5573230040b0 0x5573230064d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ebf1e18afbfc5f server_cookie=7a80efa463e838f4 in_seq=0 out_seq=0 2023-10-06T22:23:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:14.195+0000 7f2463fff700 10 client.16818 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:15.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:15 smithi057 ceph-mon[98492]: pgmap v263: 97 pgs: 18 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 50 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2023-10-06T22:23:15.759 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:15 smithi156 ceph-mon[134358]: pgmap v263: 97 pgs: 18 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 50 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2023-10-06T22:23:15.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:15 smithi190 ceph-mon[134891]: pgmap v263: 97 pgs: 18 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 50 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2023-10-06T22:23:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.399+0000 7f2463fff700 1 -- 172.21.15.57:0/4023342265 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557322ed8ad0 con 0x5573230040b0 2023-10-06T22:23:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.399+0000 7f2463fff700 10 client.16818 handle_command_reply: tid=0 2023-10-06T22:23:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.399+0000 7f249e9c9b80 1 client.16818 shutdown 2023-10-06T22:23:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 msgr2=0x557322ff7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322ff7bc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f246c006010 tx=0x7f246c0057e0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 shutdown_connections 2023-10-06T22:23:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5573230040b0 0x5573230064d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557322f973d0 0x557323002510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322f8bc00 0x557322ff7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 --2- 172.21.15.57:0/4023342265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557322f8c520 0x557322ff7bc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 >> 172.21.15.57:0/4023342265 conn(0x557322f01200 msgr2=0x557322f94740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.400+0000 7f2463fff700 0 client.16818 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.401+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 shutdown_connections 2023-10-06T22:23:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.401+0000 7f249e9c9b80 1 -- 172.21.15.57:0/4023342265 wait complete. 2023-10-06T22:23:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.402+0000 7f249e9c9b80 20 client.16818 trim_cache size 0 max 16384 2023-10-06T22:23:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:23:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:23:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:23:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:23:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.403+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f24700a1740 msgr2=0x7f24700a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.403+0000 7f248afff700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f24700a1740 0x7f24700a3c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24740caca0 tx=0x7f2474007c40 comp rx=0 tx=0).stop 2023-10-06T22:23:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.403+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102060 msgr2=0x7f24841991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.403+0000 7f248afff700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102060 0x7f24841991c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f247400ef10 tx=0x7f247400c3a0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.403+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 shutdown_connections 2023-10-06T22:23:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.404+0000 7f248afff700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f24700a1740 0x7f24700a3c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.404+0000 7f248afff700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f248410b480 0x7f248419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.404+0000 7f248afff700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2484102060 0x7f24841991c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.404+0000 7f248afff700 1 --2- 172.21.15.57:0/4038045593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2484102980 0x7f2484199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.404+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 >> 172.21.15.57:0/4038045593 conn(0x7f24840fb860 msgr2=0x7f248410d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.404+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 shutdown_connections 2023-10-06T22:23:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:16.404+0000 7f248afff700 1 -- 172.21.15.57:0/4038045593 wait complete. 2023-10-06T22:23:17.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:17 smithi057 ceph-mon[98492]: pgmap v264: 97 pgs: 21 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 41 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2023-10-06T22:23:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:17 smithi156 ceph-mon[134358]: pgmap v264: 97 pgs: 21 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 41 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2023-10-06T22:23:17.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:17 smithi190 ceph-mon[134891]: pgmap v264: 97 pgs: 21 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 41 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2023-10-06T22:23:19.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:19 smithi190 ceph-mon[134891]: pgmap v265: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.19k op/s 2023-10-06T22:23:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:19 smithi057 ceph-mon[98492]: pgmap v265: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.19k op/s 2023-10-06T22:23:19.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:19 smithi156 ceph-mon[134358]: pgmap v265: 97 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 64 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.19k op/s 2023-10-06T22:23:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:21 smithi190 ceph-mon[134891]: pgmap v266: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2023-10-06T22:23:21.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:21 smithi057 ceph-mon[98492]: pgmap v266: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2023-10-06T22:23:21.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:21 smithi156 ceph-mon[134358]: pgmap v266: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2023-10-06T22:23:23.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:23 smithi190 ceph-mon[134891]: pgmap v267: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 1.22k op/s 2023-10-06T22:23:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:23 smithi057 ceph-mon[98492]: pgmap v267: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 1.22k op/s 2023-10-06T22:23:23.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:23 smithi156 ceph-mon[134358]: pgmap v267: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 1.22k op/s 2023-10-06T22:23:25.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:25 smithi190 ceph-mon[134891]: pgmap v268: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 1.12k op/s 2023-10-06T22:23:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:25 smithi057 ceph-mon[98492]: pgmap v268: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 1.12k op/s 2023-10-06T22:23:25.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:25 smithi156 ceph-mon[134358]: pgmap v268: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 1.12k op/s 2023-10-06T22:23:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:27 smithi057 ceph-mon[98492]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T22:23:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:27 smithi057 ceph-mon[98492]: pgmap v270: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 1.16k op/s 2023-10-06T22:23:27.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:27 smithi156 ceph-mon[134358]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T22:23:27.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:27 smithi156 ceph-mon[134358]: pgmap v270: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 1.16k op/s 2023-10-06T22:23:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:27 smithi190 ceph-mon[134891]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T22:23:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:27 smithi190 ceph-mon[134891]: pgmap v270: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 1.16k op/s 2023-10-06T22:23:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.564+0000 7ff9971ff700 1 -- 172.21.15.57:0/500683578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901010d0 msgr2=0x7ff9901014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.564+0000 7ff9971ff700 1 --2- 172.21.15.57:0/500683578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901010d0 0x7ff9901014b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b0a0 tx=0x7ff98400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.565+0000 7ff9971ff700 1 -- 172.21.15.57:0/500683578 shutdown_connections 2023-10-06T22:23:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.565+0000 7ff9971ff700 1 --2- 172.21.15.57:0/500683578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9901061f0 0x7ff99010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.565+0000 7ff9971ff700 1 --2- 172.21.15.57:0/500683578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901010d0 0x7ff9901014b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.565+0000 7ff9971ff700 1 --2- 172.21.15.57:0/500683578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff990101a80 0x7ff990105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.565+0000 7ff9971ff700 1 -- 172.21.15.57:0/500683578 >> 172.21.15.57:0/500683578 conn(0x7ff9900fc930 msgr2=0x7ff9900fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.565+0000 7ff9971ff700 1 -- 172.21.15.57:0/500683578 shutdown_connections 2023-10-06T22:23:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.565+0000 7ff9971ff700 1 -- 172.21.15.57:0/500683578 wait complete. 2023-10-06T22:23:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 Processor -- start 2023-10-06T22:23:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 -- start start 2023-10-06T22:23:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9901010d0 0x7ff990197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901061f0 0x7ff9901976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99019bd10 0x7ff99019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019c790 con 0x7ff99019bd10 2023-10-06T22:23:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019c900 con 0x7ff9901010d0 2023-10-06T22:23:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff9971ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019cac0 con 0x7ff9901061f0 2023-10-06T22:23:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9901010d0 0x7ff990197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9901010d0 0x7ff990197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:54406/0 (socket says 172.21.15.57:54406) 2023-10-06T22:23:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff994f9b700 1 -- 172.21.15.57:0/3685938431 learned_addr learned my addr 172.21.15.57:0/3685938431 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.566+0000 7ff99579c700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99019bd10 0x7ff99019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff98ffff700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901061f0 0x7ff9901976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff994f9b700 1 -- 172.21.15.57:0/3685938431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901061f0 msgr2=0x7ff9901976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff994f9b700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901061f0 0x7ff9901976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff994f9b700 1 -- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99019bd10 msgr2=0x7ff99019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff994f9b700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99019bd10 0x7ff99019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff994f9b700 1 -- 172.21.15.57:0/3685938431 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984009d00 con 0x7ff9901010d0 2023-10-06T22:23:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff994f9b700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9901010d0 0x7ff990197160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b070 tx=0x7ff984007990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.567+0000 7ff98dffb700 1 -- 172.21.15.57:0/3685938431 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400e040 con 0x7ff9901010d0 2023-10-06T22:23:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.568+0000 7ff98dffb700 1 -- 172.21.15.57:0/3685938431 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98400bde0 con 0x7ff9901010d0 2023-10-06T22:23:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.568+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9901a3fc0 con 0x7ff9901010d0 2023-10-06T22:23:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.568+0000 7ff98dffb700 1 -- 172.21.15.57:0/3685938431 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98401dea0 con 0x7ff9901010d0 2023-10-06T22:23:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.568+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901a43e0 con 0x7ff9901010d0 2023-10-06T22:23:29.576 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:29 smithi190 ceph-mon[134891]: pgmap v271: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 433 op/s 2023-10-06T22:23:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.572+0000 7ff98dffb700 1 -- 172.21.15.57:0/3685938431 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff98401d480 con 0x7ff9901010d0 2023-10-06T22:23:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.573+0000 7ff9aac0fb80 1 Processor -- start 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.573+0000 7ff9aac0fb80 1 -- start start 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3d33d0 0x56385c3d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c331400 con 0x56385c3c7c00 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c322bc0 con 0x56385c3d33d0 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c3204e0 con 0x56385c3c8520 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3d33d0 0x56385c3d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3d33d0 0x56385c3d57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:54420/0 (socket says 172.21.15.57:54420) 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff99579c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff98ffff700 1 -- 172.21.15.57:0/961915383 learned_addr learned my addr 172.21.15.57:0/961915383 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff98dffb700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff9800a1670 0x7ff9800a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff98ffff700 1 -- 172.21.15.57:0/961915383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 msgr2=0x56385c3d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.574+0000 7ff98ffff700 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff98ffff700 1 -- 172.21.15.57:0/961915383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 msgr2=0x56385c3c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff98ffff700 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff98ffff700 1 -- 172.21.15.57:0/961915383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56385c146d80 con 0x56385c3d33d0 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff99579c700 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff99579c700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff9800a1670 0x7ff9800a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff98dffb700 1 -- 172.21.15.57:0/3685938431 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ff9840cdfa0 con 0x7ff9901010d0 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff98ffff700 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3d33d0 0x56385c3d57c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009cf0 tx=0x7ff97c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a67ee69910ba634 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff97affd700 1 -- 172.21.15.57:0/961915383 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c011d80 con 0x56385c3d33d0 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff97affd700 1 -- 172.21.15.57:0/961915383 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97c004510 con 0x56385c3d33d0 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.575+0000 7ff97affd700 1 -- 172.21.15.57:0/961915383 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c005280 con 0x56385c3d33d0 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/961915383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3d33d0 msgr2=0x56385c3d57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3d33d0 0x56385c3d57c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009cf0 tx=0x7ff97c011440 comp rx=0 tx=0).stop 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff99579c700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff9800a1670 0x7ff9800a3b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9880045e0 tx=0x7ff988014040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/961915383 shutdown_connections 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3d33d0 0x56385c3d57c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/961915383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/961915383 >> 172.21.15.57:0/961915383 conn(0x56385c33d200 msgr2=0x56385c3c4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/961915383 shutdown_connections 2023-10-06T22:23:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/961915383 wait complete. 2023-10-06T22:23:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.576+0000 7ff9aac0fb80 1 -- start start 2023-10-06T22:23:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 1 Processor -- start 2023-10-06T22:23:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c7c00 0x56385c3d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c31bdd0 con 0x56385c3d33d0 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c2b86b0 con 0x56385c3c8520 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c2b8350 con 0x56385c3c7c00 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff9aac0fb80 10 client.0 _ll_register_callbacks cb 0x56385c1157a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c7c00 0x56385c3d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c7c00 0x56385c3d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45560/0 (socket says 172.21.15.57:45560) 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff994f9b700 1 -- 172.21.15.57:0/2973970967 learned_addr learned my addr 172.21.15.57:0/2973970967 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.577+0000 7ff98ffff700 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff99579c700 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff994f9b700 1 -- 172.21.15.57:0/2973970967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 msgr2=0x56385c3d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff994f9b700 1 -- 172.21.15.57:0/2973970967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 msgr2=0x56385c3d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff994f9b700 1 -- 172.21.15.57:0/2973970967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97c009920 con 0x56385c3c7c00 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c7c00 0x56385c3d2df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9840cb710 tx=0x7ff984026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b22e376b2826c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff96ffff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff96ffff700 1 -- 172.21.15.57:0/2973970967 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98409ab10 con 0x56385c3c7c00 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56385c3bbd50 con 0x56385c3c7c00 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff96ffff700 1 -- 172.21.15.57:0/2973970967 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ff98409b6d0 con 0x56385c3c7c00 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff96ffff700 1 -- 172.21.15.57:0/2973970967 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff98409a930 con 0x56385c3c7c00 2023-10-06T22:23:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff9aac0fb80 10 client.26872 fetch_fsmap learned FSMap version 57 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.578+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x56385c12fc00 con 0x56385c3c7c00 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff96ffff700 1 -- 172.21.15.57:0/2973970967 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56385c3bbd50 con 0x56385c3c7c00 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff96ffff700 1 -- 172.21.15.57:0/2973970967 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 57) v1 ==== 7958+0+0 (secure 0 0 0) 0x7ff9840976e0 con 0x56385c3c7c00 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff9aac0fb80 10 client.26872 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff9aac0fb80 10 client.26872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff9aac0fb80 20 client.26872 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c3e2c60 0x56385c3e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff9aac0fb80 4 client.26872 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56385c314db0 con 0x56385c3e2c60 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.579+0000 7ff98ffff700 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c3e2c60 0x56385c3e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.580+0000 7ff98ffff700 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c3e2c60 0x56385c3e5080 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6b22d7fdcdf179 server_cookie=8540d60c324857b2 in_seq=0 out_seq=0 2023-10-06T22:23:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.580+0000 7ff96ffff700 10 client.26872 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff96ffff700 1 -- 172.21.15.57:0/2973970967 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56385c314db0 con 0x56385c3e2c60 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff96ffff700 10 client.26872 handle_command_reply: tid=0 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 client.26872 shutdown 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c7c00 msgr2=0x56385c3d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c7c00 0x56385c3d2df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9840cb710 tx=0x7ff984026000 comp rx=0 tx=0).stop 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 shutdown_connections 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c3e2c60 0x56385c3e5080 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c7c00 0x56385c3d2df0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff96ffff700 0 client.26872 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2973970967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 >> 172.21.15.57:0/2973970967 conn(0x56385c33d200 msgr2=0x56385c3c3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 shutdown_connections 2023-10-06T22:23:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2973970967 wait complete. 2023-10-06T22:23:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.581+0000 7ff9aac0fb80 20 client.26872 trim_cache size 0 max 16384 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.596+0000 7ff9aac0fb80 1 Processor -- start 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff9aac0fb80 1 -- start start 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3d33d0 0x56385c3cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c19c8c0 con 0x56385c3c7c00 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c3d1650 con 0x56385c3c8520 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c3da490 con 0x56385c3d33d0 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3d9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3d9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45774/0 (socket says 172.21.15.57:45774) 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff994f9b700 1 -- 172.21.15.57:0/2448636071 learned_addr learned my addr 172.21.15.57:0/2448636071 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff99579c700 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3d33d0 0x56385c3cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff98ffff700 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff994f9b700 1 -- 172.21.15.57:0/2448636071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3d33d0 msgr2=0x56385c3cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3d33d0 0x56385c3cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff994f9b700 1 -- 172.21.15.57:0/2448636071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 msgr2=0x56385c3d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.597+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.598+0000 7ff994f9b700 1 -- 172.21.15.57:0/2448636071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56385c16d1f0 con 0x56385c3c7c00 2023-10-06T22:23:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.598+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3d9f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff984097a40 tx=0x7ff98406e320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaf9b39db33468c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.598+0000 7ff97b7fe700 1 -- 172.21.15.57:0/2448636071 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98409ab10 con 0x56385c3c7c00 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.598+0000 7ff97b7fe700 1 -- 172.21.15.57:0/2448636071 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98409b6d0 con 0x56385c3c7c00 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.598+0000 7ff97b7fe700 1 -- 172.21.15.57:0/2448636071 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984097870 con 0x56385c3c7c00 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2448636071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 msgr2=0x56385c3d9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3d9f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff984097a40 tx=0x7ff98406e320 comp rx=0 tx=0).stop 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2448636071 shutdown_connections 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3d33d0 0x56385c3cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3c7c00 0x56385c3d9f50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2448636071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c8520 0x56385c3d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2448636071 >> 172.21.15.57:0/2448636071 conn(0x56385c33d200 msgr2=0x56385c3cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2448636071 shutdown_connections 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2448636071 wait complete. 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 -- start start 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.599+0000 7ff9aac0fb80 1 Processor -- start 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c7c00 0x56385c3d9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff9aac0fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c3d81c0 con 0x56385c3d33d0 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c3d8330 con 0x56385c3c7c00 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff9aac0fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385c3d84f0 con 0x56385c3c8520 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff9aac0fb80 10 client.0 _ll_register_callbacks cb 0x7ff9840074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c7c00 0x56385c3d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c7c00 0x56385c3d9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:54450/0 (socket says 172.21.15.57:54450) 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff98ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45576/0 (socket says 172.21.15.57:45576) 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 -- 172.21.15.57:0/2216751657 learned_addr learned my addr 172.21.15.57:0/2216751657 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff99579c700 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 -- 172.21.15.57:0/2216751657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 msgr2=0x56385c3d9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 -- 172.21.15.57:0/2216751657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 msgr2=0x56385c3d7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.600+0000 7ff994f9b700 1 -- 172.21.15.57:0/2216751657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56385c12fc00 con 0x56385c3c7c00 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff994f9b700 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c7c00 0x56385c3d9660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff984009fd0 tx=0x7ff98409ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0a23a3952b98d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff96ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:23:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff96ffff700 1 -- 172.21.15.57:0/2216751657 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984095de0 con 0x56385c3c7c00 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff96ffff700 1 -- 172.21.15.57:0/2216751657 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ff98406a070 con 0x56385c3c7c00 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff97c004510 con 0x56385c3c7c00 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff96ffff700 1 -- 172.21.15.57:0/2216751657 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff98409f120 con 0x56385c3c7c00 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff9aac0fb80 10 client.26867 fetch_fsmap learned FSMap version 57 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x56385c145250 con 0x56385c3c7c00 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.601+0000 7ff96ffff700 1 -- 172.21.15.57:0/2216751657 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98407d020 con 0x56385c3c7c00 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff96ffff700 1 -- 172.21.15.57:0/2216751657 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7958+0+0 (secure 0 0 0) 0x7ff9840aed20 con 0x56385c3c7c00 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff9aac0fb80 10 client.26867 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff9aac0fb80 10 client.26867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff9aac0fb80 20 client.26867 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c437650 0x56385c439a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff9aac0fb80 4 client.26867 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:23:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56385c314ad0 con 0x56385c437650 2023-10-06T22:23:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff98ffff700 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c437650 0x56385c439a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff98ffff700 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c437650 0x56385c439a40 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a31cca0ce5808a server_cookie=bd8c344d894d9685 in_seq=0 out_seq=0 2023-10-06T22:23:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.602+0000 7ff96ffff700 10 client.26867 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff96ffff700 1 -- 172.21.15.57:0/2216751657 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56385c314ad0 con 0x56385c437650 2023-10-06T22:23:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff96ffff700 10 client.26867 handle_command_reply: tid=0 2023-10-06T22:23:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 client.26867 shutdown 2023-10-06T22:23:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c7c00 msgr2=0x56385c3d9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c7c00 0x56385c3d9660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff984009fd0 tx=0x7ff98409ada0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 shutdown_connections 2023-10-06T22:23:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56385c437650 0x56385c439a40 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56385c3d33d0 0x56385c3d7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56385c3c7c00 0x56385c3d9660 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 --2- 172.21.15.57:0/2216751657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56385c3c8520 0x56385c3d9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff96ffff700 0 client.26867 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 >> 172.21.15.57:0/2216751657 conn(0x56385c33d200 msgr2=0x56385c3c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.633+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 shutdown_connections 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9aac0fb80 1 -- 172.21.15.57:0/2216751657 wait complete. 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9aac0fb80 20 client.26867 trim_cache size 0 max 16384 2023-10-06T22:23:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:23:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:23:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:23:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:23:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff9800a1670 msgr2=0x7ff9800a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff9800a1670 0x7ff9800a3b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9880045e0 tx=0x7ff988014040 comp rx=0 tx=0).stop 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9901010d0 msgr2=0x7ff990197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9901010d0 0x7ff990197160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b070 tx=0x7ff984007990 comp rx=0 tx=0).stop 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 shutdown_connections 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff9800a1670 0x7ff9800a3b30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff99019bd10 0x7ff99019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9901010d0 0x7ff990197160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 --2- 172.21.15.57:0/3685938431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9901061f0 0x7ff9901976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.634+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 >> 172.21.15.57:0/3685938431 conn(0x7ff9900fc930 msgr2=0x7ff9900fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.635+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 shutdown_connections 2023-10-06T22:23:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:29.635+0000 7ff9971ff700 1 -- 172.21.15.57:0/3685938431 wait complete. 2023-10-06T22:23:29.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:29 smithi057 ceph-mon[98492]: pgmap v271: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 433 op/s 2023-10-06T22:23:29.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:29 smithi156 ceph-mon[134358]: pgmap v271: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 433 op/s 2023-10-06T22:23:31.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:31 smithi190 ceph-mon[134891]: pgmap v272: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 243 op/s 2023-10-06T22:23:31.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:31 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:23:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:31 smithi057 ceph-mon[98492]: pgmap v272: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 243 op/s 2023-10-06T22:23:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:31 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:23:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:31 smithi156 ceph-mon[134358]: pgmap v272: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 243 op/s 2023-10-06T22:23:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:31 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:23:33.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:33 smithi190 ceph-mon[134891]: pgmap v273: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2023-10-06T22:23:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:33 smithi057 ceph-mon[98492]: pgmap v273: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2023-10-06T22:23:33.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:33 smithi156 ceph-mon[134358]: pgmap v273: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2023-10-06T22:23:33.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:23:34.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.103+0000 7f0bda5ff700 1 -- 172.21.15.57:0/1379631340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 msgr2=0x7f0bd410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.103+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/1379631340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 0x7f0bd410ff70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009f90 tx=0x7f0bcc00b330 comp rx=0 tx=0).stop 2023-10-06T22:23:34.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.104+0000 7f0bda5ff700 1 -- 172.21.15.57:0/1379631340 shutdown_connections 2023-10-06T22:23:34.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.104+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/1379631340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 0x7f0bd410ff70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.104+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/1379631340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 0x7f0bd41036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.104+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/1379631340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd4103c80 0x7f0bd4107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.104+0000 7f0bda5ff700 1 -- 172.21.15.57:0/1379631340 >> 172.21.15.57:0/1379631340 conn(0x7f0bd40febb0 msgr2=0x7f0bd4100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:34.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.104+0000 7f0bda5ff700 1 -- 172.21.15.57:0/1379631340 shutdown_connections 2023-10-06T22:23:34.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.104+0000 7f0bda5ff700 1 -- 172.21.15.57:0/1379631340 wait complete. 2023-10-06T22:23:34.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 Processor -- start 2023-10-06T22:23:34.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 -- start start 2023-10-06T22:23:34.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 0x7f0bd41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 0x7f0bd4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd419dc90 0x7f0bd419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd419e710 con 0x7f0bd41083f0 2023-10-06T22:23:34.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd419e880 con 0x7f0bd41032d0 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bda5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd419ea40 con 0x7f0bd419dc90 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 0x7f0bd41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 0x7f0bd41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41296/0 (socket says 172.21.15.57:41296) 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bd3fff700 1 -- 172.21.15.57:0/4089344600 learned_addr learned my addr 172.21.15.57:0/4089344600 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.105+0000 7f0bd8b9c700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd419dc90 0x7f0bd419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 0x7f0bd4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd3fff700 1 -- 172.21.15.57:0/4089344600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd419dc90 msgr2=0x7f0bd419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd419dc90 0x7f0bd419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd3fff700 1 -- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 msgr2=0x7f0bd4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 0x7f0bd4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd3fff700 1 -- 172.21.15.57:0/4089344600 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc8009520 con 0x7f0bd41032d0 2023-10-06T22:23:34.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 0x7f0bd41992f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9c0 tx=0x7f0bc400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:34.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd17fa700 1 -- 172.21.15.57:0/4089344600 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4004a30 con 0x7f0bd41032d0 2023-10-06T22:23:34.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd17fa700 1 -- 172.21.15.57:0/4089344600 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc400cc80 con 0x7f0bd41032d0 2023-10-06T22:23:34.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc009c40 con 0x7f0bd41032d0 2023-10-06T22:23:34.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.106+0000 7f0bd17fa700 1 -- 172.21.15.57:0/4089344600 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40054e0 con 0x7f0bd41032d0 2023-10-06T22:23:34.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.107+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd41a6290 con 0x7f0bd41032d0 2023-10-06T22:23:34.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.110+0000 7f0bd17fa700 1 -- 172.21.15.57:0/4089344600 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0bc4004b90 con 0x7f0bd41032d0 2023-10-06T22:23:34.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 Processor -- start 2023-10-06T22:23:34.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 -- start start 2023-10-06T22:23:34.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da60c0 0x558044da64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da69e0 0x558044db1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044db3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044d0f460 con 0x558044da69e0 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044d00c20 con 0x558044da60c0 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bede85b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044cfe540 con 0x558044db1890 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.111+0000 7f0bd17fa700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0bbc0a15f0 0x7f0bbc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd8b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da60c0 0x558044da64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd17fa700 1 -- 172.21.15.57:0/4089344600 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f0bc4014070 con 0x7f0bd41032d0 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da69e0 0x558044db1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044db3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd8b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da60c0 0x558044da64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41302/0 (socket says 172.21.15.57:41302) 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd8b9c700 1 -- 172.21.15.57:0/201277057 learned_addr learned my addr 172.21.15.57:0/201277057 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:34.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd8b9c700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0bbc0a15f0 0x7f0bbc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd37fe700 1 -- 172.21.15.57:0/201277057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da60c0 msgr2=0x558044da64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/201277057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da60c0 0x558044da64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd37fe700 1 -- 172.21.15.57:0/201277057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da69e0 msgr2=0x558044db1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/201277057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da69e0 0x558044db1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.112+0000 7f0bd37fe700 1 -- 172.21.15.57:0/201277057 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558044b24de0 con 0x558044db1890 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.113+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/201277057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044db3c80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4067c10 tx=0x7f0bc8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5279233cd40dc1d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.113+0000 7f0bd8b9c700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0bbc0a15f0 0x7f0bbc0a3ab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc007640 tx=0x7f0bcc0096c0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:34.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.113+0000 7f0bba7fc700 1 -- 172.21.15.57:0/201277057 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8019630 con 0x558044db1890 2023-10-06T22:23:34.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.113+0000 7f0bba7fc700 1 -- 172.21.15.57:0/201277057 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc8019c50 con 0x558044db1890 2023-10-06T22:23:34.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.113+0000 7f0bba7fc700 1 -- 172.21.15.57:0/201277057 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800ebf0 con 0x558044db1890 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.113+0000 7f0bede85b80 1 -- 172.21.15.57:0/201277057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 msgr2=0x558044db3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.113+0000 7f0bede85b80 1 --2- 172.21.15.57:0/201277057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044db3c80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4067c10 tx=0x7f0bc8011720 comp rx=0 tx=0).stop 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 -- 172.21.15.57:0/201277057 shutdown_connections 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 --2- 172.21.15.57:0/201277057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044db3c80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 --2- 172.21.15.57:0/201277057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da60c0 0x558044da64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 --2- 172.21.15.57:0/201277057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da69e0 0x558044db1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 -- 172.21.15.57:0/201277057 >> 172.21.15.57:0/201277057 conn(0x558044d1b260 msgr2=0x558044da2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 -- 172.21.15.57:0/201277057 shutdown_connections 2023-10-06T22:23:34.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 -- 172.21.15.57:0/201277057 wait complete. 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 -- start start 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.114+0000 7f0bede85b80 1 Processor -- start 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044dae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044daed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bede85b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044cf9e30 con 0x558044da60c0 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bede85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044c96710 con 0x558044da69e0 2023-10-06T22:23:34.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bede85b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044c963b0 con 0x558044db1890 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bede85b80 10 client.0 _ll_register_callbacks cb 0x558044af3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044dae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bd8b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044daed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044dae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41316/0 (socket says 172.21.15.57:41316) 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1989597553 learned_addr learned my addr 172.21.15.57:0/1989597553 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.115+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1989597553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 msgr2=0x558044daed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044daed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1989597553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 msgr2=0x558044db1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1989597553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc8009a70 con 0x558044da69e0 2023-10-06T22:23:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044dae5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0bc801c040 tx=0x7f0bc8005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a030198deafd7ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bb37fe700 1 -- 172.21.15.57:0/1989597553 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8019840 con 0x558044da69e0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558044d99f10 con 0x558044da69e0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.116+0000 7f0bb37fe700 1 -- 172.21.15.57:0/1989597553 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f0bc80199a0 con 0x558044da69e0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bb37fe700 1 -- 172.21.15.57:0/1989597553 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc800e430 con 0x558044da69e0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 10 client.26879 fetch_fsmap learned FSMap version 58 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x558044b0dc60 con 0x558044da69e0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bb37fe700 1 -- 172.21.15.57:0/1989597553 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558044d99f10 con 0x558044da69e0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bb37fe700 1 -- 172.21.15.57:0/1989597553 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f0bc8028020 con 0x558044da69e0 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 10 client.26879 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 10 client.26879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 20 client.26879 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044dc1220 0x558044dc3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 4 client.26879 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558044cf2e10 con 0x558044dc1220 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.117+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044dc1220 0x558044dc3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.118+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044dc1220 0x558044dc3640 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d5902f6684b19f server_cookie=a47ac0b75c1c0fcd in_seq=0 out_seq=0 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.118+0000 7f0bb37fe700 10 client.26879 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bb37fe700 1 -- 172.21.15.57:0/1989597553 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558044cf2e10 con 0x558044dc1220 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bb37fe700 10 client.26879 handle_command_reply: tid=0 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bede85b80 1 client.26879 shutdown 2023-10-06T22:23:34.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 msgr2=0x558044dae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044dae5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0bc801c040 tx=0x7f0bc8005e20 comp rx=0 tx=0).stop 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 shutdown_connections 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044dc1220 0x558044dc3640 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044daed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.119+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.120+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1989597553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044dae5d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.120+0000 7f0bb37fe700 0 client.26879 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.120+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 >> 172.21.15.57:0/1989597553 conn(0x558044d1b260 msgr2=0x558044da1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.120+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 shutdown_connections 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.120+0000 7f0bede85b80 1 -- 172.21.15.57:0/1989597553 wait complete. 2023-10-06T22:23:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.120+0000 7f0bede85b80 20 client.26879 trim_cache size 0 max 16384 2023-10-06T22:23:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.137+0000 7f0bede85b80 1 Processor -- start 2023-10-06T22:23:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.137+0000 7f0bede85b80 1 -- start start 2023-10-06T22:23:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044da9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044da69e0 0x558044da98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044db1890 0x558044da9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bede85b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044b7a920 con 0x558044da60c0 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bede85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044da78e0 con 0x558044db1890 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bede85b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044da3280 con 0x558044da69e0 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044da69e0 0x558044da98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bd8b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044db1890 0x558044da9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044da69e0 0x558044da98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46042/0 (socket says 172.21.15.57:46042) 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1206465019 learned_addr learned my addr 172.21.15.57:0/1206465019 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044da9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1206465019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044db1890 msgr2=0x558044da9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.138+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044db1890 0x558044da9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1206465019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 msgr2=0x558044da9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044da9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bd37fe700 1 -- 172.21.15.57:0/1206465019 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558044d9e140 con 0x558044da69e0 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044da9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bd8b9c700 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044db1890 0x558044da9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044da69e0 0x558044da98f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8001eb0 tx=0x7f0bc8005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85ef99f253600a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bbaffd700 1 -- 172.21.15.57:0/1206465019 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800e5b0 con 0x558044da69e0 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bbaffd700 1 -- 172.21.15.57:0/1206465019 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc800e710 con 0x558044da69e0 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.139+0000 7f0bbaffd700 1 -- 172.21.15.57:0/1206465019 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8003680 con 0x558044da69e0 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 -- 172.21.15.57:0/1206465019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044da69e0 msgr2=0x558044da98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044da69e0 0x558044da98f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8001eb0 tx=0x7f0bc8005a40 comp rx=0 tx=0).stop 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 -- 172.21.15.57:0/1206465019 shutdown_connections 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044db1890 0x558044da9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044da9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 --2- 172.21.15.57:0/1206465019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044da69e0 0x558044da98f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 -- 172.21.15.57:0/1206465019 >> 172.21.15.57:0/1206465019 conn(0x558044d1b260 msgr2=0x558044db0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 -- 172.21.15.57:0/1206465019 shutdown_connections 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 -- 172.21.15.57:0/1206465019 wait complete. 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 -- start start 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.140+0000 7f0bede85b80 1 Processor -- start 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044db6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bede85b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044e14050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bede85b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044e14590 con 0x558044da60c0 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bede85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044e146d0 con 0x558044da69e0 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bede85b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044e14840 con 0x558044db1890 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bede85b80 10 client.0 _ll_register_callbacks cb 0x7f0bc4004f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044db6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bd8b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044e14050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33710/0 (socket says 172.21.15.57:33710) 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bd8b9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044e14050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:46048/0 (socket says 172.21.15.57:46048) 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bd3fff700 1 -- 172.21.15.57:0/303902826 learned_addr learned my addr 172.21.15.57:0/303902826 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:34.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.141+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044db6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:41342/0 (socket says 172.21.15.57:41342) 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bd3fff700 1 -- 172.21.15.57:0/303902826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 msgr2=0x558044e14050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044e14050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bd3fff700 1 -- 172.21.15.57:0/303902826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 msgr2=0x558044db6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044db6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bd3fff700 1 -- 172.21.15.57:0/303902826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd41a6290 con 0x558044da60c0 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044db6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bd3fff700 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db65b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4096c00 tx=0x7f0bc409a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879368859f1addb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bb37fe700 1 -- 172.21.15.57:0/303902826 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc409a190 con 0x558044da60c0 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bb37fe700 1 -- 172.21.15.57:0/303902826 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f0bc409e2e0 con 0x558044da60c0 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bc8019c50 con 0x558044da60c0 2023-10-06T22:23:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bb37fe700 1 -- 172.21.15.57:0/303902826 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc40a2520 con 0x558044da60c0 2023-10-06T22:23:34.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bede85b80 10 client.16854 fetch_fsmap learned FSMap version 58 2023-10-06T22:23:34.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.142+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7f0bcc029040 con 0x558044da60c0 2023-10-06T22:23:34.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bb37fe700 1 -- 172.21.15.57:0/303902826 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc8019c50 con 0x558044da60c0 2023-10-06T22:23:34.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bb37fe700 1 -- 172.21.15.57:0/303902826 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f0bc409a3f0 con 0x558044da60c0 2023-10-06T22:23:34.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bede85b80 10 client.16854 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bede85b80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bede85b80 20 client.16854 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bede85b80 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044e16020 0x558044e18440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bede85b80 4 client.16854 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558044cf2b30 con 0x558044e16020 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.143+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044e16020 0x558044e18440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bd37fe700 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044e16020 0x558044e18440 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2541c780ee1b57ce server_cookie=78018ab37a6ebc6 in_seq=0 out_seq=0 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bb37fe700 10 client.16854 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bb37fe700 1 -- 172.21.15.57:0/303902826 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558044cf2b30 con 0x558044e16020 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bb37fe700 10 client.16854 handle_command_reply: tid=0 2023-10-06T22:23:34.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 client.16854 shutdown 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 msgr2=0x558044db65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db65b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4096c00 tx=0x7f0bc409a080 comp rx=0 tx=0).stop 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 shutdown_connections 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x558044e16020 0x558044e18440 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558044db1890 0x558044e14050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558044da60c0 0x558044db65b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 --2- 172.21.15.57:0/303902826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558044da69e0 0x558044db6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 >> 172.21.15.57:0/303902826 conn(0x558044d1b260 msgr2=0x558044da1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bb37fe700 0 client.16854 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 shutdown_connections 2023-10-06T22:23:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.144+0000 7f0bede85b80 1 -- 172.21.15.57:0/303902826 wait complete. 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bede85b80 20 client.16854 trim_cache size 0 max 16384 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:23:34.153 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:23:34.154 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:23:34.154 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:23:34.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0bbc0a15f0 msgr2=0x7f0bbc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0bbc0a15f0 0x7f0bbc0a3ab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc007640 tx=0x7f0bcc0096c0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 msgr2=0x7f0bd41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 0x7f0bd41992f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9c0 tx=0x7f0bc400ecd0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 shutdown_connections 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0bbc0a15f0 0x7f0bbc0a3ab0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd419dc90 0x7f0bd419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bd41032d0 0x7f0bd41992f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 --2- 172.21.15.57:0/4089344600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd41083f0 0x7f0bd4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 >> 172.21.15.57:0/4089344600 conn(0x7f0bd40febb0 msgr2=0x7f0bd40ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 shutdown_connections 2023-10-06T22:23:34.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:23:34.145+0000 7f0bda5ff700 1 -- 172.21.15.57:0/4089344600 wait complete. 2023-10-06T22:23:34.165 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:23:35.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:35 smithi190 ceph-mon[134891]: pgmap v274: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2023-10-06T22:23:35.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:35 smithi057 ceph-mon[98492]: pgmap v274: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2023-10-06T22:23:35.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:35 smithi156 ceph-mon[134358]: pgmap v274: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2023-10-06T22:23:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:37 smithi057 ceph-mon[98492]: pgmap v275: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-10-06T22:23:37.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:37 smithi156 ceph-mon[134358]: pgmap v275: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-10-06T22:23:37.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:37 smithi190 ceph-mon[134891]: pgmap v275: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-10-06T22:23:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:39 smithi190 ceph-mon[134891]: pgmap v276: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2023-10-06T22:23:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:39 smithi057 ceph-mon[98492]: pgmap v276: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2023-10-06T22:23:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:39 smithi156 ceph-mon[134358]: pgmap v276: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2023-10-06T22:23:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:41 smithi190 ceph-mon[134891]: pgmap v277: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 249 op/s 2023-10-06T22:23:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:41 smithi057 ceph-mon[98492]: pgmap v277: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 249 op/s 2023-10-06T22:23:41.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:41 smithi156 ceph-mon[134358]: pgmap v277: 97 pgs: 97 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 249 op/s 2023-10-06T22:23:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.787+0000 7fd8263ff700 1 -- 172.21.15.57:0/2217646399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd820101a80 msgr2=0x7fd820105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.787+0000 7fd8263ff700 1 --2- 172.21.15.57:0/2217646399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd820101a80 0x7fd820105ac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8140098f0 tx=0x7fd814009c00 comp rx=0 tx=0).stop 2023-10-06T22:23:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.787+0000 7fd8263ff700 1 -- 172.21.15.57:0/2217646399 shutdown_connections 2023-10-06T22:23:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.787+0000 7fd8263ff700 1 --2- 172.21.15.57:0/2217646399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8201061f0 0x7fd82010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.787+0000 7fd8263ff700 1 --2- 172.21.15.57:0/2217646399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8201010d0 0x7fd8201014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.787+0000 7fd8263ff700 1 --2- 172.21.15.57:0/2217646399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd820101a80 0x7fd820105ac0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.787+0000 7fd8263ff700 1 -- 172.21.15.57:0/2217646399 >> 172.21.15.57:0/2217646399 conn(0x7fd8200fc950 msgr2=0x7fd8200fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.788+0000 7fd8263ff700 1 -- 172.21.15.57:0/2217646399 shutdown_connections 2023-10-06T22:23:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.788+0000 7fd8263ff700 1 -- 172.21.15.57:0/2217646399 wait complete. 2023-10-06T22:23:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 Processor -- start 2023-10-06T22:23:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 -- start start 2023-10-06T22:23:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8201010d0 0x7fd820199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8201061f0 0x7fd820199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd82019ddc0 0x7fd82019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019e840 con 0x7fd8201010d0 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019e9b0 con 0x7fd82019ddc0 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd8263ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019eb70 con 0x7fd8201061f0 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8201061f0 0x7fd820199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd82499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd82019ddc0 0x7fd82019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd82499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd82019ddc0 0x7fd82019e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:50604/0 (socket says 172.21.15.57:50604) 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.789+0000 7fd82499c700 1 -- 172.21.15.57:0/590551549 learned_addr learned my addr 172.21.15.57:0/590551549 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd81ffff700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8201010d0 0x7fd820199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd82499c700 1 -- 172.21.15.57:0/590551549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8201061f0 msgr2=0x7fd820199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd82499c700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8201061f0 0x7fd820199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd82499c700 1 -- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8201010d0 msgr2=0x7fd820199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd82499c700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8201010d0 0x7fd820199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd82499c700 1 -- 172.21.15.57:0/590551549 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009520 con 0x7fd82019ddc0 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8201061f0 0x7fd820199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd82499c700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd82019ddc0 0x7fd82019e240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd81800ef30 tx=0x7fd81800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd81d7fa700 1 -- 172.21.15.57:0/590551549 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818018070 con 0x7fd82019ddc0 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.790+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8201a6070 con 0x7fd82019ddc0 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.791+0000 7fd81d7fa700 1 -- 172.21.15.57:0/590551549 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81800f040 con 0x7fd82019ddc0 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.791+0000 7fd81d7fa700 1 -- 172.21.15.57:0/590551549 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81800cca0 con 0x7fd82019ddc0 2023-10-06T22:23:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.791+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8201a6550 con 0x7fd82019ddc0 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.794+0000 7fd81d7fa700 1 -- 172.21.15.57:0/590551549 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd818003a00 con 0x7fd82019ddc0 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.795+0000 7fd839d5ab80 1 Processor -- start 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.795+0000 7fd839d5ab80 1 -- start start 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81d7fa700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd8080a15f0 0x7fd8080a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c3afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c45e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81d7fa700 1 -- 172.21.15.57:0/590551549 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fd8180ccb70 con 0x7fd82019ddc0 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ba4400 con 0x563257c463d0 2023-10-06T22:23:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257b95bc0 con 0x563257c3ac00 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257b934e0 con 0x563257c3b520 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c3afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c3afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:50618/0 (socket says 172.21.15.57:50618) 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81ffff700 1 -- 172.21.15.57:0/925183443 learned_addr learned my addr 172.21.15.57:0/925183443 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c45e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd82499c700 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd8080a15f0 0x7fd8080a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81ffff700 1 -- 172.21.15.57:0/925183443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 msgr2=0x563257c45e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81ffff700 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c45e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.796+0000 7fd81ffff700 1 -- 172.21.15.57:0/925183443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 msgr2=0x563257c487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.797+0000 7fd81ffff700 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.797+0000 7fd81ffff700 1 -- 172.21.15.57:0/925183443 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632579b9d80 con 0x563257c3ac00 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.797+0000 7fd81ffff700 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c3afe0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd810009d60 tx=0x7fd810011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3642b97a543407de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.797+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd8080a15f0 0x7fd8080a3ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd814009fd0 tx=0x7fd814005dc0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.797+0000 7fd8067fc700 1 -- 172.21.15.57:0/925183443 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810019630 con 0x563257c3ac00 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.797+0000 7fd8067fc700 1 -- 172.21.15.57:0/925183443 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd810019c50 con 0x563257c3ac00 2023-10-06T22:23:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.797+0000 7fd8067fc700 1 -- 172.21.15.57:0/925183443 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000ebf0 con 0x563257c3ac00 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 -- 172.21.15.57:0/925183443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 msgr2=0x563257c3afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c3afe0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd810009d60 tx=0x7fd810011720 comp rx=0 tx=0).stop 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 -- 172.21.15.57:0/925183443 shutdown_connections 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c3afe0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/925183443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c45e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 -- 172.21.15.57:0/925183443 >> 172.21.15.57:0/925183443 conn(0x563257bb0200 msgr2=0x563257c37080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 -- 172.21.15.57:0/925183443 shutdown_connections 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.798+0000 7fd839d5ab80 1 -- 172.21.15.57:0/925183443 wait complete. 2023-10-06T22:23:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.799+0000 7fd839d5ab80 1 -- start start 2023-10-06T22:23:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.799+0000 7fd839d5ab80 1 Processor -- start 2023-10-06T22:23:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c45db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c43030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c43690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257b8edd0 con 0x563257c3b520 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257b2b6b0 con 0x563257c3ac00 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257b2b350 con 0x563257c463d0 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd839d5ab80 10 client.0 _ll_register_callbacks cb 0x5632579887a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c45db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c45db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:50620/0 (socket says 172.21.15.57:50620) 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81ffff700 1 -- 172.21.15.57:0/56509998 learned_addr learned my addr 172.21.15.57:0/56509998 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c43030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd82499c700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c43690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81ffff700 1 -- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 msgr2=0x563257c43690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81ffff700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c43690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81ffff700 1 -- 172.21.15.57:0/56509998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 msgr2=0x563257c43030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.800+0000 7fd81ffff700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c43030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd81ffff700 1 -- 172.21.15.57:0/56509998 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8100099c0 con 0x563257c3ac00 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd81ffff700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c45db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd81000a010 tx=0x7fd810009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4d47e2d0220967 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd7ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/56509998 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810004220 con 0x563257c3ac00 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563257c2ed50 con 0x563257c3ac00 2023-10-06T22:23:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/56509998 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fd810019dc0 con 0x563257c3ac00 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/56509998 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8100056f0 con 0x563257c3ac00 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd839d5ab80 10 client.26897 fetch_fsmap learned FSMap version 58 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.801+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5632579a2c00 con 0x563257c3ac00 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/56509998 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563257c2ed50 con 0x563257c3ac00 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/56509998 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fd810029030 con 0x563257c3ac00 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd839d5ab80 10 client.26897 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd839d5ab80 10 client.26897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd839d5ab80 20 client.26897 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257c55de0 0x563257c58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd839d5ab80 4 client.26897 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563257b87db0 con 0x563257c55de0 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.802+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257c55de0 0x563257c58200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.803+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257c55de0 0x563257c58200 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9297cf6d0f091e6b server_cookie=e46b791d060e95d2 in_seq=0 out_seq=0 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.803+0000 7fd7ff7fe700 10 client.26897 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/56509998 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563257b87db0 con 0x563257c55de0 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd7ff7fe700 10 client.26897 handle_command_reply: tid=0 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 client.26897 shutdown 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 msgr2=0x563257c45db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c45db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd81000a010 tx=0x7fd810009580 comp rx=0 tx=0).stop 2023-10-06T22:23:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 shutdown_connections 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257c55de0 0x563257c58200 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c43690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c45db0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/56509998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 >> 172.21.15.57:0/56509998 conn(0x563257bb0200 msgr2=0x563257c36c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd7ff7fe700 0 client.26897 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 shutdown_connections 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 1 -- 172.21.15.57:0/56509998 wait complete. 2023-10-06T22:23:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.804+0000 7fd839d5ab80 20 client.26897 trim_cache size 0 max 16384 2023-10-06T22:23:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 Processor -- start 2023-10-06T22:23:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 -- start start 2023-10-06T22:23:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c45950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c3e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257a0f8c0 con 0x563257c3b520 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257c41590 con 0x563257c3ac00 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.820+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257c3eea0 con 0x563257c463d0 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:50630/0 (socket says 172.21.15.57:50630) 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 -- 172.21.15.57:0/3705959929 learned_addr learned my addr 172.21.15.57:0/3705959929 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c45950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd82499c700 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c3e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 -- 172.21.15.57:0/3705959929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 msgr2=0x563257c3e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c3e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 -- 172.21.15.57:0/3705959929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 msgr2=0x563257c45950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c45950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 -- 172.21.15.57:0/3705959929 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632579e01f0 con 0x563257c3ac00 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd81ffff700 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c487c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd810000c00 tx=0x7fd810011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997e651e8c4a3f4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd806ffd700 1 -- 172.21.15.57:0/3705959929 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000e8c0 con 0x563257c3ac00 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd806ffd700 1 -- 172.21.15.57:0/3705959929 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8100056f0 con 0x563257c3ac00 2023-10-06T22:23:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.821+0000 7fd806ffd700 1 -- 172.21.15.57:0/3705959929 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810004b20 con 0x563257c3ac00 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 -- 172.21.15.57:0/3705959929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 msgr2=0x563257c487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c487c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd810000c00 tx=0x7fd810011720 comp rx=0 tx=0).stop 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 -- 172.21.15.57:0/3705959929 shutdown_connections 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c463d0 0x563257c3e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c487c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/3705959929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c3b520 0x563257c45950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 -- 172.21.15.57:0/3705959929 >> 172.21.15.57:0/3705959929 conn(0x563257bb0200 msgr2=0x563257c5cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 -- 172.21.15.57:0/3705959929 shutdown_connections 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.822+0000 7fd839d5ab80 1 -- 172.21.15.57:0/3705959929 wait complete. 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 -- start start 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 Processor -- start 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c49460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ca8be0 con 0x563257c463d0 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ca8d20 con 0x563257c3ac00 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ca8e60 con 0x563257c3b520 2023-10-06T22:23:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.823+0000 7fd839d5ab80 10 client.0 _ll_register_callbacks cb 0x7fd818095ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c49460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c49460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:48422/0 (socket says 172.21.15.57:48422) 2023-10-06T22:23:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c48f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:50642/0 (socket says 172.21.15.57:50642) 2023-10-06T22:23:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 -- 172.21.15.57:0/2722997056 learned_addr learned my addr 172.21.15.57:0/2722997056 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd82499c700 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 -- 172.21.15.57:0/2722997056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 msgr2=0x563257c48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 -- 172.21.15.57:0/2722997056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 msgr2=0x563257c499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 -- 172.21.15.57:0/2722997056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632579a2c00 con 0x563257c3b520 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81ffff700 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd81f7fe700 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c49460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd8140098c0 tx=0x7fd814009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132ef3a5924073ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.824+0000 7fd7ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/2722997056 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400b700 con 0x563257c3b520 2023-10-06T22:23:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/2722997056 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fd81400b860 con 0x563257c3b520 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd810019c50 con 0x563257c3b520 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/2722997056 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd81400ba20 con 0x563257c3b520 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd839d5ab80 10 client.26914 fetch_fsmap learned FSMap version 58 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5632579b8250 con 0x563257c3b520 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/2722997056 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd810019c50 con 0x563257c3b520 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/2722997056 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fd814005340 con 0x563257c3b520 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd839d5ab80 10 client.26914 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.825+0000 7fd839d5ab80 10 client.26914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.826+0000 7fd839d5ab80 20 client.26914 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.826+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257caa730 0x563257cacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.826+0000 7fd839d5ab80 4 client.26914 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:23:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.826+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563257b87ad0 con 0x563257caa730 2023-10-06T22:23:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.826+0000 7fd81ffff700 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257caa730 0x563257cacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.826+0000 7fd81ffff700 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257caa730 0x563257cacb50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=545d4f7422a3b75 server_cookie=c8bebd2e21f497b3 in_seq=0 out_seq=0 2023-10-06T22:23:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.826+0000 7fd7ff7fe700 10 client.26914 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd7ff7fe700 1 -- 172.21.15.57:0/2722997056 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563257b87ad0 con 0x563257caa730 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd7ff7fe700 10 client.26914 handle_command_reply: tid=0 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 client.26914 shutdown 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 msgr2=0x563257c49460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c49460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd8140098c0 tx=0x7fd814009c00 comp rx=0 tx=0).stop 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 shutdown_connections 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x563257caa730 0x563257cacb50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563257c463d0 0x563257c499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563257c3ac00 0x563257c48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd7ff7fe700 0 client.26914 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 --2- 172.21.15.57:0/2722997056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c3b520 0x563257c49460 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 >> 172.21.15.57:0/2722997056 conn(0x563257bb0200 msgr2=0x563257c3e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 shutdown_connections 2023-10-06T22:23:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 1 -- 172.21.15.57:0/2722997056 wait complete. 2023-10-06T22:23:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.840+0000 7fd839d5ab80 20 client.26914 trim_cache size 0 max 16384 2023-10-06T22:23:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:23:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:23:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:23:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd8080a15f0 msgr2=0x7fd8080a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd8080a15f0 0x7fd8080a3ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd814009fd0 tx=0x7fd814005dc0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd82019ddc0 msgr2=0x7fd82019e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd82019ddc0 0x7fd82019e240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd81800ef30 tx=0x7fd81800c3a0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 shutdown_connections 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd8080a15f0 0x7fd8080a3ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd82019ddc0 0x7fd82019e240 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8201010d0 0x7fd820199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 --2- 172.21.15.57:0/590551549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8201061f0 0x7fd820199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 >> 172.21.15.57:0/590551549 conn(0x7fd8200fc950 msgr2=0x7fd8200fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 shutdown_connections 2023-10-06T22:23:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:42.841+0000 7fd8263ff700 1 -- 172.21.15.57:0/590551549 wait complete. 2023-10-06T22:23:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:43 smithi190 ceph-mon[134891]: pgmap v278: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2023-10-06T22:23:43.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:43 smithi057 ceph-mon[98492]: pgmap v278: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2023-10-06T22:23:43.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:43 smithi156 ceph-mon[134358]: pgmap v278: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2023-10-06T22:23:45.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:45 smithi190 ceph-mon[134891]: pgmap v279: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2023-10-06T22:23:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:45 smithi057 ceph-mon[98492]: pgmap v279: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2023-10-06T22:23:45.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:45 smithi156 ceph-mon[134358]: pgmap v279: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2023-10-06T22:23:47.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:47 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:23:47.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:47 smithi057 ceph-mon[98492]: pgmap v280: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-10-06T22:23:47.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:47 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:23:47.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:47 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:23:47.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:47 smithi156 ceph-mon[134358]: pgmap v280: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-10-06T22:23:47.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:47 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:23:47.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:47 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:23:47.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:47 smithi190 ceph-mon[134891]: pgmap v280: 97 pgs: 97 active+clean; 1.7 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-10-06T22:23:47.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:47 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:23:49.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:49 smithi190 ceph-mon[134891]: pgmap v281: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 330 op/s 2023-10-06T22:23:49.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:23:49.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:23:49.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:49 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:23:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:49 smithi057 ceph-mon[98492]: pgmap v281: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 330 op/s 2023-10-06T22:23:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:23:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:23:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:49 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:23:49.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:49 smithi156 ceph-mon[134358]: pgmap v281: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 330 op/s 2023-10-06T22:23:49.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:23:49.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:23:49.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:49 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:23:51.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:51 smithi190 ceph-mon[134891]: pgmap v282: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-10-06T22:23:51.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:51 smithi057 ceph-mon[98492]: pgmap v282: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-10-06T22:23:51.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:51 smithi156 ceph-mon[134358]: pgmap v282: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-10-06T22:23:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:53 smithi190 ceph-mon[134891]: pgmap v283: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 337 op/s 2023-10-06T22:23:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:53 smithi057 ceph-mon[98492]: pgmap v283: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 337 op/s 2023-10-06T22:23:53.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:53 smithi156 ceph-mon[134358]: pgmap v283: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 337 op/s 2023-10-06T22:23:55.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:55 smithi190 ceph-mon[134891]: pgmap v284: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2023-10-06T22:23:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:55 smithi057 ceph-mon[98492]: pgmap v284: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2023-10-06T22:23:55.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:55 smithi156 ceph-mon[134358]: pgmap v284: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2023-10-06T22:23:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.060+0000 7f786c5ff700 1 -- 172.21.15.57:0/3127629797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7864106f60 msgr2=0x7f7864107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.060+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3127629797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7864106f60 0x7f7864107340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7860009990 tx=0x7f7860009ca0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.061+0000 7f786c5ff700 1 -- 172.21.15.57:0/3127629797 shutdown_connections 2023-10-06T22:23:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.061+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3127629797 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7864068c90 0x7f786410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.061+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3127629797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7864106f60 0x7f7864107340 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.061+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3127629797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78640682d0 0x7f7864068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.061+0000 7f786c5ff700 1 -- 172.21.15.57:0/3127629797 >> 172.21.15.57:0/3127629797 conn(0x7f7864076b70 msgr2=0x7f7864076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.061+0000 7f786c5ff700 1 -- 172.21.15.57:0/3127629797 shutdown_connections 2023-10-06T22:23:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.061+0000 7f786c5ff700 1 -- 172.21.15.57:0/3127629797 wait complete. 2023-10-06T22:23:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.062+0000 7f786c5ff700 1 Processor -- start 2023-10-06T22:23:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.062+0000 7f786c5ff700 1 -- start start 2023-10-06T22:23:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786c5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78640682d0 0x7f78641991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786c5ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7864068c90 0x7f7864199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786c5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7864106f60 0x7f786419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786c5ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786419e3b0 con 0x7f7864068c90 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786c5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786419e520 con 0x7f7864106f60 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786c5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786419e6e0 con 0x7f78640682d0 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78640682d0 0x7f78641991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78640682d0 0x7f78641991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44258/0 (socket says 172.21.15.57:44258) 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 -- 172.21.15.57:0/3732404076 learned_addr learned my addr 172.21.15.57:0/3732404076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786ab9c700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7864106f60 0x7f786419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f7869b9a700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7864068c90 0x7f7864199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 -- 172.21.15.57:0/3732404076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7864106f60 msgr2=0x7f786419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7864106f60 0x7f786419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 -- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7864068c90 msgr2=0x7f7864199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7864068c90 0x7f7864199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.063+0000 7f786a39b700 1 -- 172.21.15.57:0/3732404076 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78600095f0 con 0x7f78640682d0 2023-10-06T22:23:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.064+0000 7f786a39b700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78640682d0 0x7f78641991c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7860005910 tx=0x7f786000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.064+0000 7f785b7fe700 1 -- 172.21.15.57:0/3732404076 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786001d070 con 0x7f78640682d0 2023-10-06T22:23:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.064+0000 7f785b7fe700 1 -- 172.21.15.57:0/3732404076 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f786000fb80 con 0x7f78640682d0 2023-10-06T22:23:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.064+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786419e960 con 0x7f78640682d0 2023-10-06T22:23:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.065+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78641a6200 con 0x7f78640682d0 2023-10-06T22:23:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.064+0000 7f785b7fe700 1 -- 172.21.15.57:0/3732404076 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860017650 con 0x7f78640682d0 2023-10-06T22:23:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.072+0000 7f787fe58b80 1 Processor -- start 2023-10-06T22:23:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.072+0000 7f785b7fe700 1 -- 172.21.15.57:0/3732404076 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7860022470 con 0x7f78640682d0 2023-10-06T22:23:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.072+0000 7f787fe58b80 1 -- start start 2023-10-06T22:23:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x5576758377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837d30 0x5576758426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x557675844fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f787fe58b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557675798400 con 0x557675837410 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f787fe58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557675789bc0 con 0x557675837d30 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f787fe58b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576757874e0 con 0x557675842be0 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f786a39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837d30 0x5576758426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f7869b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x557675844fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f786a39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837d30 0x5576758426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:36476/0 (socket says 172.21.15.57:36476) 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f786a39b700 1 -- 172.21.15.57:0/1846203739 learned_addr learned my addr 172.21.15.57:0/1846203739 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.073+0000 7f786ab9c700 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x5576758377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.074+0000 7f786ab9c700 1 -- 172.21.15.57:0/1846203739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 msgr2=0x557675844fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.074+0000 7f786ab9c700 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x557675844fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.074+0000 7f786ab9c700 1 -- 172.21.15.57:0/1846203739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837d30 msgr2=0x5576758426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.074+0000 7f786ab9c700 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837d30 0x5576758426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.074+0000 7f786ab9c700 1 -- 172.21.15.57:0/1846203739 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576755add80 con 0x557675837410 2023-10-06T22:23:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.075+0000 7f786ab9c700 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x5576758377f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f785c0093a0 tx=0x7f785c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f440648d8e154614 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.075+0000 7f785b7fe700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f78500a16f0 0x7f78500a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.075+0000 7f7858bfe700 1 -- 172.21.15.57:0/1846203739 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00e070 con 0x557675837410 2023-10-06T22:23:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.075+0000 7f7858bfe700 1 -- 172.21.15.57:0/1846203739 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785c00f040 con 0x557675837410 2023-10-06T22:23:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.075+0000 7f7858bfe700 1 -- 172.21.15.57:0/1846203739 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c015e10 con 0x557675837410 2023-10-06T22:23:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.075+0000 7f785b7fe700 1 -- 172.21.15.57:0/3732404076 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f78600ce7d0 con 0x7f78640682d0 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f786a39b700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f78500a16f0 0x7f78500a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f787fe58b80 1 -- 172.21.15.57:0/1846203739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 msgr2=0x5576758377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f787fe58b80 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x5576758377f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f785c0093a0 tx=0x7f785c009920 comp rx=0 tx=0).stop 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f787fe58b80 1 -- 172.21.15.57:0/1846203739 shutdown_connections 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f787fe58b80 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x557675844fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f787fe58b80 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x5576758377f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f787fe58b80 1 --2- 172.21.15.57:0/1846203739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837d30 0x5576758426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.076+0000 7f787fe58b80 1 -- 172.21.15.57:0/1846203739 >> 172.21.15.57:0/1846203739 conn(0x5576757a4200 msgr2=0x55767582aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.077+0000 7f787fe58b80 1 -- 172.21.15.57:0/1846203739 shutdown_connections 2023-10-06T22:23:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.077+0000 7f787fe58b80 1 -- 172.21.15.57:0/1846203739 wait complete. 2023-10-06T22:23:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.077+0000 7f786a39b700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f78500a16f0 0x7f78500a3bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7860009350 tx=0x7f78600d4530 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.077+0000 7f787fe58b80 1 -- start start 2023-10-06T22:23:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.078+0000 7f787fe58b80 1 Processor -- start 2023-10-06T22:23:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675842660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837d30 0x55767583f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x55767583ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f787fe58b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557675782dd0 con 0x557675837d30 2023-10-06T22:23:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f787fe58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767571f6b0 con 0x557675837410 2023-10-06T22:23:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f787fe58b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767571f350 con 0x557675842be0 2023-10-06T22:23:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f787fe58b80 10 client.0 _ll_register_callbacks cb 0x55767557c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f786ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675842660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f786ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675842660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:36480/0 (socket says 172.21.15.57:36480) 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f786ab9c700 1 -- 172.21.15.57:0/3721828584 learned_addr learned my addr 172.21.15.57:0/3721828584 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.079+0000 7f786ab9c700 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x55767583ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f7869b9a700 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837d30 0x55767583f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f786ab9c700 1 -- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 msgr2=0x55767583ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f786ab9c700 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x55767583ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f786ab9c700 1 -- 172.21.15.57:0/3721828584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837d30 msgr2=0x55767583f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f786ab9c700 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837d30 0x55767583f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f786ab9c700 1 -- 172.21.15.57:0/3721828584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c009050 con 0x557675837410 2023-10-06T22:23:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f786ab9c700 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675842660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f785c004cf0 tx=0x7f785c009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec59f75c5bc4afc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.080+0000 7f784dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:23:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.081+0000 7f784dffb700 1 -- 172.21.15.57:0/3721828584 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c025070 con 0x557675837410 2023-10-06T22:23:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.081+0000 7f784dffb700 1 -- 172.21.15.57:0/3721828584 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f785c003a00 con 0x557675837410 2023-10-06T22:23:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.081+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557675822d50 con 0x557675837410 2023-10-06T22:23:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.085+0000 7f784dffb700 1 -- 172.21.15.57:0/3721828584 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f785c00e040 con 0x557675837410 2023-10-06T22:23:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.085+0000 7f787fe58b80 10 client.26912 fetch_fsmap learned FSMap version 58 2023-10-06T22:23:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.085+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x557675596c00 con 0x557675837410 2023-10-06T22:23:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f784dffb700 1 -- 172.21.15.57:0/3721828584 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785c030020 con 0x557675837410 2023-10-06T22:23:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f784dffb700 1 -- 172.21.15.57:0/3721828584 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f785c01f3f0 con 0x557675837410 2023-10-06T22:23:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f787fe58b80 10 client.26912 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:23:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f787fe58b80 10 client.26912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f787fe58b80 20 client.26912 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f787fe58b80 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557675852730 0x557675854b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f787fe58b80 4 client.26912 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:23:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.086+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55767577bdb0 con 0x557675852730 2023-10-06T22:23:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.087+0000 7f7869b9a700 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557675852730 0x557675854b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.087+0000 7f7869b9a700 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557675852730 0x557675854b50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a265052322d718e2 server_cookie=87f0c1e4fed2922e in_seq=0 out_seq=0 2023-10-06T22:23:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.087+0000 7f784dffb700 10 client.26912 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.088+0000 7f784dffb700 1 -- 172.21.15.57:0/3721828584 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55767577bdb0 con 0x557675852730 2023-10-06T22:23:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.088+0000 7f784dffb700 10 client.26912 handle_command_reply: tid=0 2023-10-06T22:23:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.088+0000 7f787fe58b80 1 client.26912 shutdown 2023-10-06T22:23:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 msgr2=0x557675842660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675842660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f785c004cf0 tx=0x7f785c009490 comp rx=0 tx=0).stop 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 shutdown_connections 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x557675852730 0x557675854b50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675842be0 0x55767583ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675842660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 --2- 172.21.15.57:0/3721828584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837d30 0x55767583f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f784dffb700 0 client.26912 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 >> 172.21.15.57:0/3721828584 conn(0x5576757a4200 msgr2=0x55767582abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 shutdown_connections 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.089+0000 7f787fe58b80 1 -- 172.21.15.57:0/3721828584 wait complete. 2023-10-06T22:23:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.090+0000 7f787fe58b80 20 client.26912 trim_cache size 0 max 16384 2023-10-06T22:23:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.107+0000 7f787fe58b80 1 Processor -- start 2023-10-06T22:23:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.107+0000 7f787fe58b80 1 -- start start 2023-10-06T22:23:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x55767583ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675849720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675842be0 0x557675842190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f787fe58b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576756038c0 con 0x557675837410 2023-10-06T22:23:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f787fe58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557675849d10 con 0x557675842be0 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f787fe58b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55767583b7d0 con 0x557675837d30 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f786ab9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x55767583ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f7869b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675849720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f786ab9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x55767583ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46922/0 (socket says 172.21.15.57:46922) 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f786ab9c700 1 -- 172.21.15.57:0/169150927 learned_addr learned my addr 172.21.15.57:0/169150927 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.108+0000 7f786ab9c700 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675842be0 0x557675842190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f786ab9c700 1 -- 172.21.15.57:0/169150927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 msgr2=0x557675849720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f786ab9c700 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675849720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f786ab9c700 1 -- 172.21.15.57:0/169150927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675842be0 msgr2=0x557675842190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f786ab9c700 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675842be0 0x557675842190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f786ab9c700 1 -- 172.21.15.57:0/169150927 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576755d41f0 con 0x557675837410 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f7869b9a700 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675849720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f786ab9c700 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x55767583ead0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f785c0093a0 tx=0x7f785c008fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ac74e0afa962b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f78593ff700 1 -- 172.21.15.57:0/169150927 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00e040 con 0x557675837410 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f78593ff700 1 -- 172.21.15.57:0/169150927 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785c030070 con 0x557675837410 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.109+0000 7f78593ff700 1 -- 172.21.15.57:0/169150927 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c01f3f0 con 0x557675837410 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 -- 172.21.15.57:0/169150927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 msgr2=0x55767583ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x55767583ead0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f785c0093a0 tx=0x7f785c008fe0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 -- 172.21.15.57:0/169150927 shutdown_connections 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675842be0 0x557675842190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675837410 0x55767583ead0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 --2- 172.21.15.57:0/169150927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675849720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 -- 172.21.15.57:0/169150927 >> 172.21.15.57:0/169150927 conn(0x5576757a4200 msgr2=0x5576758297b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 -- 172.21.15.57:0/169150927 shutdown_connections 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 -- 172.21.15.57:0/169150927 wait complete. 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.110+0000 7f787fe58b80 1 -- start start 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 1 Processor -- start 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675844040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675844580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675842be0 0x557675842fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557675844cb0 con 0x557675842be0 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557675844e20 con 0x557675837410 2023-10-06T22:23:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557675843530 con 0x557675837d30 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f787fe58b80 10 client.0 _ll_register_callbacks cb 0x7f785c009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f786ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675844040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f7869b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675844580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f786ab9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675842be0 0x557675842fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f7869b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675844580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44302/0 (socket says 172.21.15.57:44302) 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.111+0000 7f7869b9a700 1 -- 172.21.15.57:0/4021279869 learned_addr learned my addr 172.21.15.57:0/4021279869 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f786ab9c700 1 -- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 msgr2=0x557675844580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f786ab9c700 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675844580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f786ab9c700 1 -- 172.21.15.57:0/4021279869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 msgr2=0x557675844040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f786ab9c700 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675844040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f786ab9c700 1 -- 172.21.15.57:0/4021279869 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785400e040 con 0x557675842be0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f7869b9a700 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675844580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f786ab9c700 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675842be0 0x557675842fc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f785c007ec0 tx=0x7f785c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be888628f05047 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f784dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f784dffb700 1 -- 172.21.15.57:0/4021279869 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c025070 con 0x557675842be0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f785c00f040 con 0x557675842be0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.112+0000 7f784dffb700 1 -- 172.21.15.57:0/4021279869 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f785c031080 con 0x557675842be0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f784dffb700 1 -- 172.21.15.57:0/4021279869 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f785c00e040 con 0x557675842be0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 10 client.16890 fetch_fsmap learned FSMap version 58 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x557675596c00 con 0x557675842be0 2023-10-06T22:23:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f784dffb700 1 -- 172.21.15.57:0/4021279869 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785c00f040 con 0x557675842be0 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f784dffb700 1 -- 172.21.15.57:0/4021279869 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f785c033070 con 0x557675842be0 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 10 client.16890 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 20 client.16890 populate_metadata read hostname 'smithi057' 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5576758a6f20 0x5576758a9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 4 client.16890 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.113+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55767577bad0 con 0x5576758a6f20 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.114+0000 7f7869b9a700 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5576758a6f20 0x5576758a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.114+0000 7f7869b9a700 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5576758a6f20 0x5576758a9340 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b7e966819aaf33 server_cookie=8accdf9ef5a95ae1 in_seq=0 out_seq=0 2023-10-06T22:23:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.114+0000 7f784dffb700 10 client.16890 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f784dffb700 1 -- 172.21.15.57:0/4021279869 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55767577bad0 con 0x5576758a6f20 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f784dffb700 10 client.16890 handle_command_reply: tid=0 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 client.16890 shutdown 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675842be0 msgr2=0x557675842fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675842be0 0x557675842fc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f785c007ec0 tx=0x7f785c000f40 comp rx=0 tx=0).stop 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 shutdown_connections 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5576758a6f20 0x5576758a9340 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557675842be0 0x557675842fc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557675837410 0x557675844040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f784dffb700 0 client.16890 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:23:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 --2- 172.21.15.57:0/4021279869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557675837d30 0x557675844580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 >> 172.21.15.57:0/4021279869 conn(0x5576757a4200 msgr2=0x55767583ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 shutdown_connections 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 1 -- 172.21.15.57:0/4021279869 wait complete. 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.126+0000 7f787fe58b80 20 client.16890 trim_cache size 0 max 16384 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:23:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:23:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f78500a16f0 msgr2=0x7f78500a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f78500a16f0 0x7f78500a3bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7860009350 tx=0x7f78600d4530 comp rx=0 tx=0).stop 2023-10-06T22:23:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78640682d0 msgr2=0x7f78641991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:23:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78640682d0 0x7f78641991c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7860005910 tx=0x7f786000b610 comp rx=0 tx=0).stop 2023-10-06T22:23:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 shutdown_connections 2023-10-06T22:23:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f78500a16f0 0x7f78500a3bb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7864106f60 0x7f786419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78640682d0 0x7f78641991c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 --2- 172.21.15.57:0/3732404076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7864068c90 0x7f7864199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:23:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.127+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 >> 172.21.15.57:0/3732404076 conn(0x7f7864076b70 msgr2=0x7f786410c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:23:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.128+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 shutdown_connections 2023-10-06T22:23:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:23:56.128+0000 7f786c5ff700 1 -- 172.21.15.57:0/3732404076 wait complete. 2023-10-06T22:23:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:57 smithi057 ceph-mon[98492]: pgmap v285: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-10-06T22:23:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:57 smithi156 ceph-mon[134358]: pgmap v285: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-10-06T22:23:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:57 smithi190 ceph-mon[134891]: pgmap v285: 97 pgs: 97 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-10-06T22:23:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:23:59 smithi190 ceph-mon[134891]: pgmap v286: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2023-10-06T22:23:59.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:23:59 smithi057 ceph-mon[98492]: pgmap v286: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2023-10-06T22:23:59.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:23:59 smithi156 ceph-mon[134358]: pgmap v286: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2023-10-06T22:24:01.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:01 smithi190 ceph-mon[134891]: pgmap v287: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 243 op/s 2023-10-06T22:24:01.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:01 smithi057 ceph-mon[98492]: pgmap v287: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 243 op/s 2023-10-06T22:24:01.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:01 smithi156 ceph-mon[134358]: pgmap v287: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 243 op/s 2023-10-06T22:24:03.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:03 smithi190 ceph-mon[134891]: pgmap v288: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2023-10-06T22:24:03.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:03 smithi057 ceph-mon[98492]: pgmap v288: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2023-10-06T22:24:03.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:03 smithi156 ceph-mon[134358]: pgmap v288: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2023-10-06T22:24:04.166 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:24:04.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 -- 172.21.15.57:0/1599196897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 msgr2=0x7f22f00736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 --2- 172.21.15.57:0/1599196897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f00736c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009990 tx=0x7f22e0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 -- 172.21.15.57:0/1599196897 shutdown_connections 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 --2- 172.21.15.57:0/1599196897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22f010d680 0x7f22f010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 --2- 172.21.15.57:0/1599196897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f00736c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 --2- 172.21.15.57:0/1599196897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22f0073c00 0x7f22f010d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 -- 172.21.15.57:0/1599196897 >> 172.21.15.57:0/1599196897 conn(0x7f22f00fc950 msgr2=0x7f22f00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 -- 172.21.15.57:0/1599196897 shutdown_connections 2023-10-06T22:24:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.376+0000 7f22f4dff700 1 -- 172.21.15.57:0/1599196897 wait complete. 2023-10-06T22:24:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.377+0000 7f22f4dff700 1 Processor -- start 2023-10-06T22:24:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.378+0000 7f22f4dff700 1 -- start start 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.378+0000 7f22f4dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.378+0000 7f22f4dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22f0073c00 0x7f22f0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.378+0000 7f22f4dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22f010d680 0x7f22f019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.378+0000 7f22f4dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f019e1e0 con 0x7f22f0073c00 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.378+0000 7f22f4dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f019e350 con 0x7f22f00732e0 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.378+0000 7f22f4dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f019e510 con 0x7f22f010d680 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f0199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:45094/0 (socket says 172.21.15.57:45094) 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 -- 172.21.15.57:0/3361093131 learned_addr learned my addr 172.21.15.57:0/3361093131 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22e7fff700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22f0073c00 0x7f22f0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22eed9d700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22f010d680 0x7f22f019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 -- 172.21.15.57:0/3361093131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22f010d680 msgr2=0x7f22f019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22f010d680 0x7f22f019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 -- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22f0073c00 msgr2=0x7f22f0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22f0073c00 0x7f22f0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 -- 172.21.15.57:0/3361093131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8009520 con 0x7f22f00732e0 2023-10-06T22:24:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.379+0000 7f22ee59c700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f0199200 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005910 tx=0x7f22e0004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.380+0000 7f22e77fe700 1 -- 172.21.15.57:0/3361093131 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e001d070 con 0x7f22f00732e0 2023-10-06T22:24:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.380+0000 7f22e77fe700 1 -- 172.21.15.57:0/3361093131 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22e0022470 con 0x7f22f00732e0 2023-10-06T22:24:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.380+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e8009c40 con 0x7f22f00732e0 2023-10-06T22:24:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.380+0000 7f22e77fe700 1 -- 172.21.15.57:0/3361093131 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0017650 con 0x7f22f00732e0 2023-10-06T22:24:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.380+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e00095f0 con 0x7f22f00732e0 2023-10-06T22:24:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.383+0000 7f22e77fe700 1 -- 172.21.15.57:0/3361093131 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f22e00225e0 con 0x7f22f00732e0 2023-10-06T22:24:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.384+0000 7f23086ecb80 1 Processor -- start 2023-10-06T22:24:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f22e77fe700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f22dc0a1670 0x7f22dc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f23086ecb80 1 -- start start 2023-10-06T22:24:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f22e77fe700 1 -- 172.21.15.57:0/3361093131 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f22e00ce300 con 0x7f22f00732e0 2023-10-06T22:24:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f22eed9d700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f22dc0a1670 0x7f22dc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f23086ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f862e460 con 0x55e1f86d0890 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f23086ecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f861fc20 con 0x55e1f86c50c0 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.385+0000 7f23086ecb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f861d540 con 0x55e1f86c59e0 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47452/0 (socket says 172.21.15.57:47452) 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 -- 172.21.15.57:0/876107888 learned_addr learned my addr 172.21.15.57:0/876107888 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:04.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22ee59c700 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22eed9d700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f22dc0a1670 0x7f22dc0a3b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009fd0 tx=0x7f22e800b040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 -- 172.21.15.57:0/876107888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 msgr2=0x55e1f86c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22eed9d700 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 -- 172.21.15.57:0/876107888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 msgr2=0x55e1f86d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 -- 172.21.15.57:0/876107888 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1f8443de0 con 0x55e1f86c59e0 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22eed9d700 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e7fff700 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86d0350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009eb0 tx=0x7f22d8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984c8e80b30f9c0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.386+0000 7f22e4ff9700 1 -- 172.21.15.57:0/876107888 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800e670 con 0x55e1f86c59e0 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f22e4ff9700 1 -- 172.21.15.57:0/876107888 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d800ec90 con 0x55e1f86c59e0 2023-10-06T22:24:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f22e4ff9700 1 -- 172.21.15.57:0/876107888 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8019d30 con 0x55e1f86c59e0 2023-10-06T22:24:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 -- 172.21.15.57:0/876107888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 msgr2=0x55e1f86d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86d0350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009eb0 tx=0x7f22d8011720 comp rx=0 tx=0).stop 2023-10-06T22:24:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 -- 172.21.15.57:0/876107888 shutdown_connections 2023-10-06T22:24:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 --2- 172.21.15.57:0/876107888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86d0350 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 -- 172.21.15.57:0/876107888 >> 172.21.15.57:0/876107888 conn(0x55e1f863a260 msgr2=0x55e1f86c1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 -- 172.21.15.57:0/876107888 shutdown_connections 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.387+0000 7f23086ecb80 1 -- 172.21.15.57:0/876107888 wait complete. 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.388+0000 7f23086ecb80 1 -- start start 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.388+0000 7f23086ecb80 1 Processor -- start 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f23086ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f8618e30 con 0x55e1f86d0890 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f23086ecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f85b5710 con 0x55e1f86c50c0 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f23086ecb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f85b53b0 con 0x55e1f86c59e0 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f23086ecb80 10 client.0 _ll_register_callbacks cb 0x55e1f8412800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47456/0 (socket says 172.21.15.57:47456) 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22eed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 -- 172.21.15.57:0/3030905967 learned_addr learned my addr 172.21.15.57:0/3030905967 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 -- 172.21.15.57:0/3030905967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 msgr2=0x55e1f86d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 -- 172.21.15.57:0/3030905967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 msgr2=0x55e1f86cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22e7fff700 1 -- 172.21.15.57:0/3030905967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d8009b10 con 0x55e1f86c59e0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.389+0000 7f22ee59c700 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f22e7fff700 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86cd520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f22d8019870 tx=0x7f22d8005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e2848de951c0c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f22d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f22d5ffb700 1 -- 172.21.15.57:0/3030905967 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800e470 con 0x55e1f86c59e0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1f86b8f10 con 0x55e1f86c59e0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f22d5ffb700 1 -- 172.21.15.57:0/3030905967 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f22d800e5d0 con 0x55e1f86c59e0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f22d5ffb700 1 -- 172.21.15.57:0/3030905967 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d800e790 con 0x55e1f86c59e0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f23086ecb80 10 client.26944 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.390+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e1f842cc60 con 0x55e1f86c59e0 2023-10-06T22:24:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f22d5ffb700 1 -- 172.21.15.57:0/3030905967 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1f86b8f10 con 0x55e1f86c59e0 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f22d5ffb700 1 -- 172.21.15.57:0/3030905967 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f22d80052d0 con 0x55e1f86c59e0 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f23086ecb80 10 client.26944 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f23086ecb80 10 client.26944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f23086ecb80 20 client.26944 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f23086ecb80 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f86e0170 0x55e1f86e2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f23086ecb80 4 client.26944 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1f8611e10 con 0x55e1f86e0170 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.391+0000 7f22ee59c700 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f86e0170 0x55e1f86e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.392+0000 7f22ee59c700 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f86e0170 0x55e1f86e2590 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8cc3a59ed0fb7f server_cookie=3f512cf0d77a28c in_seq=0 out_seq=0 2023-10-06T22:24:04.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.392+0000 7f22d5ffb700 10 client.26944 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:04.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f22d5ffb700 1 -- 172.21.15.57:0/3030905967 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e1f8611e10 con 0x55e1f86e0170 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f22d5ffb700 10 client.26944 handle_command_reply: tid=0 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 client.26944 shutdown 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 msgr2=0x55e1f86cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86cd520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f22d8019870 tx=0x7f22d8005e40 comp rx=0 tx=0).stop 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 shutdown_connections 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f86e0170 0x55e1f86e2590 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f22d5ffb700 0 client.26944 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 --2- 172.21.15.57:0/3030905967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c59e0 0x55e1f86cd520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 >> 172.21.15.57:0/3030905967 conn(0x55e1f863a260 msgr2=0x55e1f86c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 shutdown_connections 2023-10-06T22:24:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 1 -- 172.21.15.57:0/3030905967 wait complete. 2023-10-06T22:24:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.393+0000 7f23086ecb80 20 client.26944 trim_cache size 0 max 16384 2023-10-06T22:24:04.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.411+0000 7f23086ecb80 1 Processor -- start 2023-10-06T22:24:04.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f23086ecb80 1 -- start start 2023-10-06T22:24:04.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86c59e0 0x55e1f86c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86d0890 0x55e1f86c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f23086ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f8499920 con 0x55e1f86c59e0 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f23086ecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f86dcd70 con 0x55e1f86c50c0 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f23086ecb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f86c2300 con 0x55e1f86d0890 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f22e7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86c59e0 0x55e1f86c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:45108/0 (socket says 172.21.15.57:45108) 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f22ee59c700 1 -- 172.21.15.57:0/486776785 learned_addr learned my addr 172.21.15.57:0/486776785 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:04.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.412+0000 7f22eed9d700 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86d0890 0x55e1f86c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22ee59c700 1 -- 172.21.15.57:0/486776785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86d0890 msgr2=0x55e1f86c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22ee59c700 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86d0890 0x55e1f86c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22ee59c700 1 -- 172.21.15.57:0/486776785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86c59e0 msgr2=0x55e1f86c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22ee59c700 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86c59e0 0x55e1f86c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22ee59c700 1 -- 172.21.15.57:0/486776785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1f86bd140 con 0x55e1f86c50c0 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22e7fff700 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86c59e0 0x55e1f86c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22ee59c700 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c7f60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005cb0 tx=0x7f22e001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a202e0724dc9820 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22e57fa700 1 -- 172.21.15.57:0/486776785 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e009ac90 con 0x55e1f86c50c0 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22e57fa700 1 -- 172.21.15.57:0/486776785 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22e009ba30 con 0x55e1f86c50c0 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.413+0000 7f22e57fa700 1 -- 172.21.15.57:0/486776785 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0099540 con 0x55e1f86c50c0 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 -- 172.21.15.57:0/486776785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 msgr2=0x55e1f86c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c7f60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005cb0 tx=0x7f22e001a040 comp rx=0 tx=0).stop 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 -- 172.21.15.57:0/486776785 shutdown_connections 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86d0890 0x55e1f86c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c50c0 0x55e1f86c7f60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 --2- 172.21.15.57:0/486776785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86c59e0 0x55e1f86c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 -- 172.21.15.57:0/486776785 >> 172.21.15.57:0/486776785 conn(0x55e1f863a260 msgr2=0x55e1f86bfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 -- 172.21.15.57:0/486776785 shutdown_connections 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 -- 172.21.15.57:0/486776785 wait complete. 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 -- start start 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.414+0000 7f23086ecb80 1 Processor -- start 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c50c0 0x55e1f86d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c59e0 0x55e1f86d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f23086ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f23086ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f87330b0 con 0x55e1f86d0890 2023-10-06T22:24:04.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f23086ecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f8733220 con 0x55e1f86c59e0 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f23086ecb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f87333e0 con 0x55e1f86c50c0 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f23086ecb80 10 client.0 _ll_register_callbacks cb 0x7f22e0005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f22e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c59e0 0x55e1f86d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f22e7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c59e0 0x55e1f86d38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:45116/0 (socket says 172.21.15.57:45116) 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c50c0 0x55e1f86d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f22e7fff700 1 -- 172.21.15.57:0/2004365642 learned_addr learned my addr 172.21.15.57:0/2004365642 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.415+0000 7f22eed9d700 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22e7fff700 1 -- 172.21.15.57:0/2004365642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c50c0 msgr2=0x55e1f86d33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22e7fff700 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c50c0 0x55e1f86d33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22e7fff700 1 -- 172.21.15.57:0/2004365642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 msgr2=0x55e1f86d3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22e7fff700 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22e7fff700 1 -- 172.21.15.57:0/2004365642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1f842cc60 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22e7fff700 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c59e0 0x55e1f86d38f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22d801ad80 tx=0x7f22d8019ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb726452387653f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22d5ffb700 1 -- 172.21.15.57:0/2004365642 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8011b20 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22d800ec90 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22d5ffb700 1 -- 172.21.15.57:0/2004365642 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f22d8000e40 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22d5ffb700 1 -- 172.21.15.57:0/2004365642 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d80048a0 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f23086ecb80 10 client.26936 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7f22e00cd040 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.416+0000 7f22d5ffb700 1 -- 172.21.15.57:0/2004365642 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d800ec90 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f22d5ffb700 1 -- 172.21.15.57:0/2004365642 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f22d800e870 con 0x55e1f86c59e0 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f23086ecb80 10 client.26936 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f23086ecb80 10 client.26936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f23086ecb80 20 client.26936 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f23086ecb80 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f8734740 0x55e1f8736b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f23086ecb80 4 client.26936 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e1f8611b30 con 0x55e1f8734740 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f22ee59c700 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f8734740 0x55e1f8736b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f22ee59c700 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f8734740 0x55e1f8736b60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151707b6619af58a server_cookie=c8b257b1c6708c29 in_seq=0 out_seq=0 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.417+0000 7f22d5ffb700 10 client.26936 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f22d5ffb700 1 -- 172.21.15.57:0/2004365642 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e1f8611b30 con 0x55e1f8734740 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f22d5ffb700 10 client.26936 handle_command_reply: tid=0 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 client.26936 shutdown 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c59e0 msgr2=0x55e1f86d38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c59e0 0x55e1f86d38f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22d801ad80 tx=0x7f22d8019ea0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 shutdown_connections 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e1f8734740 0x55e1f8736b60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1f86d0890 0x55e1f86d3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e1f86c50c0 0x55e1f86d33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f22d5ffb700 0 client.26936 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 --2- 172.21.15.57:0/2004365642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f86c59e0 0x55e1f86d38f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 >> 172.21.15.57:0/2004365642 conn(0x55e1f863a260 msgr2=0x55e1f86c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 shutdown_connections 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 1 -- 172.21.15.57:0/2004365642 wait complete. 2023-10-06T22:24:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.418+0000 7f23086ecb80 20 client.26936 trim_cache size 0 max 16384 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:24:04.427 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:24:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f22dc0a1670 msgr2=0x7f22dc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f22dc0a1670 0x7f22dc0a3b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009fd0 tx=0x7f22e800b040 comp rx=0 tx=0).stop 2023-10-06T22:24:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 msgr2=0x7f22f0199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f0199200 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005910 tx=0x7f22e0004710 comp rx=0 tx=0).stop 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 shutdown_connections 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f22dc0a1670 0x7f22dc0a3b30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22f010d680 0x7f22f019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22f00732e0 0x7f22f0199200 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 --2- 172.21.15.57:0/3361093131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22f0073c00 0x7f22f0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 >> 172.21.15.57:0/3361093131 conn(0x7f22f00fc950 msgr2=0x7f22f010f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 shutdown_connections 2023-10-06T22:24:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:04.419+0000 7f22f4dff700 1 -- 172.21.15.57:0/3361093131 wait complete. 2023-10-06T22:24:04.439 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:24:05.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:05 smithi190 ceph-mon[134891]: pgmap v289: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2023-10-06T22:24:05.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:05 smithi057 ceph-mon[98492]: pgmap v289: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2023-10-06T22:24:05.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:05 smithi156 ceph-mon[134358]: pgmap v289: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2023-10-06T22:24:07.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:07 smithi057 ceph-mon[98492]: pgmap v290: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2023-10-06T22:24:07.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:07 smithi156 ceph-mon[134358]: pgmap v290: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2023-10-06T22:24:07.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:07 smithi190 ceph-mon[134891]: pgmap v290: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2023-10-06T22:24:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 -- 172.21.15.57:0/2774948127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa410101a80 msgr2=0x7fa410105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 --2- 172.21.15.57:0/2774948127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa410101a80 0x7fa410105ac0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa4040098f0 tx=0x7fa404009c00 comp rx=0 tx=0).stop 2023-10-06T22:24:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 -- 172.21.15.57:0/2774948127 shutdown_connections 2023-10-06T22:24:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 --2- 172.21.15.57:0/2774948127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101061f0 0x7fa41010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 --2- 172.21.15.57:0/2774948127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4101010d0 0x7fa4101014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 --2- 172.21.15.57:0/2774948127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa410101a80 0x7fa410105ac0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 -- 172.21.15.57:0/2774948127 >> 172.21.15.57:0/2774948127 conn(0x7fa4100fc930 msgr2=0x7fa4100fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 -- 172.21.15.57:0/2774948127 shutdown_connections 2023-10-06T22:24:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.281+0000 7fa4157ff700 1 -- 172.21.15.57:0/2774948127 wait complete. 2023-10-06T22:24:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.282+0000 7fa4157ff700 1 Processor -- start 2023-10-06T22:24:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.282+0000 7fa4157ff700 1 -- start start 2023-10-06T22:24:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.282+0000 7fa4157ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101010d0 0x7fa410199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.282+0000 7fa4157ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4101061f0 0x7fa410199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa4157ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa41019ddd0 0x7fa41019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa4157ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41019e850 con 0x7fa41019ddd0 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa4157ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41019e9c0 con 0x7fa4101010d0 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa4157ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41019eb80 con 0x7fa4101061f0 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101010d0 0x7fa410199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101010d0 0x7fa410199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:45138/0 (socket says 172.21.15.57:45138) 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 -- 172.21.15.57:0/3701024216 learned_addr learned my addr 172.21.15.57:0/3701024216 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4101061f0 0x7fa410199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40f7fe700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa41019ddd0 0x7fa41019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 -- 172.21.15.57:0/3701024216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4101061f0 msgr2=0x7fa410199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4101061f0 0x7fa410199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 -- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa41019ddd0 msgr2=0x7fa41019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa41019ddd0 0x7fa41019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 -- 172.21.15.57:0/3701024216 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408009d00 con 0x7fa4101010d0 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4101061f0 0x7fa410199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40f7fe700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa41019ddd0 0x7fa41019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.283+0000 7fa40effd700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101010d0 0x7fa410199220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e9a0 tx=0x7fa40000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.284+0000 7fa3f7fff700 1 -- 172.21.15.57:0/3701024216 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400009790 con 0x7fa4101010d0 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.284+0000 7fa3f7fff700 1 -- 172.21.15.57:0/3701024216 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40000cbd0 con 0x7fa4101010d0 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.284+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa404009520 con 0x7fa4101010d0 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.284+0000 7fa3f7fff700 1 -- 172.21.15.57:0/3701024216 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400018680 con 0x7fa4101010d0 2023-10-06T22:24:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.284+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101a6430 con 0x7fa4101010d0 2023-10-06T22:24:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.287+0000 7fa3f7fff700 1 -- 172.21.15.57:0/3701024216 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa40000cd40 con 0x7fa4101010d0 2023-10-06T22:24:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.288+0000 7fa429070b80 1 Processor -- start 2023-10-06T22:24:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa429070b80 1 -- start start 2023-10-06T22:24:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa3f7fff700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3f80a15f0 0x7fa3f80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa429070b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b29fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa429070b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa429070b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa3f7fff700 1 -- 172.21.15.57:0/3701024216 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fa400014070 con 0x7fa4101010d0 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa429070b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1a93400 con 0x55aaf1b2a520 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa429070b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1a84bc0 con 0x55aaf1b3dbe0 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa429070b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1a824e0 con 0x55aaf1b29c00 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa40f7fe700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3f80a15f0 0x7fa3f80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa40effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b29fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa40effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b29fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47512/0 (socket says 172.21.15.57:47512) 2023-10-06T22:24:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa40effd700 1 -- 172.21.15.57:0/4248542607 learned_addr learned my addr 172.21.15.57:0/4248542607 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.289+0000 7fa40f7fe700 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa40effd700 1 -- 172.21.15.57:0/4248542607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 msgr2=0x55aaf1b3ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa40effd700 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa40effd700 1 -- 172.21.15.57:0/4248542607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 msgr2=0x55aaf1b3d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa40effd700 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa40effd700 1 -- 172.21.15.57:0/4248542607 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf18a8d80 con 0x55aaf1b29c00 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa40effd700 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b29fe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa4000a21f0 tx=0x7fa400096800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2467750176c83f93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa40f7fe700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3f80a15f0 0x7fa3f80a3ab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa408007980 tx=0x7fa408009190 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa3f57fa700 1 -- 172.21.15.57:0/4248542607 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400055950 con 0x55aaf1b29c00 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa3f57fa700 1 -- 172.21.15.57:0/4248542607 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40009aa90 con 0x55aaf1b29c00 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.290+0000 7fa3f57fa700 1 -- 172.21.15.57:0/4248542607 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40009a5f0 con 0x55aaf1b29c00 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 -- 172.21.15.57:0/4248542607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 msgr2=0x55aaf1b29fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b29fe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa4000a21f0 tx=0x7fa400096800 comp rx=0 tx=0).stop 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 -- 172.21.15.57:0/4248542607 shutdown_connections 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b29fe0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 --2- 172.21.15.57:0/4248542607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 -- 172.21.15.57:0/4248542607 >> 172.21.15.57:0/4248542607 conn(0x55aaf1a9f200 msgr2=0x55aaf1b26080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 -- 172.21.15.57:0/4248542607 shutdown_connections 2023-10-06T22:24:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.291+0000 7fa429070b80 1 -- 172.21.15.57:0/4248542607 wait complete. 2023-10-06T22:24:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.292+0000 7fa429070b80 1 -- start start 2023-10-06T22:24:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.292+0000 7fa429070b80 1 Processor -- start 2023-10-06T22:24:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa429070b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b3d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa429070b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa429070b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa429070b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1a7ddd0 con 0x55aaf1b29c00 2023-10-06T22:24:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa429070b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1a1a6b0 con 0x55aaf1b3dbe0 2023-10-06T22:24:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa429070b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1a1a350 con 0x55aaf1b2a520 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa429070b80 10 client.0 _ll_register_callbacks cb 0x55aaf18777a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b3d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47518/0 (socket says 172.21.15.57:47518) 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b3d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41810/0 (socket says 172.21.15.57:41810) 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 -- 172.21.15.57:0/712698881 learned_addr learned my addr 172.21.15.57:0/712698881 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 -- 172.21.15.57:0/712698881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 msgr2=0x55aaf1b3b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 -- 172.21.15.57:0/712698881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 msgr2=0x55aaf1b3d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b3d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40e7fc700 1 -- 172.21.15.57:0/712698881 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101a6430 con 0x55aaf1b2a520 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.293+0000 7fa40effd700 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b3d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3a8c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa404005e90 tx=0x7fa40400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64b60536c14a14cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa3f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/712698881 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40401c040 con 0x55aaf1b2a520 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaf1b1dd50 con 0x55aaf1b2a520 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/712698881 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fa40400bb60 con 0x55aaf1b2a520 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/712698881 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40400be90 con 0x55aaf1b2a520 2023-10-06T22:24:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa429070b80 10 client.26962 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55aaf1891c00 con 0x55aaf1b2a520 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.294+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/712698881 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aaf1b1dd50 con 0x55aaf1b2a520 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/712698881 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fa404015020 con 0x55aaf1b2a520 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa429070b80 10 client.26962 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa429070b80 10 client.26962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa429070b80 20 client.26962 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa429070b80 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1b4d790 0x55aaf1b4fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa429070b80 4 client.26962 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaf1a76db0 con 0x55aaf1b4d790 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa40effd700 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1b4d790 0x55aaf1b4fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa40effd700 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1b4d790 0x55aaf1b4fbb0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567f1f9c8ded980f server_cookie=ed363772500f640e in_seq=0 out_seq=0 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.295+0000 7fa3f2ff5700 10 client.26962 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.296+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/712698881 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aaf1a76db0 con 0x55aaf1b4d790 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.296+0000 7fa3f2ff5700 10 client.26962 handle_command_reply: tid=0 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.296+0000 7fa429070b80 1 client.26962 shutdown 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 msgr2=0x55aaf1b3a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3a8c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa404005e90 tx=0x7fa40400b730 comp rx=0 tx=0).stop 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 shutdown_connections 2023-10-06T22:24:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1b4d790 0x55aaf1b4fbb0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b3b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b3d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 --2- 172.21.15.57:0/712698881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3a8c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa3f2ff5700 0 client.26962 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 >> 172.21.15.57:0/712698881 conn(0x55aaf1a9f200 msgr2=0x55aaf1b25c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 shutdown_connections 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 1 -- 172.21.15.57:0/712698881 wait complete. 2023-10-06T22:24:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.297+0000 7fa429070b80 20 client.26962 trim_cache size 0 max 16384 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 Processor -- start 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 -- start start 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b44720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b35c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf18fe8c0 con 0x55aaf1b29c00 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1b44cd0 con 0x55aaf1b3dbe0 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.312+0000 7fa429070b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1b38e70 con 0x55aaf1b2a520 2023-10-06T22:24:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b35c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b44720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47534/0 (socket says 172.21.15.57:47534) 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b35c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:45156/0 (socket says 172.21.15.57:45156) 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 -- 172.21.15.57:0/1730005843 learned_addr learned my addr 172.21.15.57:0/1730005843 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 -- 172.21.15.57:0/1730005843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 msgr2=0x55aaf1b35c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/1730005843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b35c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 -- 172.21.15.57:0/1730005843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 msgr2=0x55aaf1b44720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/1730005843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b44720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 -- 172.21.15.57:0/1730005843 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf18cf1f0 con 0x55aaf1b2a520 2023-10-06T22:24:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.313+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/1730005843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d270 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa404010ff0 tx=0x7fa404009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a009d4534a808a57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa3f5ffb700 1 -- 172.21.15.57:0/1730005843 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40401c040 con 0x55aaf1b2a520 2023-10-06T22:24:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa3f5ffb700 1 -- 172.21.15.57:0/1730005843 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40401d040 con 0x55aaf1b2a520 2023-10-06T22:24:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa3f5ffb700 1 -- 172.21.15.57:0/1730005843 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404003ae0 con 0x55aaf1b2a520 2023-10-06T22:24:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 -- 172.21.15.57:0/1730005843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 msgr2=0x55aaf1b3d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 --2- 172.21.15.57:0/1730005843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d270 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa404010ff0 tx=0x7fa404009c00 comp rx=0 tx=0).stop 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 -- 172.21.15.57:0/1730005843 shutdown_connections 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 --2- 172.21.15.57:0/1730005843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b35c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 --2- 172.21.15.57:0/1730005843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b29c00 0x55aaf1b44720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 --2- 172.21.15.57:0/1730005843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b2a520 0x55aaf1b3d270 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 -- 172.21.15.57:0/1730005843 >> 172.21.15.57:0/1730005843 conn(0x55aaf1a9f200 msgr2=0x55aaf1b499c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 -- 172.21.15.57:0/1730005843 shutdown_connections 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.314+0000 7fa429070b80 1 -- 172.21.15.57:0/1730005843 wait complete. 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 -- start start 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 Processor -- start 2023-10-06T22:24:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b36b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1b37b70 con 0x55aaf1b2a520 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1b37ce0 con 0x55aaf1b3dbe0 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf1b37ea0 con 0x55aaf1b29c00 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.315+0000 7fa429070b80 10 client.0 _ll_register_callbacks cb 0x7fa40400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b36b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b36b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:47544/0 (socket says 172.21.15.57:47544) 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 -- 172.21.15.57:0/2383852479 learned_addr learned my addr 172.21.15.57:0/2383852479 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40f7fe700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 -- 172.21.15.57:0/2383852479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 msgr2=0x55aaf1b375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b375e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 -- 172.21.15.57:0/2383852479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 msgr2=0x55aaf1b370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b370a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 -- 172.21.15.57:0/2383852479 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaf1891c00 con 0x55aaf1b29c00 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40f7fe700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa40effd700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b36b60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa40009a320 tx=0x7fa400096760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37489e7ea41989b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa3f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/2383852479 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40009e120 con 0x55aaf1b29c00 2023-10-06T22:24:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.316+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa40009aa90 con 0x55aaf1b29c00 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/2383852479 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fa40009e280 con 0x55aaf1b29c00 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/2383852479 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa400099880 con 0x55aaf1b29c00 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 10 client.26968 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55aaf18a7250 con 0x55aaf1b29c00 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/2383852479 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40009aa90 con 0x55aaf1b29c00 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/2383852479 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fa40009a5f0 con 0x55aaf1b29c00 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 10 client.26968 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 10 client.26968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 20 client.26968 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1ba1e50 0x55aaf1ba4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 4 client.26968 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:24:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.317+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aaf1a76ad0 con 0x55aaf1ba1e50 2023-10-06T22:24:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.318+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1ba1e50 0x55aaf1ba4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.318+0000 7fa40e7fc700 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1ba1e50 0x55aaf1ba4270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8794b267c848df08 server_cookie=5a3eb226ab4ad771 in_seq=0 out_seq=0 2023-10-06T22:24:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.318+0000 7fa3f2ff5700 10 client.26968 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.341+0000 7fa3f2ff5700 1 -- 172.21.15.57:0/2383852479 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aaf1a76ad0 con 0x55aaf1ba1e50 2023-10-06T22:24:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa3f2ff5700 10 client.26968 handle_command_reply: tid=0 2023-10-06T22:24:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 client.26968 shutdown 2023-10-06T22:24:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 msgr2=0x55aaf1b36b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b36b60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa40009a320 tx=0x7fa400096760 comp rx=0 tx=0).stop 2023-10-06T22:24:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 shutdown_connections 2023-10-06T22:24:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf1ba1e50 0x55aaf1ba4270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf1b3dbe0 0x55aaf1b375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf1b29c00 0x55aaf1b36b60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 --2- 172.21.15.57:0/2383852479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf1b2a520 0x55aaf1b370a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa3f2ff5700 0 client.26968 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 >> 172.21.15.57:0/2383852479 conn(0x55aaf1a9f200 msgr2=0x55aaf1b35c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 shutdown_connections 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 1 -- 172.21.15.57:0/2383852479 wait complete. 2023-10-06T22:24:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.342+0000 7fa429070b80 20 client.26968 trim_cache size 0 max 16384 2023-10-06T22:24:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:24:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:24:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:24:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3f80a15f0 msgr2=0x7fa3f80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3f80a15f0 0x7fa3f80a3ab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa408007980 tx=0x7fa408009190 comp rx=0 tx=0).stop 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101010d0 msgr2=0x7fa410199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101010d0 0x7fa410199220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e9a0 tx=0x7fa40000ecb0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 shutdown_connections 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fa3f80a15f0 0x7fa3f80a3ab0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa41019ddd0 0x7fa41019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4101010d0 0x7fa410199220 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 --2- 172.21.15.57:0/3701024216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4101061f0 0x7fa410199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 >> 172.21.15.57:0/3701024216 conn(0x7fa4100fc930 msgr2=0x7fa4100fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 shutdown_connections 2023-10-06T22:24:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:09.343+0000 7fa4157ff700 1 -- 172.21.15.57:0/3701024216 wait complete. 2023-10-06T22:24:09.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:09 smithi190 ceph-mon[134891]: pgmap v291: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2023-10-06T22:24:09.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:09 smithi057 ceph-mon[98492]: pgmap v291: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2023-10-06T22:24:09.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:09 smithi156 ceph-mon[134358]: pgmap v291: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2023-10-06T22:24:11.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:11 smithi190 ceph-mon[134891]: pgmap v292: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2023-10-06T22:24:11.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:11 smithi190 ceph-mon[134891]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T22:24:11.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:11 smithi057 ceph-mon[98492]: pgmap v292: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2023-10-06T22:24:11.715 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:11 smithi057 ceph-mon[98492]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T22:24:11.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:11 smithi156 ceph-mon[134358]: pgmap v292: 97 pgs: 97 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2023-10-06T22:24:11.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:11 smithi156 ceph-mon[134358]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T22:24:13.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:13 smithi190 ceph-mon[134891]: pgmap v294: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2023-10-06T22:24:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:13 smithi057 ceph-mon[98492]: pgmap v294: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2023-10-06T22:24:13.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:13 smithi156 ceph-mon[134358]: pgmap v294: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2023-10-06T22:24:15.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:15 smithi190 ceph-mon[134891]: pgmap v295: 97 pgs: 21 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 39 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 225 op/s 2023-10-06T22:24:15.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:15 smithi057 ceph-mon[98492]: pgmap v295: 97 pgs: 21 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 39 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 225 op/s 2023-10-06T22:24:15.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:15 smithi156 ceph-mon[134358]: pgmap v295: 97 pgs: 21 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 39 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 225 op/s 2023-10-06T22:24:17.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:17 smithi057 ceph-mon[98492]: pgmap v296: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2023-10-06T22:24:17.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:17 smithi156 ceph-mon[134358]: pgmap v296: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2023-10-06T22:24:17.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:17 smithi190 ceph-mon[134891]: pgmap v296: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2023-10-06T22:24:19.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:19 smithi190 ceph-mon[134891]: pgmap v297: 97 pgs: 16 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 51 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 230 op/s 2023-10-06T22:24:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:19 smithi057 ceph-mon[98492]: pgmap v297: 97 pgs: 16 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 51 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 230 op/s 2023-10-06T22:24:19.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:19 smithi156 ceph-mon[134358]: pgmap v297: 97 pgs: 16 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 51 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 230 op/s 2023-10-06T22:24:21.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:21 smithi190 ceph-mon[134891]: pgmap v298: 97 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 61 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2023-10-06T22:24:21.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:21 smithi057 ceph-mon[98492]: pgmap v298: 97 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 61 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2023-10-06T22:24:21.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:21 smithi156 ceph-mon[134358]: pgmap v298: 97 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 61 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 -- 172.21.15.57:0/4203209708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac106040 msgr2=0x7faeac10dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/4203209708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac106040 0x7faeac10dbc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faea400b0a0 tx=0x7faea400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 -- 172.21.15.57:0/4203209708 shutdown_connections 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/4203209708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac106040 0x7faeac10dbc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/4203209708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac100f20 0x7faeac101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/4203209708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac1018d0 0x7faeac105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 -- 172.21.15.57:0/4203209708 >> 172.21.15.57:0/4203209708 conn(0x7faeac0fc7c0 msgr2=0x7faeac0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 -- 172.21.15.57:0/4203209708 shutdown_connections 2023-10-06T22:24:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.494+0000 7faeb2b9d700 1 -- 172.21.15.57:0/4203209708 wait complete. 2023-10-06T22:24:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.495+0000 7faeb2b9d700 1 Processor -- start 2023-10-06T22:24:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb2b9d700 1 -- start start 2023-10-06T22:24:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb2b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac100f20 0x7faeac198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb2b9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac106040 0x7faeac199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb2b9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac19d8a0 0x7faeac19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb2b9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac19e320 con 0x7faeac106040 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb2b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac19e490 con 0x7faeac100f20 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb2b9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac19e650 con 0x7faeac19d8a0 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb139a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac106040 0x7faeac199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb1b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac100f20 0x7faeac198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb139a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac106040 0x7faeac199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50568/0 (socket says 172.21.15.57:50568) 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb239c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac19d8a0 0x7faeac19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.496+0000 7faeb139a700 1 -- 172.21.15.57:0/3386466564 learned_addr learned my addr 172.21.15.57:0/3386466564 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb1b9b700 1 -- 172.21.15.57:0/3386466564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac19d8a0 msgr2=0x7faeac19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac19d8a0 0x7faeac19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb1b9b700 1 -- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac106040 msgr2=0x7faeac199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac106040 0x7faeac199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb1b9b700 1 -- 172.21.15.57:0/3386466564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4009d00 con 0x7faeac100f20 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb239c700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac19d8a0 0x7faeac19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb139a700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac106040 0x7faeac199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac100f20 0x7faeac198f00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faea800e9f0 tx=0x7faea800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faea2ffd700 1 -- 172.21.15.57:0/3386466564 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800c9c0 con 0x7faeac100f20 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faea2ffd700 1 -- 172.21.15.57:0/3386466564 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea8010430 con 0x7faeac100f20 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac1a5bb0 con 0x7faeac100f20 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faea2ffd700 1 -- 172.21.15.57:0/3386466564 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8018680 con 0x7faeac100f20 2023-10-06T22:24:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.497+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac1a60e0 con 0x7faeac100f20 2023-10-06T22:24:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.501+0000 7faea2ffd700 1 -- 172.21.15.57:0/3386466564 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7faea8010860 con 0x7faeac100f20 2023-10-06T22:24:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faec76f2b80 1 Processor -- start 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faea2ffd700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fae980a1670 0x7fae980a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faec76f2b80 1 -- start start 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faea2ffd700 1 -- 172.21.15.57:0/3386466564 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8536+0+0 (secure 0 0 0) 0x7faea8014070 con 0x7faeac100f20 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faeb239c700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fae980a1670 0x7fae980a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752ad60 0x56112752b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752b680 0x561127535ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.502+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 0x561127538920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faec76f2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561127494400 con 0x56112752b680 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faec76f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561127485bc0 con 0x561127536530 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faec76f2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611274834e0 con 0x56112752ad60 2023-10-06T22:24:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752ad60 0x56112752b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752ad60 0x56112752b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:34852/0 (socket says 172.21.15.57:34852) 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 -- 172.21.15.57:0/1053725795 learned_addr learned my addr 172.21.15.57:0/1053725795 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb139a700 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752b680 0x561127535ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 -- 172.21.15.57:0/1053725795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 msgr2=0x561127538920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 0x561127538920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 -- 172.21.15.57:0/1053725795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752b680 msgr2=0x561127535ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752b680 0x561127535ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 -- 172.21.15.57:0/1053725795 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611272a9d80 con 0x56112752ad60 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb239c700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fae980a1670 0x7fae980a3b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faea4009040 tx=0x7faea400bc30 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.503+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752ad60 0x56112752b140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faea8096c30 tx=0x7faea8099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c0e8b9c0dd3b64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7fae97fff700 1 -- 172.21.15.57:0/1053725795 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea809e120 con 0x56112752ad60 2023-10-06T22:24:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7fae97fff700 1 -- 172.21.15.57:0/1053725795 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea809e310 con 0x56112752ad60 2023-10-06T22:24:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7fae97fff700 1 -- 172.21.15.57:0/1053725795 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80996b0 con 0x56112752ad60 2023-10-06T22:24:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 -- 172.21.15.57:0/1053725795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752ad60 msgr2=0x56112752b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752ad60 0x56112752b140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faea8096c30 tx=0x7faea8099c20 comp rx=0 tx=0).stop 2023-10-06T22:24:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 -- 172.21.15.57:0/1053725795 shutdown_connections 2023-10-06T22:24:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 0x561127538920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752ad60 0x56112752b140 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1053725795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752b680 0x561127535ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 -- 172.21.15.57:0/1053725795 >> 172.21.15.57:0/1053725795 conn(0x5611274a0200 msgr2=0x5611275271b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.504+0000 7faec76f2b80 1 -- 172.21.15.57:0/1053725795 shutdown_connections 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.505+0000 7faec76f2b80 1 -- 172.21.15.57:0/1053725795 wait complete. 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.505+0000 7faec76f2b80 1 -- start start 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.505+0000 7faec76f2b80 1 Processor -- start 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.506+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x561127533250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.506+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752b680 0x561127533790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.506+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561127536530 0x561127533cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.506+0000 7faec76f2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112747edd0 con 0x56112752ad60 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.506+0000 7faec76f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112741b6b0 con 0x56112752b680 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.506+0000 7faec76f2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112741b350 con 0x561127536530 2023-10-06T22:24:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.506+0000 7faec76f2b80 10 client.0 _ll_register_callbacks cb 0x5611272787a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752b680 0x561127533790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb239c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561127536530 0x561127533cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752b680 0x561127533790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:40014/0 (socket says 172.21.15.57:40014) 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 -- 172.21.15.57:0/1222726873 learned_addr learned my addr 172.21.15.57:0/1222726873 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x561127533250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 -- 172.21.15.57:0/1222726873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561127536530 msgr2=0x561127533cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561127536530 0x561127533cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 -- 172.21.15.57:0/1222726873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 msgr2=0x561127533250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x561127533250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 -- 172.21.15.57:0/1222726873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac1a60e0 con 0x56112752b680 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x561127533250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:24:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7faeb139a700 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752b680 0x561127533790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fae9c012cb0 tx=0x7fae9c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d733f02bc9088cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7fae957fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.507+0000 7fae957fa700 1 -- 172.21.15.57:0/1222726873 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00fb70 con 0x56112752b680 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56112751eeb0 con 0x56112752b680 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7fae957fa700 1 -- 172.21.15.57:0/1222726873 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8536+0+0 (secure 0 0 0) 0x7fae9c00fcd0 con 0x56112752b680 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7fae957fa700 1 -- 172.21.15.57:0/1222726873 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae9c00fe90 con 0x56112752b680 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 10 client.26963 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x561127292c00 con 0x56112752b680 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7fae957fa700 1 -- 172.21.15.57:0/1222726873 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56112751eeb0 con 0x56112752b680 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7fae957fa700 1 -- 172.21.15.57:0/1222726873 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fae9c005030 con 0x56112752b680 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 10 client.26963 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 10 client.26963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 20 client.26963 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611275460f0 0x561127548510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 4 client.26963 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:24:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.508+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561127477db0 con 0x5611275460f0 2023-10-06T22:24:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.509+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611275460f0 0x561127548510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.509+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611275460f0 0x561127548510 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560c6ed6bc516e03 server_cookie=d31f23d2f658bd0a in_seq=0 out_seq=0 2023-10-06T22:24:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.509+0000 7fae957fa700 10 client.26963 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7fae957fa700 1 -- 172.21.15.57:0/1222726873 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561127477db0 con 0x5611275460f0 2023-10-06T22:24:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7fae957fa700 10 client.26963 handle_command_reply: tid=0 2023-10-06T22:24:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 client.26963 shutdown 2023-10-06T22:24:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752b680 msgr2=0x561127533790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752b680 0x561127533790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fae9c012cb0 tx=0x7fae9c009e30 comp rx=0 tx=0).stop 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 shutdown_connections 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5611275460f0 0x561127548510 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561127536530 0x561127533cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7fae957fa700 0 client.26963 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x561127533250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 --2- 172.21.15.57:0/1222726873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752b680 0x561127533790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 >> 172.21.15.57:0/1222726873 conn(0x5611274a0200 msgr2=0x561127526da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 shutdown_connections 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.510+0000 7faec76f2b80 1 -- 172.21.15.57:0/1222726873 wait complete. 2023-10-06T22:24:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.511+0000 7faec76f2b80 20 client.26963 trim_cache size 0 max 16384 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.525+0000 7faec76f2b80 1 Processor -- start 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faec76f2b80 1 -- start start 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x56112752dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112753d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 0x5611275337c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faec76f2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611272ff8c0 con 0x56112752ad60 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faec76f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112752b4c0 con 0x561127536530 2023-10-06T22:24:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faec76f2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561127521420 con 0x56112752b680 2023-10-06T22:24:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb1b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x56112752dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb1b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x56112752dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50600/0 (socket says 172.21.15.57:50600) 2023-10-06T22:24:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb1b9b700 1 -- 172.21.15.57:0/3098658162 learned_addr learned my addr 172.21.15.57:0/3098658162 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb239c700 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 0x5611275337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb139a700 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112753d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb1b9b700 1 -- 172.21.15.57:0/3098658162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 msgr2=0x56112753d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112753d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.526+0000 7faeb1b9b700 1 -- 172.21.15.57:0/3098658162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 msgr2=0x5611275337c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.527+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 0x5611275337c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.527+0000 7faeb1b9b700 1 -- 172.21.15.57:0/3098658162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56112754d140 con 0x56112752ad60 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.527+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x56112752dc00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faea809a150 tx=0x7faea8099f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b808b76aa8638fe5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.527+0000 7faea0bff700 1 -- 172.21.15.57:0/3098658162 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8098560 con 0x56112752ad60 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.527+0000 7faea0bff700 1 -- 172.21.15.57:0/3098658162 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea80105a0 con 0x56112752ad60 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.527+0000 7faea0bff700 1 -- 172.21.15.57:0/3098658162 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea809a680 con 0x56112752ad60 2023-10-06T22:24:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 -- 172.21.15.57:0/3098658162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 msgr2=0x56112752dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x56112752dc00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faea809a150 tx=0x7faea8099f90 comp rx=0 tx=0).stop 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 -- 172.21.15.57:0/3098658162 shutdown_connections 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561127536530 0x5611275337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56112752ad60 0x56112752dc00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 --2- 172.21.15.57:0/3098658162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112753d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 -- 172.21.15.57:0/3098658162 >> 172.21.15.57:0/3098658162 conn(0x5611274a0200 msgr2=0x56112752e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 -- 172.21.15.57:0/3098658162 shutdown_connections 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 -- 172.21.15.57:0/3098658162 wait complete. 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 -- start start 2023-10-06T22:24:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.528+0000 7faec76f2b80 1 Processor -- start 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752ad60 0x56112758deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112758e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faec76f2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561127536530 0x56112758e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faec76f2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561127598f70 con 0x561127536530 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faec76f2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611275990b0 con 0x56112752ad60 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faec76f2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561127599270 con 0x56112752b680 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faec76f2b80 10 client.0 _ll_register_callbacks cb 0x7fae9c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faeb1b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752ad60 0x56112758deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faeb139a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112758e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faeb1b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752ad60 0x56112758deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:40034/0 (socket says 172.21.15.57:40034) 2023-10-06T22:24:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faeb1b9b700 1 -- 172.21.15.57:0/2870428732 learned_addr learned my addr 172.21.15.57:0/2870428732 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.529+0000 7faeb239c700 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561127536530 0x56112758e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faeb1b9b700 1 -- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 msgr2=0x56112758e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112758e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faeb1b9b700 1 -- 172.21.15.57:0/2870428732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561127536530 msgr2=0x56112758e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561127536530 0x56112758e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faeb1b9b700 1 -- 172.21.15.57:0/2870428732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea80b1040 con 0x56112752ad60 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faeb139a700 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112758e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faeb1b9b700 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752ad60 0x56112758deb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faea8011fd0 tx=0x7faea8096630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbe997c8fa351cc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7fae957fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7fae957fa700 1 -- 172.21.15.57:0/2870428732 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80a25d0 con 0x56112752ad60 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faea809e310 con 0x56112752ad60 2023-10-06T22:24:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7fae957fa700 1 -- 172.21.15.57:0/2870428732 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8536+0+0 (secure 0 0 0) 0x7faea8099860 con 0x56112752ad60 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7fae957fa700 1 -- 172.21.15.57:0/2870428732 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea809a420 con 0x56112752ad60 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faec76f2b80 10 client.26969 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.530+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5611272d01f0 con 0x56112752ad60 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7fae957fa700 1 -- 172.21.15.57:0/2870428732 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea809e310 con 0x56112752ad60 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7fae957fa700 1 -- 172.21.15.57:0/2870428732 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7faea80a20c0 con 0x56112752ad60 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7faec76f2b80 10 client.26969 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7faec76f2b80 10 client.26969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7faec76f2b80 20 client.26969 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7faec76f2b80 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56112759a5d0 0x56112759c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7faec76f2b80 4 client.26969 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561127477ad0 con 0x56112759a5d0 2023-10-06T22:24:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.531+0000 7faeb139a700 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56112759a5d0 0x56112759c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.532+0000 7fae957fa700 10 client.26969 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.532+0000 7faeb139a700 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56112759a5d0 0x56112759c9f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2bc660df2cc4b4 server_cookie=a068d093010a3539 in_seq=0 out_seq=0 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7fae957fa700 1 -- 172.21.15.57:0/2870428732 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561127477ad0 con 0x56112759a5d0 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7fae957fa700 10 client.26969 handle_command_reply: tid=0 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 client.26969 shutdown 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752ad60 msgr2=0x56112758deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752ad60 0x56112758deb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faea8011fd0 tx=0x7faea8096630 comp rx=0 tx=0).stop 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 shutdown_connections 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56112759a5d0 0x56112759c9f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561127536530 0x56112758e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56112752ad60 0x56112758deb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 --2- 172.21.15.57:0/2870428732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56112752b680 0x56112758e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7fae957fa700 0 client.26969 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 >> 172.21.15.57:0/2870428732 conn(0x5611274a0200 msgr2=0x5611275337c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 shutdown_connections 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.544+0000 7faec76f2b80 1 -- 172.21.15.57:0/2870428732 wait complete. 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faec76f2b80 20 client.26969 trim_cache size 0 max 16384 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:24:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:24:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fae980a1670 msgr2=0x7fae980a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fae980a1670 0x7fae980a3b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faea4009040 tx=0x7faea400bc30 comp rx=0 tx=0).stop 2023-10-06T22:24:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac100f20 msgr2=0x7faeac198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac100f20 0x7faeac198f00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faea800e9f0 tx=0x7faea800ed00 comp rx=0 tx=0).stop 2023-10-06T22:24:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 shutdown_connections 2023-10-06T22:24:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fae980a1670 0x7fae980a3b30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac19d8a0 0x7faeac19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faeac100f20 0x7faeac198f00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 --2- 172.21.15.57:0/3386466564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faeac106040 0x7faeac199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 >> 172.21.15.57:0/3386466564 conn(0x7faeac0fc7c0 msgr2=0x7faeac0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 shutdown_connections 2023-10-06T22:24:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:22.545+0000 7faeb2b9d700 1 -- 172.21.15.57:0/3386466564 wait complete. 2023-10-06T22:24:23.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:23 smithi190 ceph-mon[134891]: pgmap v299: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2023-10-06T22:24:23.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:23 smithi057 ceph-mon[98492]: pgmap v299: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2023-10-06T22:24:23.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:23 smithi156 ceph-mon[134358]: pgmap v299: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2023-10-06T22:24:25.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:25 smithi190 ceph-mon[134891]: pgmap v300: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-10-06T22:24:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:25 smithi057 ceph-mon[98492]: pgmap v300: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-10-06T22:24:25.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:25 smithi156 ceph-mon[134358]: pgmap v300: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-10-06T22:24:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:27 smithi057 ceph-mon[98492]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T22:24:27.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:27 smithi057 ceph-mon[98492]: pgmap v302: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2023-10-06T22:24:27.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:27 smithi156 ceph-mon[134358]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T22:24:27.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:27 smithi156 ceph-mon[134358]: pgmap v302: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2023-10-06T22:24:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:27 smithi190 ceph-mon[134891]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T22:24:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:27 smithi190 ceph-mon[134891]: pgmap v302: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2023-10-06T22:24:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:29 smithi190 ceph-mon[134891]: pgmap v303: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 252 op/s 2023-10-06T22:24:29.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:29 smithi057 ceph-mon[98492]: pgmap v303: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 252 op/s 2023-10-06T22:24:29.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:29 smithi156 ceph-mon[134358]: pgmap v303: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 252 op/s 2023-10-06T22:24:31.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:31 smithi190 ceph-mon[134891]: pgmap v304: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 260 op/s 2023-10-06T22:24:31.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:31 smithi057 ceph-mon[98492]: pgmap v304: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 260 op/s 2023-10-06T22:24:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:31 smithi156 ceph-mon[134358]: pgmap v304: 97 pgs: 97 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 260 op/s 2023-10-06T22:24:33.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:33 smithi190 ceph-mon[134891]: pgmap v305: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2023-10-06T22:24:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:33 smithi057 ceph-mon[98492]: pgmap v305: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2023-10-06T22:24:33.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:33 smithi156 ceph-mon[134358]: pgmap v305: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2023-10-06T22:24:34.440 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:24:34.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1828542573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c010d6f0 msgr2=0x7fb2c010fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1828542573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c010d6f0 0x7fb2c010fb20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009990 tx=0x7fb2b8009ca0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1828542573 shutdown_connections 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1828542573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c010d6f0 0x7fb2c010fb20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1828542573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c0073280 0x7fb2c0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1828542573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2c0073ba0 0x7fb2c010d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1828542573 >> 172.21.15.57:0/1828542573 conn(0x7fb2c00fc960 msgr2=0x7fb2c00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.668+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1828542573 shutdown_connections 2023-10-06T22:24:34.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.669+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1828542573 wait complete. 2023-10-06T22:24:34.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.670+0000 7fb2c69ff700 1 Processor -- start 2023-10-06T22:24:34.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.671+0000 7fb2c69ff700 1 -- start start 2023-10-06T22:24:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.671+0000 7fb2c69ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c0073280 0x7fb2c006b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.671+0000 7fb2c69ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2c0073ba0 0x7fb2c006b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.671+0000 7fb2c69ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c010d6f0 0x7fb2c01ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.671+0000 7fb2c69ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c01add00 con 0x7fb2c0073ba0 2023-10-06T22:24:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.671+0000 7fb2c69ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c01ade70 con 0x7fb2c010d6f0 2023-10-06T22:24:34.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.671+0000 7fb2c69ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c01ae030 con 0x7fb2c0073280 2023-10-06T22:24:34.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c010d6f0 0x7fb2c01ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c010d6f0 0x7fb2c01ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53780/0 (socket says 172.21.15.57:53780) 2023-10-06T22:24:34.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1042320129 learned_addr learned my addr 172.21.15.57:0/1042320129 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:34.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2b7fff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2c0073ba0 0x7fb2c006b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1042320129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c0073280 msgr2=0x7fb2c006b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c0073280 0x7fb2c006b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2c0073ba0 msgr2=0x7fb2c006b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2c0073ba0 0x7fb2c006b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.672+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1042320129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80095f0 con 0x7fb2c010d6f0 2023-10-06T22:24:34.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.673+0000 7fb2b7fff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2c0073ba0 0x7fb2c006b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:24:34.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.673+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c010d6f0 0x7fb2c01ad700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80098e0 tx=0x7fb2b800b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:34.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.673+0000 7fb2bdffb700 1 -- 172.21.15.57:0/1042320129 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b801d070 con 0x7fb2c010d6f0 2023-10-06T22:24:34.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.673+0000 7fb2bdffb700 1 -- 172.21.15.57:0/1042320129 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2b80047e0 con 0x7fb2c010d6f0 2023-10-06T22:24:34.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.673+0000 7fb2bdffb700 1 -- 172.21.15.57:0/1042320129 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80054d0 con 0x7fb2c010d6f0 2023-10-06T22:24:34.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.673+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c01ae2b0 con 0x7fb2c010d6f0 2023-10-06T22:24:34.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.674+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c01ae700 con 0x7fb2c010d6f0 2023-10-06T22:24:34.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.679+0000 7fb2bdffb700 1 -- 172.21.15.57:0/1042320129 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb2b8004950 con 0x7fb2c010d6f0 2023-10-06T22:24:34.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.681+0000 7fb2da3aab80 1 Processor -- start 2023-10-06T22:24:34.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.681+0000 7fb2da3aab80 1 -- start start 2023-10-06T22:24:34.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.681+0000 7fb2bdffb700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2ac0a1800 0x7fb2ac0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbebb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb9e0 0x55cfbbec6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbec6890 0x55cfbbec8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbe24460 con 0x55cfbbec6890 2023-10-06T22:24:34.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbe15c20 con 0x55cfbbebb0c0 2023-10-06T22:24:34.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbe13540 con 0x55cfbbebb9e0 2023-10-06T22:24:34.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2bdffb700 1 -- 172.21.15.57:0/1042320129 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fb2b80cdff0 con 0x7fb2c010d6f0 2023-10-06T22:24:34.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2b7fff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2ac0a1800 0x7fb2ac0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2b7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb9e0 0x55cfbbec6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2bffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbebb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2b7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb9e0 0x55cfbbec6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57846/0 (socket says 172.21.15.57:57846) 2023-10-06T22:24:34.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2b7fff700 1 -- 172.21.15.57:0/1914139820 learned_addr learned my addr 172.21.15.57:0/1914139820 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:34.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.682+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbec6890 0x55cfbbec8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.683+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1914139820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb9e0 msgr2=0x55cfbbec6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.683+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb9e0 0x55cfbbec6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.683+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1914139820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 msgr2=0x55cfbbebb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.683+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbebb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.683+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1914139820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbbc39de0 con 0x55cfbbec6890 2023-10-06T22:24:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.683+0000 7fb2b7fff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2ac0a1800 0x7fb2ac0a3cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80159b0 tx=0x7fb2a8009c00 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.683+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbec6890 0x55cfbbec8c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80cb700 tx=0x7fb2b809b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189966289b9e9c17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.684+0000 7fb2b67fc700 1 -- 172.21.15.57:0/1914139820 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b809f170 con 0x55cfbbec6890 2023-10-06T22:24:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.684+0000 7fb2b67fc700 1 -- 172.21.15.57:0/1914139820 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2b80a3320 con 0x55cfbbec6890 2023-10-06T22:24:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.684+0000 7fb2b67fc700 1 -- 172.21.15.57:0/1914139820 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b809b280 con 0x55cfbbec6890 2023-10-06T22:24:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1914139820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbec6890 msgr2=0x55cfbbec8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbec6890 0x55cfbbec8c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80cb700 tx=0x7fb2b809b7a0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1914139820 shutdown_connections 2023-10-06T22:24:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbec6890 0x55cfbbec8c80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbebb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1914139820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb9e0 0x55cfbbec6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1914139820 >> 172.21.15.57:0/1914139820 conn(0x55cfbbe30260 msgr2=0x55cfbbeb7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1914139820 shutdown_connections 2023-10-06T22:24:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.685+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1914139820 wait complete. 2023-10-06T22:24:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.686+0000 7fb2da3aab80 1 -- start start 2023-10-06T22:24:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.686+0000 7fb2da3aab80 1 Processor -- start 2023-10-06T22:24:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.687+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb0c0 0x55cfbbecf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.687+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb9e0 0x55cfbbecf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.687+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeda500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.687+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbe0ee30 con 0x55cfbbebb0c0 2023-10-06T22:24:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.687+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbdab710 con 0x55cfbbebb9e0 2023-10-06T22:24:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.687+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbdab3b0 con 0x55cfbbec6890 2023-10-06T22:24:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.687+0000 7fb2da3aab80 10 client.0 _ll_register_callbacks cb 0x55cfbbc08800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb0c0 0x55cfbbecf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2c4f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeda500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2bffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb0c0 0x55cfbbecf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43542/0 (socket says 172.21.15.57:43542) 2023-10-06T22:24:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2c4f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeda500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:57860/0 (socket says 172.21.15.57:57860) 2023-10-06T22:24:34.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2bffff700 1 -- 172.21.15.57:0/3423850189 learned_addr learned my addr 172.21.15.57:0/3423850189 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:34.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2b7fff700 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb9e0 0x55cfbbecf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/3423850189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb9e0 msgr2=0x55cfbbecf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb9e0 0x55cfbbecf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/3423850189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb0c0 msgr2=0x55cfbbecf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb0c0 0x55cfbbecf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.688+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/3423850189 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c01ae700 con 0x55cfbbec6890 2023-10-06T22:24:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.689+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeda500 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55cfbbebd0f0 tx=0x7fb2b809b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2f99979f5dff3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.689+0000 7fb29f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:24:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.689+0000 7fb29f7fe700 1 -- 172.21.15.57:0/3423850189 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80cbb90 con 0x55cfbbec6890 2023-10-06T22:24:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.689+0000 7fb29f7fe700 1 -- 172.21.15.57:0/3423850189 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fb2b809a510 con 0x55cfbbec6890 2023-10-06T22:24:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.689+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfbbeaef10 con 0x55cfbbec6890 2023-10-06T22:24:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.689+0000 7fb29f7fe700 1 -- 172.21.15.57:0/3423850189 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2b80b9250 con 0x55cfbbec6890 2023-10-06T22:24:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.689+0000 7fb2da3aab80 10 client.26995 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55cfbbc22c60 con 0x55cfbbec6890 2023-10-06T22:24:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb29f7fe700 1 -- 172.21.15.57:0/3423850189 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cfbbeaef10 con 0x55cfbbec6890 2023-10-06T22:24:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb29f7fe700 1 -- 172.21.15.57:0/3423850189 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fb2b80a2e90 con 0x55cfbbec6890 2023-10-06T22:24:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb2da3aab80 10 client.26995 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb2da3aab80 10 client.26995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb2da3aab80 20 client.26995 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbede370 0x55cfbbee0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb2da3aab80 4 client.26995 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:24:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.690+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfbbe07e10 con 0x55cfbbede370 2023-10-06T22:24:34.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.691+0000 7fb2bffff700 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbede370 0x55cfbbee0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.692+0000 7fb2bffff700 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbede370 0x55cfbbee0790 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d55c49f5e8fcef server_cookie=9a240ee3820d21ea in_seq=0 out_seq=0 2023-10-06T22:24:34.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.692+0000 7fb29f7fe700 10 client.26995 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:34.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.692+0000 7fb29f7fe700 1 -- 172.21.15.57:0/3423850189 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cfbbe07e10 con 0x55cfbbede370 2023-10-06T22:24:34.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.692+0000 7fb29f7fe700 10 client.26995 handle_command_reply: tid=0 2023-10-06T22:24:34.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.692+0000 7fb2da3aab80 1 client.26995 shutdown 2023-10-06T22:24:34.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 msgr2=0x55cfbbeda500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeda500 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55cfbbebd0f0 tx=0x7fb2b809b890 comp rx=0 tx=0).stop 2023-10-06T22:24:34.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 shutdown_connections 2023-10-06T22:24:34.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbede370 0x55cfbbee0790 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeda500 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb0c0 0x55cfbbecf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/3423850189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb9e0 0x55cfbbecf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 >> 172.21.15.57:0/3423850189 conn(0x55cfbbe30260 msgr2=0x55cfbbeb6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:34.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb29f7fe700 0 client.26995 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:34.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 shutdown_connections 2023-10-06T22:24:34.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 1 -- 172.21.15.57:0/3423850189 wait complete. 2023-10-06T22:24:34.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.693+0000 7fb2da3aab80 20 client.26995 trim_cache size 0 max 16384 2023-10-06T22:24:34.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 Processor -- start 2023-10-06T22:24:34.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 -- start start 2023-10-06T22:24:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbec4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeb6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbc8f920 con 0x55cfbbebb9e0 2023-10-06T22:24:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbeb8880 con 0x55cfbbebb0c0 2023-10-06T22:24:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.718+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbed0040 con 0x55cfbbec6890 2023-10-06T22:24:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbec4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbec4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53822/0 (socket says 172.21.15.57:53822) 2023-10-06T22:24:34.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2b7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2c4f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeb6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 -- 172.21.15.57:0/2672598293 learned_addr learned my addr 172.21.15.57:0/2672598293 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 -- 172.21.15.57:0/2672598293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 msgr2=0x55cfbbeb6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 --2- 172.21.15.57:0/2672598293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeb6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 -- 172.21.15.57:0/2672598293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 msgr2=0x55cfbbed4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 --2- 172.21.15.57:0/2672598293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.719+0000 7fb2bffff700 1 -- 172.21.15.57:0/2672598293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbbec55b0 con 0x55cfbbebb0c0 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.720+0000 7fb2bffff700 1 --2- 172.21.15.57:0/2672598293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbec4cd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00092c0 tx=0x7fb2b0009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=268a72587c8901ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.720+0000 7fb2b6ffd700 1 -- 172.21.15.57:0/2672598293 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0010040 con 0x55cfbbebb0c0 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.720+0000 7fb2b6ffd700 1 -- 172.21.15.57:0/2672598293 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2b001b0f0 con 0x55cfbbebb0c0 2023-10-06T22:24:34.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.720+0000 7fb2b6ffd700 1 -- 172.21.15.57:0/2672598293 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0019900 con 0x55cfbbebb0c0 2023-10-06T22:24:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 -- 172.21.15.57:0/2672598293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 msgr2=0x55cfbbec4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/2672598293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbec4cd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00092c0 tx=0x7fb2b0009650 comp rx=0 tx=0).stop 2023-10-06T22:24:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 -- 172.21.15.57:0/2672598293 shutdown_connections 2023-10-06T22:24:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/2672598293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbec6890 0x55cfbbeb6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/2672598293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbebb0c0 0x55cfbbec4cd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/2672598293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 -- 172.21.15.57:0/2672598293 >> 172.21.15.57:0/2672598293 conn(0x55cfbbe30260 msgr2=0x55cfbbeb6660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 -- 172.21.15.57:0/2672598293 shutdown_connections 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.721+0000 7fb2da3aab80 1 -- 172.21.15.57:0/2672598293 wait complete. 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 -- start start 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 Processor -- start 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb0c0 0x55cfbbed42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbec6890 0x55cfbbed2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbed3180 con 0x55cfbbebb9e0 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbed32f0 con 0x55cfbbec6890 2023-10-06T22:24:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbbed34b0 con 0x55cfbbebb0c0 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.722+0000 7fb2da3aab80 10 client.0 _ll_register_callbacks cb 0x7fb2b00094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbec6890 0x55cfbbed2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2b7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbec6890 0x55cfbbed2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53838/0 (socket says 172.21.15.57:53838) 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1555009227 learned_addr learned my addr 172.21.15.57:0/1555009227 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2bffff700 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb0c0 0x55cfbbed42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb0c0 msgr2=0x55cfbbed42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb0c0 0x55cfbbed42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1555009227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 msgr2=0x55cfbbed2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 -- 172.21.15.57:0/1555009227 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b0009da0 con 0x55cfbbec6890 2023-10-06T22:24:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2b7fff700 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb2c4f9c700 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbec6890 0x55cfbbed2c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005f50 tx=0x7fb2b8097430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9c0df73e45885f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.723+0000 7fb29f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:24:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb29f7fe700 1 -- 172.21.15.57:0/1555009227 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b809a710 con 0x55cfbbec6890 2023-10-06T22:24:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2b80a3320 con 0x55cfbbec6890 2023-10-06T22:24:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb29f7fe700 1 -- 172.21.15.57:0/1555009227 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fb2b80b9250 con 0x55cfbbec6890 2023-10-06T22:24:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb29f7fe700 1 -- 172.21.15.57:0/1555009227 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2b809b720 con 0x55cfbbec6890 2023-10-06T22:24:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 10 client.26984 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55cfbbc60250 con 0x55cfbbec6890 2023-10-06T22:24:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb29f7fe700 1 -- 172.21.15.57:0/1555009227 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2b80a3320 con 0x55cfbbec6890 2023-10-06T22:24:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb29f7fe700 1 -- 172.21.15.57:0/1555009227 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fb2b8099f80 con 0x55cfbbec6890 2023-10-06T22:24:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 10 client.26984 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 20 client.26984 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbf33150 0x55cfbbf35540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 4 client.26984 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:24:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.724+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cfbbe07b30 con 0x55cfbbf33150 2023-10-06T22:24:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.725+0000 7fb2bffff700 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbf33150 0x55cfbbf35540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.725+0000 7fb2bffff700 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbf33150 0x55cfbbf35540 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212e64cc54ffb64 server_cookie=1e2d10af489101d6 in_seq=0 out_seq=0 2023-10-06T22:24:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.725+0000 7fb29f7fe700 10 client.26984 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.725+0000 7fb29f7fe700 1 -- 172.21.15.57:0/1555009227 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55cfbbe07b30 con 0x55cfbbf33150 2023-10-06T22:24:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.725+0000 7fb29f7fe700 10 client.26984 handle_command_reply: tid=0 2023-10-06T22:24:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.725+0000 7fb2da3aab80 1 client.26984 shutdown 2023-10-06T22:24:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbec6890 msgr2=0x55cfbbed2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbec6890 0x55cfbbed2c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005f50 tx=0x7fb2b8097430 comp rx=0 tx=0).stop 2023-10-06T22:24:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 shutdown_connections 2023-10-06T22:24:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55cfbbf33150 0x55cfbbf35540 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cfbbec6890 0x55cfbbed2c40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfbbebb0c0 0x55cfbbed42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 --2- 172.21.15.57:0/1555009227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cfbbebb9e0 0x55cfbbed2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 >> 172.21.15.57:0/1555009227 conn(0x55cfbbe30260 msgr2=0x55cfbbeb6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb29f7fe700 0 client.26984 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 shutdown_connections 2023-10-06T22:24:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.726+0000 7fb2da3aab80 1 -- 172.21.15.57:0/1555009227 wait complete. 2023-10-06T22:24:34.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.727+0000 7fb2da3aab80 20 client.26984 trim_cache size 0 max 16384 2023-10-06T22:24:34.737 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:24:34.737 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T22:24:34.737 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-06T22:24:34.737 INFO:teuthology.orchestra.run.smithi057.stdout: "703a333e-9f6b-48d1-9e64-6d898aa4521a": { 2023-10-06T22:24:34.738 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-06T22:24:34.738 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "703a333e-9f6b-48d1-9e64-6d898aa4521a", 2023-10-06T22:24:34.738 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-06T22:24:34.738 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:24:34.738 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:24:34.738 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.727+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2ac0a1800 msgr2=0x7fb2ac0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.727+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2ac0a1800 0x7fb2ac0a3cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80159b0 tx=0x7fb2a8009c00 comp rx=0 tx=0).stop 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.727+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c010d6f0 msgr2=0x7fb2c01ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.727+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c010d6f0 0x7fb2c01ad700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80098e0 tx=0x7fb2b800b680 comp rx=0 tx=0).stop 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 shutdown_connections 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb2ac0a1800 0x7fb2ac0a3cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2c010d6f0 0x7fb2c01ad700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c0073280 0x7fb2c006b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 --2- 172.21.15.57:0/1042320129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2c0073ba0 0x7fb2c006b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 >> 172.21.15.57:0/1042320129 conn(0x7fb2c00fc960 msgr2=0x7fb2c010f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 shutdown_connections 2023-10-06T22:24:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:24:34.728+0000 7fb2c69ff700 1 -- 172.21.15.57:0/1042320129 wait complete. 2023-10-06T22:24:34.757 INFO:tasks.cephfs.filesystem:scrub status for tag:703a333e-9f6b-48d1-9e64-6d898aa4521a - {'path': '/', 'tag': '703a333e-9f6b-48d1-9e64-6d898aa4521a', 'options': 'recursive,force'} 2023-10-06T22:24:35.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:35 smithi190 ceph-mon[134891]: pgmap v306: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2023-10-06T22:24:35.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:35 smithi057 ceph-mon[98492]: pgmap v306: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.724+0000 7fe833fff700 1 -- 172.21.15.57:0/3638397249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c10cd20 msgr2=0x7fe82c10d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.724+0000 7fe833fff700 1 --2- 172.21.15.57:0/3638397249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c10cd20 0x7fe82c10d100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00aad0 tx=0x7fe81c00ade0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.724+0000 7fe833fff700 1 -- 172.21.15.57:0/3638397249 shutdown_connections 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.724+0000 7fe833fff700 1 --2- 172.21.15.57:0/3638397249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c109100 0x7fe82c114bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.724+0000 7fe833fff700 1 --2- 172.21.15.57:0/3638397249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c10cd20 0x7fe82c10d100 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.724+0000 7fe833fff700 1 --2- 172.21.15.57:0/3638397249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe82c108740 0x7fe82c108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.724+0000 7fe833fff700 1 -- 172.21.15.57:0/3638397249 >> 172.21.15.57:0/3638397249 conn(0x7fe82c06d3e0 msgr2=0x7fe82c06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.725+0000 7fe833fff700 1 -- 172.21.15.57:0/3638397249 shutdown_connections 2023-10-06T22:24:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.725+0000 7fe833fff700 1 -- 172.21.15.57:0/3638397249 wait complete. 2023-10-06T22:24:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.726+0000 7fe833fff700 1 Processor -- start 2023-10-06T22:24:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.726+0000 7fe833fff700 1 -- start start 2023-10-06T22:24:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe833fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe82c108740 0x7fe82c10ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe833fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c109100 0x7fe82c10f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe833fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c10cd20 0x7fe82c1ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe833fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1aded0 con 0x7fe82c108740 2023-10-06T22:24:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe833fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1ae010 con 0x7fe82c10cd20 2023-10-06T22:24:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe833fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1ae180 con 0x7fe82c109100 2023-10-06T22:24:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe83259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c10cd20 0x7fe82c1ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe83259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c10cd20 0x7fe82c1ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53860/0 (socket says 172.21.15.57:53860) 2023-10-06T22:24:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.727+0000 7fe83259c700 1 -- 172.21.15.57:0/580008933 learned_addr learned my addr 172.21.15.57:0/580008933 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe831d9b700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe82c108740 0x7fe82c10ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe83159a700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c109100 0x7fe82c10f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe83259c700 1 -- 172.21.15.57:0/580008933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c109100 msgr2=0x7fe82c10f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe83259c700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c109100 0x7fe82c10f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe83259c700 1 -- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe82c108740 msgr2=0x7fe82c10ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe83259c700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe82c108740 0x7fe82c10ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe83259c700 1 -- 172.21.15.57:0/580008933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c00a700 con 0x7fe82c10cd20 2023-10-06T22:24:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.728+0000 7fe83159a700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c109100 0x7fe82c10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:24:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.729+0000 7fe83259c700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c10cd20 0x7fe82c1ad990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe82400b970 tx=0x7fe82400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.729+0000 7fe822ffd700 1 -- 172.21.15.57:0/580008933 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400c930 con 0x7fe82c10cd20 2023-10-06T22:24:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.729+0000 7fe822ffd700 1 -- 172.21.15.57:0/580008933 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe824014440 con 0x7fe82c10cd20 2023-10-06T22:24:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.729+0000 7fe822ffd700 1 -- 172.21.15.57:0/580008933 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240075d0 con 0x7fe82c10cd20 2023-10-06T22:24:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.731+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c1ae460 con 0x7fe82c10cd20 2023-10-06T22:24:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.731+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c1ae8f0 con 0x7fe82c10cd20 2023-10-06T22:24:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.737+0000 7fe822ffd700 1 -- 172.21.15.57:0/580008933 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe8240145b0 con 0x7fe82c10cd20 2023-10-06T22:24:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.738+0000 7fe8479c1b80 1 Processor -- start 2023-10-06T22:24:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.738+0000 7fe8479c1b80 1 -- start start 2023-10-06T22:24:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.739+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.739+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.739+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.739+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce703400 con 0x55d7ce7a2410 2023-10-06T22:24:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.739+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce6f4bc0 con 0x55d7ce7a2d30 2023-10-06T22:24:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.739+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce6f24e0 con 0x55d7ce7adbe0 2023-10-06T22:24:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.740+0000 7fe822ffd700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe8180a1670 0x7fe8180a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.740+0000 7fe83159a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.740+0000 7fe83159a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.740+0000 7fe83159a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43588/0 (socket says 172.21.15.57:43588) 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.740+0000 7fe83159a700 1 -- 172.21.15.57:0/3773779401 learned_addr learned my addr 172.21.15.57:0/3773779401 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.740+0000 7fe822ffd700 1 -- 172.21.15.57:0/580008933 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fe824006e40 con 0x7fe82c10cd20 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.740+0000 7fe831d9b700 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.741+0000 7fe831d9b700 1 -- 172.21.15.57:0/3773779401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 msgr2=0x55d7ce7affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.741+0000 7fe831d9b700 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.741+0000 7fe831d9b700 1 -- 172.21.15.57:0/3773779401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2410 msgr2=0x55d7ce7a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.741+0000 7fe831d9b700 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.741+0000 7fe831d9b700 1 -- 172.21.15.57:0/3773779401 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7ce518d80 con 0x55d7ce7a2d30 2023-10-06T22:24:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.741+0000 7fe831d9b700 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7ad6a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00aaa0 tx=0x7fe81c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=294d64ec8b04df87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.741+0000 7fe83259c700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe8180a1670 0x7fe8180a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.742+0000 7fe817fff700 1 -- 172.21.15.57:0/3773779401 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c00ccb0 con 0x55d7ce7a2d30 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.742+0000 7fe817fff700 1 -- 172.21.15.57:0/3773779401 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe81c005070 con 0x55d7ce7a2d30 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.742+0000 7fe817fff700 1 -- 172.21.15.57:0/3773779401 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c021810 con 0x55d7ce7a2d30 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3773779401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2d30 msgr2=0x55d7ce7ad6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe83259c700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe8180a1670 0x7fe8180a3b30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8240c9d30 tx=0x7fe82400c3a0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7ad6a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00aaa0 tx=0x7fe81c005c90 comp rx=0 tx=0).stop 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3773779401 shutdown_connections 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3773779401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7ad6a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3773779401 >> 172.21.15.57:0/3773779401 conn(0x55d7ce70f200 msgr2=0x55d7ce795ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3773779401 shutdown_connections 2023-10-06T22:24:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.743+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3773779401 wait complete. 2023-10-06T22:24:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.744+0000 7fe8479c1b80 1 -- start start 2023-10-06T22:24:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.744+0000 7fe8479c1b80 1 Processor -- start 2023-10-06T22:24:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.745+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.745+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.745+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.745+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce6eddd0 con 0x55d7ce7a2d30 2023-10-06T22:24:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.745+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce68a6b0 con 0x55d7ce7a2410 2023-10-06T22:24:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.745+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce68a350 con 0x55d7ce7adbe0 2023-10-06T22:24:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.745+0000 7fe8479c1b80 10 client.0 _ll_register_callbacks cb 0x55d7ce4e77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.747+0000 7fe831d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.747+0000 7fe831d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7aa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43592/0 (socket says 172.21.15.57:43592) 2023-10-06T22:24:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.747+0000 7fe831d9b700 1 -- 172.21.15.57:0/4072374702 learned_addr learned my addr 172.21.15.57:0/4072374702 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.748+0000 7fe83159a700 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.749+0000 7fe831d9b700 1 -- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 msgr2=0x55d7ce7ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.749+0000 7fe831d9b700 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.749+0000 7fe831d9b700 1 -- 172.21.15.57:0/4072374702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 msgr2=0x55d7ce7ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.749+0000 7fe831d9b700 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.749+0000 7fe831d9b700 1 -- 172.21.15.57:0/4072374702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c01a040 con 0x55d7ce7a2d30 2023-10-06T22:24:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.749+0000 7fe831d9b700 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7aa900 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00afd0 tx=0x7fe81c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d053cd871a6af17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.750+0000 7fe8157fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:24:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.750+0000 7fe8157fa700 1 -- 172.21.15.57:0/4072374702 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c0053f0 con 0x55d7ce7a2d30 2023-10-06T22:24:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.750+0000 7fe8157fa700 1 -- 172.21.15.57:0/4072374702 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fe81c012070 con 0x55d7ce7a2d30 2023-10-06T22:24:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.750+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7ce78dd50 con 0x55d7ce7a2d30 2023-10-06T22:24:35.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:35 smithi156 ceph-mon[134358]: pgmap v306: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2023-10-06T22:24:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.752+0000 7fe8157fa700 1 -- 172.21.15.57:0/4072374702 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe81c005550 con 0x55d7ce7a2d30 2023-10-06T22:24:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.752+0000 7fe8479c1b80 10 client.16968 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.752+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d7ce501c00 con 0x55d7ce7a2d30 2023-10-06T22:24:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.753+0000 7fe8157fa700 1 -- 172.21.15.57:0/4072374702 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7ce78dd50 con 0x55d7ce7a2d30 2023-10-06T22:24:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.753+0000 7fe8157fa700 1 -- 172.21.15.57:0/4072374702 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fe81c00e450 con 0x55d7ce7a2d30 2023-10-06T22:24:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.754+0000 7fe8479c1b80 10 client.16968 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.754+0000 7fe8479c1b80 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.754+0000 7fe8479c1b80 20 client.16968 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.754+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce7bd2d0 0x55d7ce7bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.754+0000 7fe8479c1b80 4 client.16968 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:24:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.754+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7ce6e6db0 con 0x55d7ce7bd2d0 2023-10-06T22:24:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.754+0000 7fe83159a700 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce7bd2d0 0x55d7ce7bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.755+0000 7fe83159a700 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce7bd2d0 0x55d7ce7bf6f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daeeef3c8d808e0c server_cookie=60067ede94ce328f in_seq=0 out_seq=0 2023-10-06T22:24:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.755+0000 7fe8157fa700 10 client.16968 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.756+0000 7fe8157fa700 1 -- 172.21.15.57:0/4072374702 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d7ce6e6db0 con 0x55d7ce7bd2d0 2023-10-06T22:24:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.756+0000 7fe8157fa700 10 client.16968 handle_command_reply: tid=0 2023-10-06T22:24:35.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.756+0000 7fe8479c1b80 1 client.16968 shutdown 2023-10-06T22:24:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2d30 msgr2=0x55d7ce7aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7aa900 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00afd0 tx=0x7fe81c00c760 comp rx=0 tx=0).stop 2023-10-06T22:24:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 shutdown_connections 2023-10-06T22:24:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce7bd2d0 0x55d7ce7bf6f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8157fa700 0 client.16968 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/4072374702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7a2d30 0x55d7ce7aa900 secure :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00afd0 tx=0x7fe81c00c760 comp rx=0 tx=0).stop 2023-10-06T22:24:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 >> 172.21.15.57:0/4072374702 conn(0x55d7ce70f200 msgr2=0x55d7ce795b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 shutdown_connections 2023-10-06T22:24:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.757+0000 7fe8479c1b80 1 -- 172.21.15.57:0/4072374702 wait complete. 2023-10-06T22:24:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.758+0000 7fe8479c1b80 20 client.16968 trim_cache size 0 max 16384 2023-10-06T22:24:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.781+0000 7fe8479c1b80 1 Processor -- start 2023-10-06T22:24:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.781+0000 7fe8479c1b80 1 -- start start 2023-10-06T22:24:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7ad7f0 0x55d7ce7a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce56e8c0 con 0x55d7ce7ad7f0 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce78d890 con 0x55d7ce7adbe0 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce7a8e60 con 0x55d7ce7a2410 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe831d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe831d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53894/0 (socket says 172.21.15.57:53894) 2023-10-06T22:24:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.782+0000 7fe831d9b700 1 -- 172.21.15.57:0/3932700269 learned_addr learned my addr 172.21.15.57:0/3932700269 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.783+0000 7fe83159a700 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.784+0000 7fe831d9b700 1 -- 172.21.15.57:0/3932700269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7a2410 msgr2=0x55d7ce7a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.784+0000 7fe831d9b700 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.784+0000 7fe831d9b700 1 -- 172.21.15.57:0/3932700269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7ad7f0 msgr2=0x55d7ce7a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.784+0000 7fe831d9b700 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7ad7f0 0x55d7ce7a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.784+0000 7fe831d9b700 1 -- 172.21.15.57:0/3932700269 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7ce53f1f0 con 0x55d7ce7adbe0 2023-10-06T22:24:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.784+0000 7fe831d9b700 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ad2b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00a9f0 tx=0x7fe81c00c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd82b511eaa5c0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.785+0000 7fe820ff9700 1 -- 172.21.15.57:0/3932700269 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c00ccb0 con 0x55d7ce7adbe0 2023-10-06T22:24:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.785+0000 7fe820ff9700 1 -- 172.21.15.57:0/3932700269 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe81c00e450 con 0x55d7ce7adbe0 2023-10-06T22:24:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.785+0000 7fe820ff9700 1 -- 172.21.15.57:0/3932700269 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c021c70 con 0x55d7ce7adbe0 2023-10-06T22:24:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3932700269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7adbe0 msgr2=0x55d7ce7ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ad2b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00a9f0 tx=0x7fe81c00c650 comp rx=0 tx=0).stop 2023-10-06T22:24:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3932700269 shutdown_connections 2023-10-06T22:24:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7ad7f0 0x55d7ce7a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce7a2410 0x55d7ce7a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/3932700269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7ad2b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3932700269 >> 172.21.15.57:0/3932700269 conn(0x55d7ce70f200 msgr2=0x55d7ce7b9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3932700269 shutdown_connections 2023-10-06T22:24:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.786+0000 7fe8479c1b80 1 -- 172.21.15.57:0/3932700269 wait complete. 2023-10-06T22:24:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.787+0000 7fe8479c1b80 1 -- start start 2023-10-06T22:24:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.787+0000 7fe8479c1b80 1 Processor -- start 2023-10-06T22:24:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe8479c1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce80e0f0 0x55d7ce8104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce7b2f40 con 0x55d7ce7adbe0 2023-10-06T22:24:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce810a20 con 0x55d7ce7a2410 2023-10-06T22:24:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe8479c1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ce810b90 con 0x55d7ce80e0f0 2023-10-06T22:24:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe8479c1b80 10 client.0 _ll_register_callbacks cb 0x7fe828009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.788+0000 7fe83159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.789+0000 7fe83159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:53900/0 (socket says 172.21.15.57:53900) 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.789+0000 7fe83159a700 1 -- 172.21.15.57:0/1261481036 learned_addr learned my addr 172.21.15.57:0/1261481036 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.789+0000 7fe83159a700 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce80e0f0 0x55d7ce8104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.789+0000 7fe831d9b700 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.790+0000 7fe83159a700 1 -- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce80e0f0 msgr2=0x55d7ce8104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.790+0000 7fe83159a700 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce80e0f0 0x55d7ce8104e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.790+0000 7fe83159a700 1 -- 172.21.15.57:0/1261481036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7adbe0 msgr2=0x55d7ce7b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.790+0000 7fe83159a700 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.790+0000 7fe83159a700 1 -- 172.21.15.57:0/1261481036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7ce501c00 con 0x55d7ce7a2410 2023-10-06T22:24:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.792+0000 7fe83159a700 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7b22f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d7ce7bffa0 tx=0x7fe828012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=890410882fa4b766 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.792+0000 7fe8157fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:24:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.792+0000 7fe8157fa700 1 -- 172.21.15.57:0/1261481036 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828010950 con 0x55d7ce7a2410 2023-10-06T22:24:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.792+0000 7fe8157fa700 1 -- 172.21.15.57:0/1261481036 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fe828010ab0 con 0x55d7ce7a2410 2023-10-06T22:24:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.792+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe81c005070 con 0x55d7ce7a2410 2023-10-06T22:24:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.793+0000 7fe8157fa700 1 -- 172.21.15.57:0/1261481036 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe828010c70 con 0x55d7ce7a2410 2023-10-06T22:24:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.793+0000 7fe8479c1b80 10 client.26996 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.793+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d7ce517250 con 0x55d7ce7a2410 2023-10-06T22:24:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.793+0000 7fe8157fa700 1 -- 172.21.15.57:0/1261481036 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe81c005070 con 0x55d7ce7a2410 2023-10-06T22:24:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.794+0000 7fe8157fa700 1 -- 172.21.15.57:0/1261481036 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fe82802b3c0 con 0x55d7ce7a2410 2023-10-06T22:24:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.794+0000 7fe8479c1b80 10 client.26996 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.794+0000 7fe8479c1b80 10 client.26996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.794+0000 7fe8479c1b80 20 client.26996 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.794+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce8123d0 0x55d7ce8147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.794+0000 7fe8479c1b80 4 client.26996 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:24:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.794+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7ce6e6ad0 con 0x55d7ce8123d0 2023-10-06T22:24:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.795+0000 7fe831d9b700 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce8123d0 0x55d7ce8147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.795+0000 7fe831d9b700 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce8123d0 0x55d7ce8147f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe1468a7e39b313 server_cookie=1e2662d65bacb615 in_seq=0 out_seq=0 2023-10-06T22:24:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.795+0000 7fe8157fa700 10 client.26996 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.814+0000 7fe8157fa700 1 -- 172.21.15.57:0/1261481036 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7ce6e6ad0 con 0x55d7ce8123d0 2023-10-06T22:24:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.814+0000 7fe8157fa700 10 client.26996 handle_command_reply: tid=0 2023-10-06T22:24:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.814+0000 7fe8479c1b80 1 client.26996 shutdown 2023-10-06T22:24:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 msgr2=0x55d7ce7b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7b22f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d7ce7bffa0 tx=0x7fe828012040 comp rx=0 tx=0).stop 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 shutdown_connections 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55d7ce8123d0 0x55d7ce8147f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7ce80e0f0 0x55d7ce8104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d7ce7a2410 0x55d7ce7b22f0 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d7ce7bffa0 tx=0x7fe828012040 comp rx=0 tx=0).stop 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8157fa700 0 client.26996 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 --2- 172.21.15.57:0/1261481036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d7ce7adbe0 0x55d7ce7b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 >> 172.21.15.57:0/1261481036 conn(0x55d7ce70f200 msgr2=0x55d7ce7a5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 shutdown_connections 2023-10-06T22:24:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.815+0000 7fe8479c1b80 1 -- 172.21.15.57:0/1261481036 wait complete. 2023-10-06T22:24:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.817+0000 7fe8479c1b80 20 client.26996 trim_cache size 0 max 16384 2023-10-06T22:24:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:24:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:24:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:24:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:24:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.817+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe8180a1670 msgr2=0x7fe8180a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.817+0000 7fe833fff700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe8180a1670 0x7fe8180a3b30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8240c9d30 tx=0x7fe82400c3a0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.817+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c10cd20 msgr2=0x7fe82c1ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.817+0000 7fe833fff700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c10cd20 0x7fe82c1ad990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe82400b970 tx=0x7fe82400bc80 comp rx=0 tx=0).stop 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 shutdown_connections 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fe8180a1670 0x7fe8180a3b30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe82c10cd20 0x7fe82c1ad990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe82c108740 0x7fe82c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 --2- 172.21.15.57:0/580008933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe82c109100 0x7fe82c10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 >> 172.21.15.57:0/580008933 conn(0x7fe82c06d3e0 msgr2=0x7fe82c10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 shutdown_connections 2023-10-06T22:24:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:35.818+0000 7fe833fff700 1 -- 172.21.15.57:0/580008933 wait complete. 2023-10-06T22:24:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:37 smithi057 ceph-mon[98492]: pgmap v307: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2023-10-06T22:24:37.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:37 smithi156 ceph-mon[134358]: pgmap v307: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2023-10-06T22:24:37.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:37 smithi190 ceph-mon[134891]: pgmap v307: 97 pgs: 97 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2023-10-06T22:24:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:39 smithi190 ceph-mon[134891]: pgmap v308: 97 pgs: 97 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2023-10-06T22:24:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:39 smithi057 ceph-mon[98492]: pgmap v308: 97 pgs: 97 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2023-10-06T22:24:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:39 smithi156 ceph-mon[134358]: pgmap v308: 97 pgs: 97 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2023-10-06T22:24:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:41 smithi190 ceph-mon[134891]: pgmap v309: 97 pgs: 97 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-10-06T22:24:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:41 smithi057 ceph-mon[98492]: pgmap v309: 97 pgs: 97 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-10-06T22:24:41.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:41 smithi156 ceph-mon[134358]: pgmap v309: 97 pgs: 97 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-10-06T22:24:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:43 smithi190 ceph-mon[134891]: pgmap v310: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 377 op/s 2023-10-06T22:24:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:43 smithi057 ceph-mon[98492]: pgmap v310: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 377 op/s 2023-10-06T22:24:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:43 smithi156 ceph-mon[134358]: pgmap v310: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 377 op/s 2023-10-06T22:24:45.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:45 smithi190 ceph-mon[134891]: pgmap v311: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2023-10-06T22:24:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:45 smithi057 ceph-mon[98492]: pgmap v311: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2023-10-06T22:24:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:45 smithi156 ceph-mon[134358]: pgmap v311: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2023-10-06T22:24:46.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:46 smithi190 ceph-mon[134891]: scrub summary: idle+waiting paths [/] 2023-10-06T22:24:46.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:46 smithi057 ceph-mon[98492]: scrub summary: idle+waiting paths [/] 2023-10-06T22:24:46.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:46 smithi156 ceph-mon[134358]: scrub summary: idle+waiting paths [/] 2023-10-06T22:24:47.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:47 smithi190 ceph-mon[134891]: pgmap v312: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-10-06T22:24:47.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:47 smithi057 ceph-mon[98492]: pgmap v312: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-10-06T22:24:47.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:47 smithi156 ceph-mon[134358]: pgmap v312: 97 pgs: 97 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-10-06T22:24:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 -- 172.21.15.57:0/1222965002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281010d0 msgr2=0x7efc281014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/1222965002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281010d0 0x7efc281014b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efc180098f0 tx=0x7efc18009c00 comp rx=0 tx=0).stop 2023-10-06T22:24:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 -- 172.21.15.57:0/1222965002 shutdown_connections 2023-10-06T22:24:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/1222965002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc281061f0 0x7efc2810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/1222965002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281010d0 0x7efc281014b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/1222965002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc28101a80 0x7efc28105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 -- 172.21.15.57:0/1222965002 >> 172.21.15.57:0/1222965002 conn(0x7efc280fc930 msgr2=0x7efc280fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 -- 172.21.15.57:0/1222965002 shutdown_connections 2023-10-06T22:24:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.991+0000 7efc2d1ff700 1 -- 172.21.15.57:0/1222965002 wait complete. 2023-10-06T22:24:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.993+0000 7efc2d1ff700 1 Processor -- start 2023-10-06T22:24:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.993+0000 7efc2d1ff700 1 -- start start 2023-10-06T22:24:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.993+0000 7efc2d1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc281010d0 0x7efc28197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc2d1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281061f0 0x7efc28197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc2d1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc2819bbb0 0x7efc2819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc2d1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819c630 con 0x7efc281061f0 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc2d1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819c7a0 con 0x7efc2819bbb0 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc2d1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819c960 con 0x7efc281010d0 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc281010d0 0x7efc28197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc2759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc2819bbb0 0x7efc2819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc281010d0 0x7efc28197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33878/0 (socket says 172.21.15.57:33878) 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc1ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281061f0 0x7efc28197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 -- 172.21.15.57:0/3417576011 learned_addr learned my addr 172.21.15.57:0/3417576011 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 -- 172.21.15.57:0/3417576011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc2819bbb0 msgr2=0x7efc2819c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc2819bbb0 0x7efc2819c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 -- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281061f0 msgr2=0x7efc28197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281061f0 0x7efc28197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc26d9d700 1 -- 172.21.15.57:0/3417576011 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18009520 con 0x7efc281010d0 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc2759e700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc2819bbb0 0x7efc2819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.994+0000 7efc1ffff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281061f0 0x7efc28197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.995+0000 7efc26d9d700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc281010d0 0x7efc28197000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc18009810 tx=0x7efc1800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.995+0000 7efc24d99700 1 -- 172.21.15.57:0/3417576011 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1801c070 con 0x7efc281010d0 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.995+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc281a3e60 con 0x7efc281010d0 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.995+0000 7efc24d99700 1 -- 172.21.15.57:0/3417576011 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc1800ba40 con 0x7efc281010d0 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.995+0000 7efc24d99700 1 -- 172.21.15.57:0/3417576011 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18004930 con 0x7efc281010d0 2023-10-06T22:24:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.995+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc281a4280 con 0x7efc281010d0 2023-10-06T22:24:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.998+0000 7efc24d99700 1 -- 172.21.15.57:0/3417576011 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efc18020070 con 0x7efc281010d0 2023-10-06T22:24:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:48.999+0000 7efc40ab3b80 1 Processor -- start 2023-10-06T22:24:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc40ab3b80 1 -- start start 2023-10-06T22:24:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e44fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e45520 0x55aaf8e542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e56bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8dae400 con 0x55aaf8e547e0 2023-10-06T22:24:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc24d99700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7efc140a1670 0x7efc140a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8d9fbc0 con 0x55aaf8e45520 2023-10-06T22:24:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8d9d4e0 con 0x55aaf8e44c00 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e45520 0x55aaf8e542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc2759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e44fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e45520 0x55aaf8e542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55692/0 (socket says 172.21.15.57:55692) 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc24d99700 1 -- 172.21.15.57:0/3417576011 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7efc180cdda0 con 0x7efc281010d0 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc26d9d700 1 -- 172.21.15.57:0/963611214 learned_addr learned my addr 172.21.15.57:0/963611214 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc2759e700 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e56bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.000+0000 7efc1ffff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7efc140a1670 0x7efc140a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc2759e700 1 -- 172.21.15.57:0/963611214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e45520 msgr2=0x55aaf8e542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc2759e700 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e45520 0x55aaf8e542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc2759e700 1 -- 172.21.15.57:0/963611214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 msgr2=0x55aaf8e56bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc2759e700 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e56bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc2759e700 1 -- 172.21.15.57:0/963611214 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf8bc3d80 con 0x55aaf8e44c00 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc2759e700 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e44fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efc20008bb0 tx=0x7efc200099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0102365a6cd84c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc1d7fa700 1 -- 172.21.15.57:0/963611214 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20008ec0 con 0x55aaf8e44c00 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc1d7fa700 1 -- 172.21.15.57:0/963611214 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc2000f040 con 0x55aaf8e44c00 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.001+0000 7efc1d7fa700 1 -- 172.21.15.57:0/963611214 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20004030 con 0x55aaf8e44c00 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 -- 172.21.15.57:0/963611214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 msgr2=0x55aaf8e44fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e44fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efc20008bb0 tx=0x7efc200099e0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc1ffff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7efc140a1670 0x7efc140a3b30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efc28198820 tx=0x7efc10009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 -- 172.21.15.57:0/963611214 shutdown_connections 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e56bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e44fe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/963611214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e45520 0x55aaf8e542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 -- 172.21.15.57:0/963611214 >> 172.21.15.57:0/963611214 conn(0x55aaf8dba200 msgr2=0x55aaf8e41080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 -- 172.21.15.57:0/963611214 shutdown_connections 2023-10-06T22:24:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.002+0000 7efc40ab3b80 1 -- 172.21.15.57:0/963611214 wait complete. 2023-10-06T22:24:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 -- start start 2023-10-06T22:24:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 Processor -- start 2023-10-06T22:24:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e514c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e547e0 0x55aaf8e51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8d98dd0 con 0x55aaf8e45520 2023-10-06T22:24:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8d356b0 con 0x55aaf8e547e0 2023-10-06T22:24:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8d35350 con 0x55aaf8e44c00 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.003+0000 7efc40ab3b80 10 client.0 _ll_register_callbacks cb 0x55aaf8b927a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33896/0 (socket says 172.21.15.57:33896) 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 -- 172.21.15.57:0/1151007985 learned_addr learned my addr 172.21.15.57:0/1151007985 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc1ffff700 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc2759e700 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e547e0 0x55aaf8e51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 -- 172.21.15.57:0/1151007985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e547e0 msgr2=0x55aaf8e51b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e547e0 0x55aaf8e51b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 -- 172.21.15.57:0/1151007985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 msgr2=0x55aaf8e514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 -- 172.21.15.57:0/1151007985 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc20008860 con 0x55aaf8e44c00 2023-10-06T22:24:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc26d9d700 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e541d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efc18097840 tx=0x7efc1809ac20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4b9cc29d142fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc0e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc0e7fc700 1 -- 172.21.15.57:0/1151007985 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1809b230 con 0x55aaf8e44c00 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc0e7fc700 1 -- 172.21.15.57:0/1151007985 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7efc1809b4d0 con 0x55aaf8e44c00 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.004+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaf8e38d50 con 0x55aaf8e44c00 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc0e7fc700 1 -- 172.21.15.57:0/1151007985 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc180cb940 con 0x55aaf8e44c00 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 10 client.27031 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55aaf8bacc00 con 0x55aaf8e44c00 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc0e7fc700 1 -- 172.21.15.57:0/1151007985 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc1807d020 con 0x55aaf8e44c00 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc0e7fc700 1 -- 172.21.15.57:0/1151007985 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7efc180a2c40 con 0x55aaf8e44c00 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 10 client.27031 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 10 client.27031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 20 client.27031 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8e64050 0x55aaf8e66470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 4 client.27031 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaf8d91db0 con 0x55aaf8e64050 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.005+0000 7efc1ffff700 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8e64050 0x55aaf8e66470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.006+0000 7efc1ffff700 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8e64050 0x55aaf8e66470 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05993b0907b68b9 server_cookie=f76b49f0ebdd8297 in_seq=0 out_seq=0 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.006+0000 7efc0e7fc700 10 client.27031 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.006+0000 7efc0e7fc700 1 -- 172.21.15.57:0/1151007985 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aaf8d91db0 con 0x55aaf8e64050 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.006+0000 7efc0e7fc700 10 client.27031 handle_command_reply: tid=0 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 client.27031 shutdown 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 msgr2=0x55aaf8e541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e541d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efc18097840 tx=0x7efc1809ac20 comp rx=0 tx=0).stop 2023-10-06T22:24:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 shutdown_connections 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8e64050 0x55aaf8e66470 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e547e0 0x55aaf8e51b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e44c00 0x55aaf8e541d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1151007985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc0e7fc700 0 client.27031 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 >> 172.21.15.57:0/1151007985 conn(0x55aaf8dba200 msgr2=0x55aaf8e40c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 shutdown_connections 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1151007985 wait complete. 2023-10-06T22:24:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.007+0000 7efc40ab3b80 20 client.27031 trim_cache size 0 max 16384 2023-10-06T22:24:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.024+0000 7efc40ab3b80 1 Processor -- start 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.024+0000 7efc40ab3b80 1 -- start start 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e53d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e45520 0x55aaf8e471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8c198c0 con 0x55aaf8e547e0 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8e3b2f0 con 0x55aaf8e44c00 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8e4fa50 con 0x55aaf8e45520 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc2759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e45520 0x55aaf8e471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e53d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e45520 0x55aaf8e471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:33900/0 (socket says 172.21.15.57:33900) 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 -- 172.21.15.57:0/1763833640 learned_addr learned my addr 172.21.15.57:0/1763833640 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc2759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58108/0 (socket says 172.21.15.57:58108) 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 -- 172.21.15.57:0/1763833640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 msgr2=0x55aaf8e53d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 --2- 172.21.15.57:0/1763833640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e53d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 -- 172.21.15.57:0/1763833640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 msgr2=0x55aaf8e476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 --2- 172.21.15.57:0/1763833640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.025+0000 7efc1ffff700 1 -- 172.21.15.57:0/1763833640 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf8bea1f0 con 0x55aaf8e45520 2023-10-06T22:24:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.026+0000 7efc1ffff700 1 --2- 172.21.15.57:0/1763833640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e45520 0x55aaf8e471a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55aaf8e52a00 tx=0x7efc10014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba3b56ca3294474 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.026+0000 7efc1dffb700 1 -- 172.21.15.57:0/1763833640 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10025070 con 0x55aaf8e45520 2023-10-06T22:24:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.026+0000 7efc1dffb700 1 -- 172.21.15.57:0/1763833640 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc10004210 con 0x55aaf8e45520 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.026+0000 7efc1dffb700 1 -- 172.21.15.57:0/1763833640 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10003dd0 con 0x55aaf8e45520 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.026+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1763833640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e45520 msgr2=0x55aaf8e471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.026+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1763833640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e45520 0x55aaf8e471a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55aaf8e52a00 tx=0x7efc10014000 comp rx=0 tx=0).stop 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1763833640 shutdown_connections 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1763833640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e547e0 0x55aaf8e476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1763833640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e53d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/1763833640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e45520 0x55aaf8e471a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1763833640 >> 172.21.15.57:0/1763833640 conn(0x55aaf8dba200 msgr2=0x55aaf8e46a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1763833640 shutdown_connections 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 -- 172.21.15.57:0/1763833640 wait complete. 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 -- start start 2023-10-06T22:24:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.027+0000 7efc40ab3b80 1 Processor -- start 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e57ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc40ab3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e547e0 0x55aaf8eb7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8e58af0 con 0x55aaf8e45520 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8eb83c0 con 0x55aaf8e44c00 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc40ab3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8eb8500 con 0x55aaf8e547e0 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc40ab3b80 10 client.0 _ll_register_callbacks cb 0x55aaf8b927a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e57ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc2759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e547e0 0x55aaf8eb7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc26d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e57ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:55720/0 (socket says 172.21.15.57:55720) 2023-10-06T22:24:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc26d9d700 1 -- 172.21.15.57:0/3304933748 learned_addr learned my addr 172.21.15.57:0/3304933748 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc1ffff700 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc2759e700 1 -- 172.21.15.57:0/3304933748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 msgr2=0x55aaf8e57ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc2759e700 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e57ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc2759e700 1 -- 172.21.15.57:0/3304933748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 msgr2=0x55aaf8e581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc2759e700 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc2759e700 1 -- 172.21.15.57:0/3304933748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc10017070 con 0x55aaf8e547e0 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.028+0000 7efc26d9d700 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e57ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc2759e700 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e547e0 0x55aaf8eb7e80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efc20009e90 tx=0x7efc20003de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e781cc38d709b16f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc0e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc0e7fc700 1 -- 172.21.15.57:0/3304933748 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2000ee20 con 0x55aaf8e547e0 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc2000f040 con 0x55aaf8e547e0 2023-10-06T22:24:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc0e7fc700 1 -- 172.21.15.57:0/3304933748 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7efc20008030 con 0x55aaf8e547e0 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc0e7fc700 1 -- 172.21.15.57:0/3304933748 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc200081f0 con 0x55aaf8e547e0 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc40ab3b80 10 client.27037 fetch_fsmap learned FSMap version 58 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.029+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55aaf8bc2250 con 0x55aaf8e547e0 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc0e7fc700 1 -- 172.21.15.57:0/3304933748 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc2000f040 con 0x55aaf8e547e0 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc0e7fc700 1 -- 172.21.15.57:0/3304933748 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7efc20008be0 con 0x55aaf8e547e0 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc40ab3b80 10 client.27037 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc40ab3b80 10 client.27037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc40ab3b80 20 client.27037 populate_metadata read hostname 'smithi057' 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8eb9c20 0x55aaf8ebc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc40ab3b80 4 client.27037 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:24:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aaf8d91ad0 con 0x55aaf8eb9c20 2023-10-06T22:24:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc26d9d700 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8eb9c20 0x55aaf8ebc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:24:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc26d9d700 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8eb9c20 0x55aaf8ebc040 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f50f13878e0479 server_cookie=59cc50e9827842b4 in_seq=0 out_seq=0 2023-10-06T22:24:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.030+0000 7efc0e7fc700 10 client.27037 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc0e7fc700 1 -- 172.21.15.57:0/3304933748 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aaf8d91ad0 con 0x55aaf8eb9c20 2023-10-06T22:24:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc0e7fc700 10 client.27037 handle_command_reply: tid=0 2023-10-06T22:24:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 client.27037 shutdown 2023-10-06T22:24:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e547e0 msgr2=0x55aaf8eb7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e547e0 0x55aaf8eb7e80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efc20009e90 tx=0x7efc20003de0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 shutdown_connections 2023-10-06T22:24:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55aaf8eb9c20 0x55aaf8ebc040 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aaf8e547e0 0x55aaf8eb7e80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aaf8e44c00 0x55aaf8e57ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 --2- 172.21.15.57:0/3304933748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55aaf8e45520 0x55aaf8e581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 >> 172.21.15.57:0/3304933748 conn(0x55aaf8dba200 msgr2=0x55aaf8e47320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc0e7fc700 0 client.27037 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:24:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 shutdown_connections 2023-10-06T22:24:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.313+0000 7efc40ab3b80 1 -- 172.21.15.57:0/3304933748 wait complete. 2023-10-06T22:24:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc40ab3b80 20 client.27037 trim_cache size 0 max 16384 2023-10-06T22:24:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:24:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:24:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:24:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:24:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7efc140a1670 msgr2=0x7efc140a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7efc140a1670 0x7efc140a3b30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efc28198820 tx=0x7efc10009260 comp rx=0 tx=0).stop 2023-10-06T22:24:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc281010d0 msgr2=0x7efc28197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:24:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc281010d0 0x7efc28197000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc18009810 tx=0x7efc1800fa00 comp rx=0 tx=0).stop 2023-10-06T22:24:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 shutdown_connections 2023-10-06T22:24:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7efc140a1670 0x7efc140a3b30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc2819bbb0 0x7efc2819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc281010d0 0x7efc28197000 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 --2- 172.21.15.57:0/3417576011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc281061f0 0x7efc28197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:24:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.314+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 >> 172.21.15.57:0/3417576011 conn(0x7efc280fc930 msgr2=0x7efc280fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:24:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.315+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 shutdown_connections 2023-10-06T22:24:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:24:49.315+0000 7efc2d1ff700 1 -- 172.21.15.57:0/3417576011 wait complete. 2023-10-06T22:24:49.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:49 smithi190 ceph-mon[134891]: pgmap v313: 97 pgs: 97 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 336 op/s 2023-10-06T22:24:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:49 smithi057 ceph-mon[98492]: pgmap v313: 97 pgs: 97 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 336 op/s 2023-10-06T22:24:49.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:49 smithi156 ceph-mon[134358]: pgmap v313: 97 pgs: 97 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 336 op/s 2023-10-06T22:24:50.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:50 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:24:50.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:50 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:24:50.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:50 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:24:51.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:51 smithi190 ceph-mon[134891]: pgmap v314: 97 pgs: 97 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2023-10-06T22:24:51.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:51 smithi057 ceph-mon[98492]: pgmap v314: 97 pgs: 97 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2023-10-06T22:24:51.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:51 smithi156 ceph-mon[134358]: pgmap v314: 97 pgs: 97 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2023-10-06T22:24:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:53 smithi057 ceph-mon[98492]: pgmap v315: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 433 op/s 2023-10-06T22:24:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:53.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:53 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:53 smithi156 ceph-mon[134358]: pgmap v315: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 433 op/s 2023-10-06T22:24:54.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:53 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:53 smithi190 ceph-mon[134891]: pgmap v315: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 433 op/s 2023-10-06T22:24:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:54.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:53 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:54.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:54 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-10-06T22:24:54.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:54 smithi057 ceph-mon[98492]: Unable to set osd_memory_target on smithi190 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-06T22:24:55.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:54 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-10-06T22:24:55.006 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:54 smithi156 ceph-mon[134358]: Unable to set osd_memory_target on smithi190 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-06T22:24:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:54 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-10-06T22:24:55.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:54 smithi190 ceph-mon[134891]: Unable to set osd_memory_target on smithi190 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-06T22:24:55.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: pgmap v316: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2023-10-06T22:24:55.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:24:55.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:24:55.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:24:55.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:24:55.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: scrub summary: idle 2023-10-06T22:24:55.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:55.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:55 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:56.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: pgmap v316: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2023-10-06T22:24:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:24:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:24:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:24:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:24:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: scrub summary: idle 2023-10-06T22:24:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:56.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:55 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:56.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: pgmap v316: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 345 op/s 2023-10-06T22:24:56.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:24:56.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:24:56.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:24:56.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:24:56.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: scrub summary: idle 2023-10-06T22:24:56.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:56.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:55 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:24:56.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:24:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:56 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:24:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:24:57.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:56 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:57.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:24:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:24:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:24:57.070 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:56 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:24:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:57 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-10-06T22:24:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:57 smithi057 ceph-mon[98492]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T22:24:57.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:57 smithi057 ceph-mon[98492]: pgmap v317: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 376 op/s 2023-10-06T22:24:58.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:57 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-10-06T22:24:58.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:57 smithi156 ceph-mon[134358]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T22:24:58.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:57 smithi156 ceph-mon[134358]: pgmap v317: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 376 op/s 2023-10-06T22:24:58.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:57 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-10-06T22:24:58.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:57 smithi190 ceph-mon[134891]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T22:24:58.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:57 smithi190 ceph-mon[134891]: pgmap v317: 97 pgs: 97 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 376 op/s 2023-10-06T22:24:59.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:24:59 smithi057 ceph-mon[98492]: pgmap v318: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 420 op/s 2023-10-06T22:25:00.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:24:59 smithi156 ceph-mon[134358]: pgmap v318: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 420 op/s 2023-10-06T22:25:00.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:24:59 smithi190 ceph-mon[134891]: pgmap v318: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 420 op/s 2023-10-06T22:25:01.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:01 smithi057 ceph-mon[98492]: pgmap v319: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 358 op/s 2023-10-06T22:25:02.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:01 smithi156 ceph-mon[134358]: pgmap v319: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 358 op/s 2023-10-06T22:25:02.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:01 smithi190 ceph-mon[134891]: pgmap v319: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 358 op/s 2023-10-06T22:25:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/761874269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41010d0 msgr2=0x7fb4f41014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/761874269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41010d0 0x7fb4f41014b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009990 tx=0x7fb4e4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/761874269 shutdown_connections 2023-10-06T22:25:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/761874269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f41061f0 0x7fb4f410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/761874269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41010d0 0x7fb4f41014b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/761874269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f4101a80 0x7fb4f4105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/761874269 >> 172.21.15.57:0/761874269 conn(0x7fb4f40fc930 msgr2=0x7fb4f40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/761874269 shutdown_connections 2023-10-06T22:25:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.481+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/761874269 wait complete. 2023-10-06T22:25:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.482+0000 7fb4fa9ff700 1 Processor -- start 2023-10-06T22:25:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.482+0000 7fb4fa9ff700 1 -- start start 2023-10-06T22:25:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4fa9ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f41010d0 0x7fb4f4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4fa9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f41061f0 0x7fb4f40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4fa9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41a30e0 0x7fb4f41a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4fa9ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f41a5a10 con 0x7fb4f41010d0 2023-10-06T22:25:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4fa9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f41a5b50 con 0x7fb4f41a30e0 2023-10-06T22:25:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4fa9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f41a5cc0 con 0x7fb4f41061f0 2023-10-06T22:25:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4f37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f41061f0 0x7fb4f40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4f37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f41061f0 0x7fb4f40738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56238/0 (socket says 172.21.15.57:56238) 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.483+0000 7fb4f37fe700 1 -- 172.21.15.57:0/1978639691 learned_addr learned my addr 172.21.15.57:0/1978639691 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f41010d0 0x7fb4f4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41a30e0 0x7fb4f41a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f3fff700 1 -- 172.21.15.57:0/1978639691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f41061f0 msgr2=0x7fb4f40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f41061f0 0x7fb4f40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f3fff700 1 -- 172.21.15.57:0/1978639691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41a30e0 msgr2=0x7fb4f41a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41a30e0 0x7fb4f41a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f3fff700 1 -- 172.21.15.57:0/1978639691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40095f0 con 0x7fb4f41010d0 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.484+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41a30e0 0x7fb4f41a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.485+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f41010d0 0x7fb4f4073390 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006010 tx=0x7fb4e4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.485+0000 7fb4f17fa700 1 -- 172.21.15.57:0/1978639691 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400fb00 con 0x7fb4f41010d0 2023-10-06T22:25:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.485+0000 7fb4f17fa700 1 -- 172.21.15.57:0/1978639691 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4e402e070 con 0x7fb4f41010d0 2023-10-06T22:25:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.485+0000 7fb4f17fa700 1 -- 172.21.15.57:0/1978639691 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4033e60 con 0x7fb4f41010d0 2023-10-06T22:25:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.486+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f4196db0 con 0x7fb4f41010d0 2023-10-06T22:25:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.486+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f4197270 con 0x7fb4f41010d0 2023-10-06T22:25:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.492+0000 7fb50e2c0b80 1 Processor -- start 2023-10-06T22:25:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.492+0000 7fb50e2c0b80 1 -- start start 2023-10-06T22:25:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb843fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb84400520 0x55fb8440ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8440d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb84369400 con 0x55fb84400520 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8435abc0 con 0x55fb8440b3d0 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb843584e0 con 0x55fb843ffc00 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb843fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb843fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56244/0 (socket says 172.21.15.57:56244) 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8440d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2477302148 learned_addr learned my addr 172.21.15.57:0/2477302148 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2477302148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 msgr2=0x55fb8440d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/2477302148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8440d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2477302148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb84400520 msgr2=0x55fb8440ae90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:25:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/2477302148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb84400520 0x55fb8440ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.493+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2477302148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb8417ed80 con 0x55fb843ffc00 2023-10-06T22:25:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.494+0000 7fb4f17fa700 1 -- 172.21.15.57:0/1978639691 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb4e400fc60 con 0x7fb4f41010d0 2023-10-06T22:25:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.494+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/2477302148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb843fffe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec003eb0 tx=0x7fb4ec009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb8ad1d41f938ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.494+0000 7fb4da7fc700 1 -- 172.21.15.57:0/2477302148 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0092d0 con 0x55fb843ffc00 2023-10-06T22:25:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.494+0000 7fb4da7fc700 1 -- 172.21.15.57:0/2477302148 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ec00f040 con 0x55fb843ffc00 2023-10-06T22:25:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.494+0000 7fb4da7fc700 1 -- 172.21.15.57:0/2477302148 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00e750 con 0x55fb843ffc00 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2477302148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 msgr2=0x55fb843fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2477302148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb843fffe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec003eb0 tx=0x7fb4ec009920 comp rx=0 tx=0).stop 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2477302148 shutdown_connections 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2477302148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8440d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2477302148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb843fffe0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2477302148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb84400520 0x55fb8440ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2477302148 >> 172.21.15.57:0/2477302148 conn(0x55fb84375200 msgr2=0x55fb843fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2477302148 shutdown_connections 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2477302148 wait complete. 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.495+0000 7fb4f17fa700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4dc0a1650 0x7fb4dc0a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.496+0000 7fb4f17fa700 1 -- 172.21.15.57:0/1978639691 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fb4e40df060 con 0x7fb4f41010d0 2023-10-06T22:25:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.496+0000 7fb4f37fe700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4dc0a1650 0x7fb4dc0a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.496+0000 7fb50e2c0b80 1 -- start start 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 1 Processor -- start 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb4f37fe700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4dc0a1650 0x7fb4dc0a3b10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009690 tx=0x7fb4e8009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb8440adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb84400520 0x55fb84408000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb84408630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb84353dd0 con 0x55fb8440b3d0 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb842f06b0 con 0x55fb84400520 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb842f0350 con 0x55fb843ffc00 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.497+0000 7fb50e2c0b80 10 client.0 _ll_register_callbacks cb 0x55fb8414d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb84400520 0x55fb84408000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb8440adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb8440adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56250/0 (socket says 172.21.15.57:56250) 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 -- 172.21.15.57:0/1201857049 learned_addr learned my addr 172.21.15.57:0/1201857049 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb84408630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 -- 172.21.15.57:0/1201857049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb84400520 msgr2=0x55fb84408000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb84400520 0x55fb84408000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 -- 172.21.15.57:0/1201857049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 msgr2=0x55fb84408630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb84408630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.498+0000 7fb4f3fff700 1 -- 172.21.15.57:0/1201857049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec003b90 con 0x55fb843ffc00 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb8440adf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40a8590 tx=0x7fb4e40abf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6634aba0ff0eb09f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb4d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:25:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb4d37fe700 1 -- 172.21.15.57:0/1201857049 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40ac4f0 con 0x55fb843ffc00 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb4d37fe700 1 -- 172.21.15.57:0/1201857049 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fb4e40ac790 con 0x55fb843ffc00 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb843f3d50 con 0x55fb843ffc00 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb4d37fe700 1 -- 172.21.15.57:0/1201857049 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e40b0020 con 0x55fb843ffc00 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb50e2c0b80 10 client.27049 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.499+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55fb84167c00 con 0x55fb843ffc00 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb4d37fe700 1 -- 172.21.15.57:0/1201857049 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb843f3d50 con 0x55fb843ffc00 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb4d37fe700 1 -- 172.21.15.57:0/1201857049 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fb4e40b3f00 con 0x55fb843ffc00 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb50e2c0b80 10 client.27049 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb50e2c0b80 10 client.27049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb50e2c0b80 20 client.27049 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb8441ab30 0x55fb8441cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb50e2c0b80 4 client.27049 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb8434cdb0 con 0x55fb8441ab30 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.500+0000 7fb4f37fe700 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb8441ab30 0x55fb8441cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.501+0000 7fb4f37fe700 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb8441ab30 0x55fb8441cf50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2af09b89895f37d server_cookie=22845c07b55b175f in_seq=0 out_seq=0 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.501+0000 7fb4d37fe700 10 client.27049 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.502+0000 7fb4d37fe700 1 -- 172.21.15.57:0/1201857049 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fb8434cdb0 con 0x55fb8441ab30 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.502+0000 7fb4d37fe700 10 client.27049 handle_command_reply: tid=0 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.502+0000 7fb50e2c0b80 1 client.27049 shutdown 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 msgr2=0x55fb8440adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb8440adf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40a8590 tx=0x7fb4e40abf50 comp rx=0 tx=0).stop 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 shutdown_connections 2023-10-06T22:25:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb8441ab30 0x55fb8441cf50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb84408630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb4d37fe700 0 client.27049 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb843ffc00 0x55fb8440adf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/1201857049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb84400520 0x55fb84408000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 >> 172.21.15.57:0/1201857049 conn(0x55fb84375200 msgr2=0x55fb843fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 shutdown_connections 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/1201857049 wait complete. 2023-10-06T22:25:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.503+0000 7fb50e2c0b80 20 client.27049 trim_cache size 0 max 16384 2023-10-06T22:25:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.522+0000 7fb50e2c0b80 1 Processor -- start 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb50e2c0b80 1 -- start start 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb843ffc00 0x55fb84407250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84411f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb8440a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb841d48c0 con 0x55fb8440b3d0 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb84412480 con 0x55fb843ffc00 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb843fae30 con 0x55fb84400520 2023-10-06T22:25:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb843ffc00 0x55fb84407250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb843ffc00 0x55fb84407250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38176/0 (socket says 172.21.15.57:38176) 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb4f3fff700 1 -- 172.21.15.57:0/2072680264 learned_addr learned my addr 172.21.15.57:0/2072680264 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb8440a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.523+0000 7fb4f37fe700 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84411f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.524+0000 7fb4f3fff700 1 -- 172.21.15.57:0/2072680264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 msgr2=0x55fb84411f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.524+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84411f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.524+0000 7fb4f3fff700 1 -- 172.21.15.57:0/2072680264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 msgr2=0x55fb8440a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.524+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb8440a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.524+0000 7fb4f3fff700 1 -- 172.21.15.57:0/2072680264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb8440d990 con 0x55fb843ffc00 2023-10-06T22:25:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.524+0000 7fb4f37fe700 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84411f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.524+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb843ffc00 0x55fb84407250 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40ac020 tx=0x7fb4e4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3185517009369078 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.525+0000 7fb4daffd700 1 -- 172.21.15.57:0/2072680264 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40abbd0 con 0x55fb843ffc00 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.525+0000 7fb4daffd700 1 -- 172.21.15.57:0/2072680264 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4e40ac790 con 0x55fb843ffc00 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.525+0000 7fb4daffd700 1 -- 172.21.15.57:0/2072680264 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40b4420 con 0x55fb843ffc00 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2072680264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb843ffc00 msgr2=0x55fb84407250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb843ffc00 0x55fb84407250 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40ac020 tx=0x7fb4e4004710 comp rx=0 tx=0).stop 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2072680264 shutdown_connections 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb8440b3d0 0x55fb8440a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb843ffc00 0x55fb84407250 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2072680264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84411f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2072680264 >> 172.21.15.57:0/2072680264 conn(0x55fb84375200 msgr2=0x55fb84403390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2072680264 shutdown_connections 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2072680264 wait complete. 2023-10-06T22:25:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.526+0000 7fb50e2c0b80 1 -- start start 2023-10-06T22:25:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 1 Processor -- start 2023-10-06T22:25:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb843ffc00 0x55fb844698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84469e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8446a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb84412500 con 0x55fb843ffc00 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb84410f50 con 0x55fb8440b3d0 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb84411110 con 0x55fb84400520 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb50e2c0b80 10 client.0 _ll_register_callbacks cb 0x7fb4e40a83d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.527+0000 7fb4f8f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8446a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8446a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38184/0 (socket says 172.21.15.57:38184) 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2525002444 learned_addr learned my addr 172.21.15.57:0/2525002444 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb843ffc00 0x55fb844698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2525002444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 msgr2=0x55fb84469e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84469e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2525002444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb843ffc00 msgr2=0x55fb844698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb843ffc00 0x55fb844698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 -- 172.21.15.57:0/2525002444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb84167c00 con 0x55fb8440b3d0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4f8f9c700 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8446a370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec004600 tx=0x7fb4ec0077c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7604a4147d4facba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.528+0000 7fb4d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.529+0000 7fb4d37fe700 1 -- 172.21.15.57:0/2525002444 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec004c10 con 0x55fb8440b3d0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.529+0000 7fb4d37fe700 1 -- 172.21.15.57:0/2525002444 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fb4ec004d70 con 0x55fb8440b3d0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.529+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4ec00f040 con 0x55fb8440b3d0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.529+0000 7fb4d37fe700 1 -- 172.21.15.57:0/2525002444 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ec005050 con 0x55fb8440b3d0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.529+0000 7fb50e2c0b80 10 client.27032 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.529+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55fb841a51f0 con 0x55fb8440b3d0 2023-10-06T22:25:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.529+0000 7fb4d37fe700 1 -- 172.21.15.57:0/2525002444 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ec00f040 con 0x55fb8440b3d0 2023-10-06T22:25:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb4d37fe700 1 -- 172.21.15.57:0/2525002444 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fb4ec00e750 con 0x55fb8440b3d0 2023-10-06T22:25:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb50e2c0b80 10 client.27032 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb50e2c0b80 10 client.27032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb50e2c0b80 20 client.27032 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb84404d00 0x55fb84405110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb50e2c0b80 4 client.27032 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:25:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb8434cad0 con 0x55fb84404d00 2023-10-06T22:25:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.530+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb84404d00 0x55fb84405110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.531+0000 7fb4f3fff700 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb84404d00 0x55fb84405110 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ee31a8ee468f4c server_cookie=d70508d63afdbe8d in_seq=0 out_seq=0 2023-10-06T22:25:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:02.531+0000 7fb4d37fe700 10 client.27032 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:03 smithi057 ceph-mon[98492]: pgmap v320: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 405 op/s 2023-10-06T22:25:04.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:03 smithi156 ceph-mon[134358]: pgmap v320: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 405 op/s 2023-10-06T22:25:04.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:03 smithi190 ceph-mon[134891]: pgmap v320: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 405 op/s 2023-10-06T22:25:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb4d37fe700 1 -- 172.21.15.57:0/2525002444 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb8434cad0 con 0x55fb84404d00 2023-10-06T22:25:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb4d37fe700 10 client.27032 handle_command_reply: tid=0 2023-10-06T22:25:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 client.27032 shutdown 2023-10-06T22:25:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 msgr2=0x55fb8446a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8446a370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec004600 tx=0x7fb4ec0077c0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 shutdown_connections 2023-10-06T22:25:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55fb84404d00 0x55fb84405110 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb8440b3d0 0x55fb8446a370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fb843ffc00 0x55fb844698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 --2- 172.21.15.57:0/2525002444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb84400520 0x55fb84469e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb4d37fe700 0 client.27032 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 >> 172.21.15.57:0/2525002444 conn(0x55fb84375200 msgr2=0x55fb84407800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.688+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 shutdown_connections 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.689+0000 7fb50e2c0b80 1 -- 172.21.15.57:0/2525002444 wait complete. 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.689+0000 7fb50e2c0b80 20 client.27032 trim_cache size 0 max 16384 2023-10-06T22:25:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:25:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:25:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:25:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:25:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4dc0a1650 msgr2=0x7fb4dc0a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4dc0a1650 0x7fb4dc0a3b10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009690 tx=0x7fb4e8009260 comp rx=0 tx=0).stop 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f41010d0 msgr2=0x7fb4f4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f41010d0 0x7fb4f4073390 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006010 tx=0x7fb4e4005ef0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 shutdown_connections 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4dc0a1650 0x7fb4dc0a3b10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4f41a30e0 0x7fb4f41a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f41010d0 0x7fb4f4073390 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 --2- 172.21.15.57:0/1978639691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f41061f0 0x7fb4f40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 >> 172.21.15.57:0/1978639691 conn(0x7fb4f40fc930 msgr2=0x7fb4f4104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 shutdown_connections 2023-10-06T22:25:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:04.690+0000 7fb4fa9ff700 1 -- 172.21.15.57:0/1978639691 wait complete. 2023-10-06T22:25:04.758 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T22:25:04.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 -- 172.21.15.57:0/510719046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 msgr2=0x7f3d34101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:04.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 --2- 172.21.15.57:0/510719046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 0x7f3d34101490 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009990 tx=0x7f3d24009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 -- 172.21.15.57:0/510719046 shutdown_connections 2023-10-06T22:25:04.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 --2- 172.21.15.57:0/510719046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 0x7f3d3410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 --2- 172.21.15.57:0/510719046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 0x7f3d34101490 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 --2- 172.21.15.57:0/510719046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d34101a60 0x7f3d34105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:04.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 -- 172.21.15.57:0/510719046 >> 172.21.15.57:0/510719046 conn(0x7f3d340fc950 msgr2=0x7f3d340fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:04.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 -- 172.21.15.57:0/510719046 shutdown_connections 2023-10-06T22:25:04.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.989+0000 7f3d38fff700 1 -- 172.21.15.57:0/510719046 wait complete. 2023-10-06T22:25:04.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.990+0000 7f3d38fff700 1 Processor -- start 2023-10-06T22:25:04.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d38fff700 1 -- start start 2023-10-06T22:25:04.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d38fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 0x7f3d34199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:04.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d38fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 0x7f3d34199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:04.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d38fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d3419dba0 0x7f3d3419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:04.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d38fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3419e620 con 0x7f3d341061d0 2023-10-06T22:25:04.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d38fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3419e790 con 0x7f3d3419dba0 2023-10-06T22:25:04.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d38fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3419e950 con 0x7f3d341010b0 2023-10-06T22:25:04.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d31d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 0x7f3d34199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:04.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d3259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 0x7f3d34199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:04.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d31d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 0x7f3d34199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42808/0 (socket says 172.21.15.57:42808) 2023-10-06T22:25:04.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d3259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 0x7f3d34199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56276/0 (socket says 172.21.15.57:56276) 2023-10-06T22:25:05.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d31d9b700 1 -- 172.21.15.57:0/1081305481 learned_addr learned my addr 172.21.15.57:0/1081305481 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.991+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d3419dba0 0x7f3d3419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d32d9d700 1 -- 172.21.15.57:0/1081305481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 msgr2=0x7f3d34199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 0x7f3d34199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d32d9d700 1 -- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 msgr2=0x7f3d34199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 0x7f3d34199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d32d9d700 1 -- 172.21.15.57:0/1081305481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d240095f0 con 0x7f3d3419dba0 2023-10-06T22:25:05.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 0x7f3d34199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:05.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d3419dba0 0x7f3d3419e020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00efc0 tx=0x7f3d2c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d237fe700 1 -- 172.21.15.57:0/1081305481 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c018070 con 0x7f3d3419dba0 2023-10-06T22:25:05.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d237fe700 1 -- 172.21.15.57:0/1081305481 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d2c00f040 con 0x7f3d3419dba0 2023-10-06T22:25:05.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d341a5e50 con 0x7f3d3419dba0 2023-10-06T22:25:05.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d237fe700 1 -- 172.21.15.57:0/1081305481 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c00cd40 con 0x7f3d3419dba0 2023-10-06T22:25:05.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.992+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d341a6330 con 0x7f3d3419dba0 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.996+0000 7f3d237fe700 1 -- 172.21.15.57:0/1081305481 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3d2c003a00 con 0x7f3d3419dba0 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 Processor -- start 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 -- start start 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d237fe700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3d1c0a1670 0x7f3d1c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a00c0 0x55e6d50a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50ab890 0x55e6d50adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d5009460 con 0x55e6d50a09e0 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4ffac20 con 0x55e6d50ab890 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4ff8540 con 0x55e6d50a00c0 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.997+0000 7f3d237fe700 1 -- 172.21.15.57:0/1081305481 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f3d2c0ccc00 con 0x7f3d3419dba0 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3d1c0a1670 0x7f3d1c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42814/0 (socket says 172.21.15.57:42814) 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 -- 172.21.15.57:0/2177285581 learned_addr learned my addr 172.21.15.57:0/2177285581 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 -- 172.21.15.57:0/2177285581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a00c0 msgr2=0x55e6d50a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/2177285581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a00c0 0x55e6d50a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 -- 172.21.15.57:0/2177285581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50ab890 msgr2=0x55e6d50adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/2177285581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50ab890 0x55e6d50adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.998+0000 7f3d31d9b700 1 -- 172.21.15.57:0/2177285581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6d4e1ede0 con 0x55e6d50a09e0 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.999+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3d1c0a1670 0x7f3d1c0a3b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d28011e90 tx=0x7f3d280114d0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.999+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/2177285581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50ab350 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3d28011fd0 tx=0x7f3d2801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906eb8ee68a5e023 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.999+0000 7f3d20ff9700 1 -- 172.21.15.57:0/2177285581 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28020700 con 0x55e6d50a09e0 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.999+0000 7f3d20ff9700 1 -- 172.21.15.57:0/2177285581 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d28009e00 con 0x55e6d50a09e0 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:04.999+0000 7f3d20ff9700 1 -- 172.21.15.57:0/2177285581 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2801fb10 con 0x55e6d50a09e0 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2177285581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 msgr2=0x55e6d50ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2177285581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50ab350 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3d28011fd0 tx=0x7f3d2801d000 comp rx=0 tx=0).stop 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2177285581 shutdown_connections 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2177285581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50ab890 0x55e6d50adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2177285581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a00c0 0x55e6d50a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2177285581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50ab350 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2177285581 >> 172.21.15.57:0/2177285581 conn(0x55e6d5015260 msgr2=0x55e6d509c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2177285581 shutdown_connections 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2177285581 wait complete. 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.000+0000 7f3d4c9dcb80 1 -- start start 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 1 Processor -- start 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a00c0 0x55e6d50a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a09e0 0x55e6d50a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4ff3e30 con 0x55e6d50a00c0 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4f90710 con 0x55e6d50a09e0 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4f903b0 con 0x55e6d50ab890 2023-10-06T22:25:05.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d4c9dcb80 10 client.0 _ll_register_callbacks cb 0x55e6d4ded800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d3259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a00c0 0x55e6d50a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.001+0000 7f3d3259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a00c0 0x55e6d50a8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42822/0 (socket says 172.21.15.57:42822) 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d3259c700 1 -- 172.21.15.57:0/2547517303 learned_addr learned my addr 172.21.15.57:0/2547517303 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d3259c700 1 -- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 msgr2=0x55e6d50a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d3259c700 1 -- 172.21.15.57:0/2547517303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a09e0 msgr2=0x55e6d50a8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a09e0 0x55e6d50a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d3259c700 1 -- 172.21.15.57:0/2547517303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d28019040 con 0x55e6d50a00c0 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a00c0 0x55e6d50a8630 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3d24015000 tx=0x7f3d24005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923cfeaefbf78d22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.002+0000 7f3d19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:05.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2547517303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401d070 con 0x55e6d50a00c0 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6d5093f10 con 0x55e6d50a00c0 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2547517303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f3d24004500 con 0x55e6d50a00c0 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2547517303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d24009290 con 0x55e6d50a00c0 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d4c9dcb80 10 client.17016 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e6d4e07c60 con 0x55e6d50a00c0 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2547517303 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6d5093f10 con 0x55e6d50a00c0 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.003+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2547517303 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f3d2400f700 con 0x55e6d50a00c0 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d4c9dcb80 10 client.17016 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d4c9dcb80 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d4c9dcb80 20 client.17016 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:05.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d50bb430 0x55e6d50bd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d4c9dcb80 4 client.17016 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6d4fece10 con 0x55e6d50bb430 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d50bb430 0x55e6d50bd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d50bb430 0x55e6d50bd850 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43da9292765a6ec server_cookie=2c98ce4215b8161f in_seq=0 out_seq=0 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.004+0000 7f3d19ffb700 10 client.17016 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2547517303 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e6d4fece10 con 0x55e6d50bb430 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d19ffb700 10 client.17016 handle_command_reply: tid=0 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 client.17016 shutdown 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a00c0 msgr2=0x55e6d50a8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a00c0 0x55e6d50a8630 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3d24015000 tx=0x7f3d24005800 comp rx=0 tx=0).stop 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 shutdown_connections 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d50bb430 0x55e6d50bd850 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a00c0 0x55e6d50a8630 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2547517303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a09e0 0x55e6d50a8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 >> 172.21.15.57:0/2547517303 conn(0x55e6d5015260 msgr2=0x55e6d509bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d19ffb700 0 client.17016 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:05.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.005+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 shutdown_connections 2023-10-06T22:25:05.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.006+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2547517303 wait complete. 2023-10-06T22:25:05.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.006+0000 7f3d4c9dcb80 20 client.17016 trim_cache size 0 max 16384 2023-10-06T22:25:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 Processor -- start 2023-10-06T22:25:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 -- start start 2023-10-06T22:25:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d4e74920 con 0x55e6d50a09e0 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d50b28b0 con 0x55e6d50a00c0 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.023+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d50a6b90 con 0x55e6d50ab890 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38212/0 (socket says 172.21.15.57:38212) 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 -- 172.21.15.57:0/2189416365 learned_addr learned my addr 172.21.15.57:0/2189416365 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 -- 172.21.15.57:0/2189416365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 msgr2=0x55e6d50a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 -- 172.21.15.57:0/2189416365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 msgr2=0x55e6d50a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50a8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 -- 172.21.15.57:0/2189416365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6d4e45250 con 0x55e6d50a00c0 2023-10-06T22:25:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.024+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b2370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d24009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ebf28f9153a9a59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d217fa700 1 -- 172.21.15.57:0/2189416365 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401d070 con 0x55e6d50a00c0 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d217fa700 1 -- 172.21.15.57:0/2189416365 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d2401e040 con 0x55e6d50a00c0 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d217fa700 1 -- 172.21.15.57:0/2189416365 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24004ca0 con 0x55e6d50a00c0 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2189416365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 msgr2=0x55e6d50b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b2370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d24009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2189416365 shutdown_connections 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50ab890 0x55e6d50a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b2370 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2189416365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50a09e0 0x55e6d50a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2189416365 >> 172.21.15.57:0/2189416365 conn(0x55e6d5015260 msgr2=0x55e6d50a3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2189416365 shutdown_connections 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.025+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2189416365 wait complete. 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.026+0000 7f3d4c9dcb80 1 -- start start 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.026+0000 7f3d4c9dcb80 1 Processor -- start 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a09e0 0x55e6d50b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d4c9dcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50ab890 0x55e6d50b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d510e400 con 0x55e6d50ab890 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d510e570 con 0x55e6d50a00c0 2023-10-06T22:25:05.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d4c9dcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6d510e730 con 0x55e6d50a09e0 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d4c9dcb80 10 client.0 _ll_register_callbacks cb 0x7f3d24009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d32d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50ab890 0x55e6d50b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d32d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50ab890 0x55e6d50b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42850/0 (socket says 172.21.15.57:42850) 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d32d9d700 1 -- 172.21.15.57:0/2971457725 learned_addr learned my addr 172.21.15.57:0/2971457725 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.027+0000 7f3d31d9b700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a09e0 0x55e6d50b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d32d9d700 1 -- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a09e0 msgr2=0x55e6d50b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a09e0 0x55e6d50b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d32d9d700 1 -- 172.21.15.57:0/2971457725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 msgr2=0x55e6d50b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d32d9d700 1 -- 172.21.15.57:0/2971457725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6d4e07c60 con 0x55e6d50ab890 2023-10-06T22:25:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d32d9d700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50ab890 0x55e6d50b0de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c096130 tx=0x7f3d2c099af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d72c4301d929bea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2971457725 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c004ad0 con 0x55e6d50ab890 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d28009e00 con 0x55e6d50ab890 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2971457725 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f3d2c09a330 con 0x55e6d50ab890 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2971457725 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d2c099c30 con 0x55e6d50ab890 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d4c9dcb80 10 client.17022 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.028+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e6d4e1d2b0 con 0x55e6d50ab890 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2971457725 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d2c07c020 con 0x55e6d50ab890 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2971457725 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f3d2c099120 con 0x55e6d50ab890 2023-10-06T22:25:05.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d4c9dcb80 10 client.17022 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:05.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d4c9dcb80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:05.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d4c9dcb80 20 client.17022 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:05.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d510ffa0 0x55e6d51123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d4c9dcb80 4 client.17022 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T22:25:05.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.029+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e6d4fecb30 con 0x55e6d510ffa0 2023-10-06T22:25:05.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.030+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d510ffa0 0x55e6d51123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.032+0000 7f3d3259c700 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d510ffa0 0x55e6d51123c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67fbf1bdfbbfe9b server_cookie=7fa02304d637799a in_seq=0 out_seq=0 2023-10-06T22:25:05.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.032+0000 7f3d19ffb700 10 client.17022 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:05.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d19ffb700 1 -- 172.21.15.57:0/2971457725 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e6d4fecb30 con 0x55e6d510ffa0 2023-10-06T22:25:05.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d19ffb700 10 client.17022 handle_command_reply: tid=0 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 client.17022 shutdown 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50ab890 msgr2=0x55e6d50b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50ab890 0x55e6d50b0de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c096130 tx=0x7f3d2c099af0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 shutdown_connections 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55e6d510ffa0 0x55e6d51123c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6d50ab890 0x55e6d50b0de0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6d50a00c0 0x55e6d50b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d19ffb700 0 client.17022 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.033+0000 7f3d4c9dcb80 1 --2- 172.21.15.57:0/2971457725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e6d50a09e0 0x55e6d50b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 >> 172.21.15.57:0/2971457725 conn(0x55e6d5015260 msgr2=0x55e6d50a6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 shutdown_connections 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d4c9dcb80 1 -- 172.21.15.57:0/2971457725 wait complete. 2023-10-06T22:25:05.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d4c9dcb80 20 client.17022 trim_cache size 0 max 16384 2023-10-06T22:25:05.038 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:25:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-06T22:25:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-06T22:25:05.038 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3d1c0a1670 msgr2=0x7f3d1c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d38fff700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3d1c0a1670 0x7f3d1c0a3b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d28011e90 tx=0x7f3d280114d0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d3419dba0 msgr2=0x7f3d3419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.034+0000 7f3d38fff700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d3419dba0 0x7f3d3419e020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00efc0 tx=0x7f3d2c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 shutdown_connections 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3d1c0a1670 0x7f3d1c0a3b30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d3419dba0 0x7f3d3419e020 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d341010b0 0x7f3d34199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 --2- 172.21.15.57:0/1081305481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d341061d0 0x7f3d34199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 >> 172.21.15.57:0/1081305481 conn(0x7f3d340fc950 msgr2=0x7f3d3410d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 shutdown_connections 2023-10-06T22:25:05.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.035+0000 7f3d38fff700 1 -- 172.21.15.57:0/1081305481 wait complete. 2023-10-06T22:25:05.055 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T22:25:05.055 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 -- 172.21.15.57:0/2654600264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33001061d0 msgr2=0x7f330010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 --2- 172.21.15.57:0/2654600264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33001061d0 0x7f330010dd50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f32f800b0a0 tx=0x7f32f800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 -- 172.21.15.57:0/2654600264 shutdown_connections 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 --2- 172.21.15.57:0/2654600264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33001061d0 0x7f330010dd50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 --2- 172.21.15.57:0/2654600264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 0x7f3300101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 --2- 172.21.15.57:0/2654600264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3300101a60 0x7f3300105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 -- 172.21.15.57:0/2654600264 >> 172.21.15.57:0/2654600264 conn(0x7f33000fc8d0 msgr2=0x7f33000fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 -- 172.21.15.57:0/2654600264 shutdown_connections 2023-10-06T22:25:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.298+0000 7f3304fff700 1 -- 172.21.15.57:0/2654600264 wait complete. 2023-10-06T22:25:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 Processor -- start 2023-10-06T22:25:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 -- start start 2023-10-06T22:25:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 0x7f330010ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33001061d0 0x7f33001104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300110b40 0x7f3300114ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33001152b0 con 0x7f33001010b0 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3300115420 con 0x7f33001061d0 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.300+0000 7f3304fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33001155e0 con 0x7f3300110b40 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.301+0000 7f32fe59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 0x7f330010ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.301+0000 7f32fe59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 0x7f330010ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42858/0 (socket says 172.21.15.57:42858) 2023-10-06T22:25:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.301+0000 7f32fe59c700 1 -- 172.21.15.57:0/3249243662 learned_addr learned my addr 172.21.15.57:0/3249243662 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.301+0000 7f32fdd9b700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33001061d0 0x7f33001104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.301+0000 7f32fe59c700 1 -- 172.21.15.57:0/3249243662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300110b40 msgr2=0x7f3300114ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T22:25:05.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.301+0000 7f32fe59c700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300110b40 0x7f3300114ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.302+0000 7f32fe59c700 1 -- 172.21.15.57:0/3249243662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33001061d0 msgr2=0x7f33001104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.302+0000 7f32fe59c700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33001061d0 0x7f33001104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.302+0000 7f32fe59c700 1 -- 172.21.15.57:0/3249243662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f8009d00 con 0x7f33001010b0 2023-10-06T22:25:05.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.302+0000 7f32fe59c700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 0x7f330010ff60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b530 tx=0x7f32f000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.303+0000 7f32ef7fe700 1 -- 172.21.15.57:0/3249243662 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011830 con 0x7f33001010b0 2023-10-06T22:25:05.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.303+0000 7f32ef7fe700 1 -- 172.21.15.57:0/3249243662 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32f0011e50 con 0x7f33001010b0 2023-10-06T22:25:05.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.303+0000 7f32ef7fe700 1 -- 172.21.15.57:0/3249243662 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000f2e0 con 0x7f33001010b0 2023-10-06T22:25:05.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.303+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33001158c0 con 0x7f33001010b0 2023-10-06T22:25:05.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.303+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33001b3010 con 0x7f33001010b0 2023-10-06T22:25:05.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.308+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e0005320 con 0x7f33001010b0 2023-10-06T22:25:05.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.311+0000 7f32ef7fe700 1 -- 172.21.15.57:0/3249243662 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f32f0011990 con 0x7f33001010b0 2023-10-06T22:25:05.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.313+0000 7f32ef7fe700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f32e80a17d0 0x7f32e80a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.313+0000 7f32ef7fe700 1 -- 172.21.15.57:0/3249243662 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f32f00cc930 con 0x7f33001010b0 2023-10-06T22:25:05.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.314+0000 7f32fdd9b700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f32e80a17d0 0x7f32e80a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.314+0000 7f32fdd9b700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f32e80a17d0 0x7f32e80a3c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3300111750 tx=0x7f32f4008040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.318+0000 7f32ef7fe700 1 -- 172.21.15.57:0/3249243662 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f32f0095d20 con 0x7f33001010b0 2023-10-06T22:25:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.595+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32e0005190 con 0x7f33001010b0 2023-10-06T22:25:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.596+0000 7f32ef7fe700 1 -- 172.21.15.57:0/3249243662 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+11408 (secure 0 0 0) 0x7f32f0015090 con 0x7f33001010b0 2023-10-06T22:25:05.598 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:25:05.599 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:23:30.168725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":131,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":128,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25784":{"gid":25784,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/1662104434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1662104434},{"type":"v1","addr":"172.21.15.156:6835","nonce":1662104434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.598+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f32e80a17d0 msgr2=0x7f32e80a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.598+0000 7f3304fff700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f32e80a17d0 0x7f32e80a3c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3300111750 tx=0x7f32f4008040 comp rx=0 tx=0).stop 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.598+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 msgr2=0x7f330010ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.598+0000 7f3304fff700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 0x7f330010ff60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b530 tx=0x7f32f000b8f0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 shutdown_connections 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f32e80a17d0 0x7f32e80a3c90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300110b40 0x7f3300114ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33001010b0 0x7f330010ff60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 --2- 172.21.15.57:0/3249243662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33001061d0 0x7f33001104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 >> 172.21.15.57:0/3249243662 conn(0x7f33000fc8d0 msgr2=0x7f33000fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 shutdown_connections 2023-10-06T22:25:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.599+0000 7f3304fff700 1 -- 172.21.15.57:0/3249243662 wait complete. 2023-10-06T22:25:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 58 2023-10-06T22:25:05.620 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-06T22:25:05.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 -- 172.21.15.57:0/2474510662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf41083f0 msgr2=0x7f0cf4108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 --2- 172.21.15.57:0/2474510662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf41083f0 0x7f0cf4108870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009990 tx=0x7f0ce4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 -- 172.21.15.57:0/2474510662 shutdown_connections 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 --2- 172.21.15.57:0/2474510662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf411a690 0x7f0cf411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 --2- 172.21.15.57:0/2474510662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cf410f640 0x7f0cf4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 --2- 172.21.15.57:0/2474510662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf41083f0 0x7f0cf4108870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 -- 172.21.15.57:0/2474510662 >> 172.21.15.57:0/2474510662 conn(0x7f0cf406d1a0 msgr2=0x7f0cf406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 -- 172.21.15.57:0/2474510662 shutdown_connections 2023-10-06T22:25:05.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.835+0000 7f0cfadff700 1 -- 172.21.15.57:0/2474510662 wait complete. 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 Processor -- start 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 -- start start 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf41083f0 0x7f0cf41a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf410f640 0x7f0cf41a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cf411a690 0x7f0cf41aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41aaea0 con 0x7f0cf411a690 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41ab010 con 0x7f0cf410f640 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.837+0000 7f0cfadff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41ab1d0 con 0x7f0cf41083f0 2023-10-06T22:25:05.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf41083f0 0x7f0cf41a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf41083f0 0x7f0cf41a5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56352/0 (socket says 172.21.15.57:56352) 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf939c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cf411a690 0x7f0cf41aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 -- 172.21.15.57:0/3702039378 learned_addr learned my addr 172.21.15.57:0/3702039378 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf410f640 0x7f0cf41a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 -- 172.21.15.57:0/3702039378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf410f640 msgr2=0x7f0cf41a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf410f640 0x7f0cf41a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 -- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cf411a690 msgr2=0x7f0cf41aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cf411a690 0x7f0cf41aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.838+0000 7f0cf8b9b700 1 -- 172.21.15.57:0/3702039378 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40095f0 con 0x7f0cf41083f0 2023-10-06T22:25:05.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.839+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf41083f0 0x7f0cf41a5f10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9a0 tx=0x7f0ce800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.839+0000 7f0cf1ffb700 1 -- 172.21.15.57:0/3702039378 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8009790 con 0x7f0cf41083f0 2023-10-06T22:25:05.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.839+0000 7f0cf1ffb700 1 -- 172.21.15.57:0/3702039378 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ce800cbd0 con 0x7f0cf41083f0 2023-10-06T22:25:05.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.839+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf41ab4b0 con 0x7f0cf41083f0 2023-10-06T22:25:05.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.839+0000 7f0cf1ffb700 1 -- 172.21.15.57:0/3702039378 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8018680 con 0x7f0cf41083f0 2023-10-06T22:25:05.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.839+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf41b2cf0 con 0x7f0cf41083f0 2023-10-06T22:25:05.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.846+0000 7f0cf1ffb700 1 -- 172.21.15.57:0/3702039378 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0ce800cd40 con 0x7f0cf41083f0 2023-10-06T22:25:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.847+0000 7f0d0e7deb80 1 Processor -- start 2023-10-06T22:25:05.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.848+0000 7f0d0e7deb80 1 -- start start 2023-10-06T22:25:05.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.848+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad0c0 0x55ea37dad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.848+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.848+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 0x55ea37dbac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.848+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37d16460 con 0x55ea37dad9e0 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.848+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37d07c20 con 0x55ea37db8890 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.848+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37d05540 con 0x55ea37dad0c0 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.849+0000 7f0cf939c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad0c0 0x55ea37dad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.849+0000 7f0cf1ffb700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0cdc0a1670 0x7f0cdc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.849+0000 7f0cf939c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad0c0 0x55ea37dad4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56354/0 (socket says 172.21.15.57:56354) 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.849+0000 7f0cf1ffb700 1 -- 172.21.15.57:0/3702039378 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f0ce8014070 con 0x7f0cf41083f0 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.849+0000 7f0cf939c700 1 -- 172.21.15.57:0/3243753666 learned_addr learned my addr 172.21.15.57:0/3243753666 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.850+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.850+0000 7f0cf939c700 1 -- 172.21.15.57:0/3243753666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 msgr2=0x55ea37dbac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.850+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 0x55ea37dbac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.850+0000 7f0cf939c700 1 -- 172.21.15.57:0/3243753666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 msgr2=0x55ea37db8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.850+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.850+0000 7f0cf939c700 1 -- 172.21.15.57:0/3243753666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea37b2bde0 con 0x55ea37dad0c0 2023-10-06T22:25:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.850+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad0c0 0x55ea37dad4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00b520 tx=0x7f0cec00b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b72feccc7a0ec11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.851+0000 7f0cdaffd700 1 -- 172.21.15.57:0/3243753666 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec009ba0 con 0x55ea37dad0c0 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.851+0000 7f0cdaffd700 1 -- 172.21.15.57:0/3243753666 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cec00acb0 con 0x55ea37dad0c0 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.851+0000 7f0cdaffd700 1 -- 172.21.15.57:0/3243753666 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0153a0 con 0x55ea37dad0c0 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.851+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0cdc0a1670 0x7f0cdc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3243753666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad0c0 msgr2=0x55ea37dad4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad0c0 0x55ea37dad4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00b520 tx=0x7f0cec00b830 comp rx=0 tx=0).stop 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3243753666 shutdown_connections 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 0x55ea37dbac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad0c0 0x55ea37dad4a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3243753666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3243753666 >> 172.21.15.57:0/3243753666 conn(0x55ea37d22260 msgr2=0x55ea37da9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0cdc0a1670 0x7f0cdc0a3b30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0cec016fd0 tx=0x7f0cec016ee0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.852+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3243753666 shutdown_connections 2023-10-06T22:25:05.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.853+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3243753666 wait complete. 2023-10-06T22:25:05.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.853+0000 7f0d0e7deb80 1 -- start start 2023-10-06T22:25:05.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.854+0000 7f0d0e7deb80 1 Processor -- start 2023-10-06T22:25:05.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37db81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37db8890 0x55ea37db59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37d00e30 con 0x55ea37dad9e0 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37c9d710 con 0x55ea37dad0c0 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37c9d3b0 con 0x55ea37db8890 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0d0e7deb80 10 client.0 _ll_register_callbacks cb 0x55ea37afa800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37db81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37db81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38290/0 (socket says 172.21.15.57:38290) 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.855+0000 7f0cf8b9b700 1 -- 172.21.15.57:0/3452276802 learned_addr learned my addr 172.21.15.57:0/3452276802 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37db8890 0x55ea37db59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf939c700 1 -- 172.21.15.57:0/3452276802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 msgr2=0x55ea37db81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37db81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf939c700 1 -- 172.21.15.57:0/3452276802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 msgr2=0x55ea37db5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf939c700 1 -- 172.21.15.57:0/3452276802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec009d00 con 0x55ea37db8890 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cf939c700 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37db8890 0x55ea37db59d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0cec016040 tx=0x7f0cec0155c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a5af951ebf6092 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.856+0000 7f0cd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.857+0000 7f0cd3fff700 1 -- 172.21.15.57:0/3452276802 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0153a0 con 0x55ea37db8890 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.857+0000 7f0cd3fff700 1 -- 172.21.15.57:0/3452276802 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f0cec0159f0 con 0x55ea37db8890 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.857+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea37da0f10 con 0x55ea37db8890 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.857+0000 7f0cd3fff700 1 -- 172.21.15.57:0/3452276802 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cec00a5f0 con 0x55ea37db8890 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.857+0000 7f0d0e7deb80 10 client.27079 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:05.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.857+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55ea37b14c60 con 0x55ea37db8890 2023-10-06T22:25:05.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0cd3fff700 1 -- 172.21.15.57:0/3452276802 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea37da0f10 con 0x55ea37db8890 2023-10-06T22:25:05.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0cd3fff700 1 -- 172.21.15.57:0/3452276802 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f0cec0355e0 con 0x55ea37db8890 2023-10-06T22:25:05.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0d0e7deb80 10 client.27079 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:05.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0d0e7deb80 10 client.27079 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:25:05.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0d0e7deb80 20 client.27079 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:05.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37dd9270 0x55ea37ddb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0d0e7deb80 4 client.27079 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:05.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.858+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea37cf9e10 con 0x55ea37dd9270 2023-10-06T22:25:05.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.859+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37dd9270 0x55ea37ddb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.859+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37dd9270 0x55ea37ddb690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e878050c1d4a5a0f server_cookie=e30c7c8d055be2ca in_seq=0 out_seq=0 2023-10-06T22:25:05.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.859+0000 7f0cd3fff700 10 client.27079 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:25:05.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.861+0000 7f0cd3fff700 1 -- 172.21.15.57:0/3452276802 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea37cf9e10 con 0x55ea37dd9270 2023-10-06T22:25:05.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.861+0000 7f0cd3fff700 10 client.27079 handle_command_reply: tid=0 2023-10-06T22:25:05.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 client.27079 shutdown 2023-10-06T22:25:05.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37db8890 msgr2=0x55ea37db59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37db8890 0x55ea37db59d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0cec016040 tx=0x7f0cec0155c0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 shutdown_connections 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37dd9270 0x55ea37ddb690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37db8890 0x55ea37db59d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37db81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/3452276802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad9e0 0x55ea37db5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 >> 172.21.15.57:0/3452276802 conn(0x55ea37d22260 msgr2=0x55ea37da9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0cd3fff700 0 client.27079 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 shutdown_connections 2023-10-06T22:25:05.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.862+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/3452276802 wait complete. 2023-10-06T22:25:05.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.864+0000 7f0d0e7deb80 20 client.27079 trim_cache size 0 max 16384 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 Processor -- start 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 -- start start 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37dd0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37db0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37db8890 0x55ea37db0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37b81920 con 0x55ea37db8890 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37db3990 con 0x55ea37dad0c0 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.882+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37db1280 con 0x55ea37dad9e0 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37db0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37dd0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf939c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37db8890 0x55ea37db0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37db0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56370/0 (socket says 172.21.15.57:56370) 2023-10-06T22:25:05.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 -- 172.21.15.57:0/4182372802 learned_addr learned my addr 172.21.15.57:0/4182372802 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37dd0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38302/0 (socket says 172.21.15.57:38302) 2023-10-06T22:25:05.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 -- 172.21.15.57:0/4182372802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 msgr2=0x55ea37dd0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37dd0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 -- 172.21.15.57:0/4182372802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37db8890 msgr2=0x55ea37db0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37db8890 0x55ea37db0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 -- 172.21.15.57:0/4182372802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea37b52250 con 0x55ea37dad9e0 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf939c700 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37db8890 0x55ea37db0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.883+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37db0710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4000c00 tx=0x7f0ce40092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd452a80dac1bfbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0cdb7fe700 1 -- 172.21.15.57:0/4182372802 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401d050 con 0x55ea37dad9e0 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0cdb7fe700 1 -- 172.21.15.57:0/4182372802 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ce401e040 con 0x55ea37dad9e0 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0cdb7fe700 1 -- 172.21.15.57:0/4182372802 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004a30 con 0x55ea37dad9e0 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/4182372802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 msgr2=0x55ea37db0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37db0710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4000c00 tx=0x7f0ce40092d0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/4182372802 shutdown_connections 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37db8890 0x55ea37db0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37dad0c0 0x55ea37dd0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/4182372802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37db0710 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.884+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/4182372802 >> 172.21.15.57:0/4182372802 conn(0x55ea37d22260 msgr2=0x55ea37dd4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.885+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/4182372802 shutdown_connections 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.885+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/4182372802 wait complete. 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.885+0000 7f0d0e7deb80 1 -- start start 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.885+0000 7f0d0e7deb80 1 Processor -- start 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad0c0 0x55ea37dcfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37dd0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0d0e7deb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 0x55ea37dce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37dce680 con 0x55ea37dad0c0 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37dce7f0 con 0x55ea37db8890 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0d0e7deb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea37dce9b0 con 0x55ea37dad9e0 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0d0e7deb80 10 client.0 _ll_register_callbacks cb 0x7f0cec009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:05.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad0c0 0x55ea37dcfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37dd0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37dd0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56374/0 (socket says 172.21.15.57:56374) 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad0c0 0x55ea37dcfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42918/0 (socket says 172.21.15.57:42918) 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 -- 172.21.15.57:0/560222814 learned_addr learned my addr 172.21.15.57:0/560222814 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf939c700 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 0x55ea37dce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 -- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 msgr2=0x55ea37dce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 0x55ea37dce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 -- 172.21.15.57:0/560222814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad0c0 msgr2=0x55ea37dcfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad0c0 0x55ea37dcfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.886+0000 7f0cf3fff700 1 -- 172.21.15.57:0/560222814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea37b14c60 con 0x55ea37dad9e0 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0cf3fff700 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37dd0120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4025040 tx=0x7f0ce4005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bd64f6e3698dfa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0cd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:25:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0cd3fff700 1 -- 172.21.15.57:0/560222814 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400faa0 con 0x55ea37dad9e0 2023-10-06T22:25:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cec00acb0 con 0x55ea37dad9e0 2023-10-06T22:25:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0cd3fff700 1 -- 172.21.15.57:0/560222814 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f0ce400fc00 con 0x55ea37dad9e0 2023-10-06T22:25:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0cd3fff700 1 -- 172.21.15.57:0/560222814 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce400fdc0 con 0x55ea37dad9e0 2023-10-06T22:25:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0d0e7deb80 10 client.27085 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.887+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7f0cf411d430 con 0x55ea37dad9e0 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0cd3fff700 1 -- 172.21.15.57:0/560222814 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cec00acb0 con 0x55ea37dad9e0 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0cd3fff700 1 -- 172.21.15.57:0/560222814 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f0ce4022490 con 0x55ea37dad9e0 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0d0e7deb80 10 client.27085 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0d0e7deb80 10 client.27085 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0d0e7deb80 20 client.27085 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37e2de30 0x55ea37e30250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0d0e7deb80 4 client.27085 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 --> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea37cf9b30 con 0x55ea37e2de30 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.888+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37e2de30 0x55ea37e30250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:05.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.889+0000 7f0cf8b9b700 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37e2de30 0x55ea37e30250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2985eda7895416cf server_cookie=34654e5c7c62b548 in_seq=0 out_seq=0 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.889+0000 7f0cd3fff700 10 client.27085 ms_handle_connect on v2:172.21.15.156:6836/3773867071 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0cd3fff700 1 -- 172.21.15.57:0/560222814 <== mds.4 v2:172.21.15.156:6836/3773867071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+28078 (crc 0 0 0) 0x55ea37cf9b30 con 0x55ea37e2de30 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0cd3fff700 10 client.27085 handle_command_reply: tid=0 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 client.27085 shutdown 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 msgr2=0x55ea37dd0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37dd0120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4025040 tx=0x7f0ce4005f00 comp rx=0 tx=0).stop 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 shutdown_connections 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:6836/3773867071,v1:172.21.15.156:6837/3773867071] conn(0x55ea37e2de30 0x55ea37e30250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea37db8890 0x55ea37dce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ea37dad0c0 0x55ea37dcfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 --2- 172.21.15.57:0/560222814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea37dad9e0 0x55ea37dd0120 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0cd3fff700 0 client.27085 ms_handle_reset on v2:172.21.15.156:6836/3773867071 2023-10-06T22:25:05.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 >> 172.21.15.57:0/560222814 conn(0x55ea37d22260 msgr2=0x55ea37db0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 shutdown_connections 2023-10-06T22:25:05.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 1 -- 172.21.15.57:0/560222814 wait complete. 2023-10-06T22:25:05.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.890+0000 7f0d0e7deb80 20 client.27085 trim_cache size 0 max 16384 2023-10-06T22:25:05.897 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 88351292, 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628711, 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-33.jpg" 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 167578459, 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628766, 2023-10-06T22:25:05.898 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-1.jpg" 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 176735633, 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628891, 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-42.xml" 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 196937144, 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628946, 2023-10-06T22:25:05.899 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-6.xml" 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 219643474, 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628756, 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-9.jpg" 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 265176836, 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628933, 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-19.xml" 2023-10-06T22:25:05.900 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 291956509, 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633886, 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-55.xml" 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 293788785, 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628764, 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-3.jpg" 2023-10-06T22:25:05.901 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 349792818, 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628749, 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-15.jpg" 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 370026347, 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511631259, 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-69.xml" 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.902 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 408144989, 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628921, 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-23.xml" 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 414889664, 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 2199023261418, 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29" 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 434022498, 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511629226, 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-73.xml" 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 484947972, 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628652, 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-49.jpg" 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.904 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 486007126, 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628892, 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-41.xml" 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 502811322, 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628704, 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-35.jpg" 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.905 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 531962698, 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628721, 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-28.jpg" 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 549721860, 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628753, 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-12.jpg" 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.906 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 566944881, 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628651, 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-32.xml" 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 602925643, 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628947, 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-5.xml" 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.907 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 612988522, 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628945, 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-7.xml" 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 818880005, 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511634190, 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-5.xml" 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 821284007, 2023-10-06T22:25:05.908 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628713, 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-32.jpg" 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 830024297, 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628725, 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-26.jpg" 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 840360994, 2023-10-06T22:25:05.909 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628939, 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-13.xml" 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 910737369, 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628640, 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-52.jpg" 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 980390732, 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628668, 2023-10-06T22:25:05.910 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-43.jpg" 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1016860047, 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628732, 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-22.jpg" 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1061577863, 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628936, 2023-10-06T22:25:05.911 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-16.xml" 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1066223658, 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628683, 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-40.jpg" 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1213486864, 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511635225, 2023-10-06T22:25:05.912 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-16.xml" 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1292070347, 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628894, 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-39.xml" 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1359819109, 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628763, 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-4.jpg" 2023-10-06T22:25:05.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1414374689, 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628889, 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-44.xml" 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1481516929, 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511629348, 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-59.jpg" 2023-10-06T22:25:05.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1506011126, 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628902, 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-33.xml" 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1528919897, 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628815, 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-97.xml" 2023-10-06T22:25:05.915 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1551034609, 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628896, 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-37.xml" 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1580449016, 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628943, 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-9.xml" 2023-10-06T22:25:05.916 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1655569438, 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628718, 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-29.jpg" 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1669385892, 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511629249, 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-23.xml" 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.917 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1689057025, 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628917, 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-25.xml" 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1732660726, 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628949, 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-3.xml" 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.918 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1774782357, 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628707, 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-34.jpg" 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1775769023, 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628716, 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-30.jpg" 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.919 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1782606662, 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628679, 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-41.jpg" 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1795987958, 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628918, 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-24.xml" 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.920 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1828692975, 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628666, 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-44.jpg" 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1832269330, 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628940, 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-12.xml" 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.921 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1874300046, 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628714, 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-31.jpg" 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1893176122, 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628932, 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-20.xml" 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.922 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1971576019, 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628951, 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-1.xml" 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1981320787, 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628727, 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-25.jpg" 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2081386983, 2023-10-06T22:25:05.923 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628647, 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-51.jpg" 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2156522129, 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628758, 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-7.jpg" 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2259722656, 2023-10-06T22:25:05.924 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628723, 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-27.jpg" 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2266873419, 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628948, 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-4.xml" 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2307909573, 2023-10-06T22:25:05.925 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628742, 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-17.jpg" 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2311000192, 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628635, 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-53.jpg" 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.926 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2311036627, 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628895, 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-38.xml" 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2314070114, 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628694, 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-37.jpg" 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.927 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2359530033, 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628935, 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-17.xml" 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2387525055, 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628928, 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-22.xml" 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.928 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2411999478, 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628893, 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-40.xml" 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2435604883, 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628950, 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-2.xml" 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.929 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2444331978, 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628698, 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-36.jpg" 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2464837917, 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628740, 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-18.jpg" 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.930 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2486115379, 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628649, 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-50.jpg" 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2492435275, 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628750, 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-14.jpg" 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.931 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2536779642, 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628735, 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-20.jpg" 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2620082748, 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628915, 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-26.xml" 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.932 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2685034247, 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628905, 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-31.xml" 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2733946414, 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628733, 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-21.jpg" 2023-10-06T22:25:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2852451909, 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628761, 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-5.jpg" 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2859144552, 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628942, 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-10.xml" 2023-10-06T22:25:05.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2942141425, 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511634189, 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-75.jpg" 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3054403192, 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628899, 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-35.xml" 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.935 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3114963693, 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628661, 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-46.jpg" 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3162312234, 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628913, 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-27.xml" 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.936 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3207878774, 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628907, 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-30.xml" 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3296114892, 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628752, 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-13.jpg" 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.937 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3359903584, 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628903, 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-32.xml" 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3367843313, 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628755, 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-10.jpg" 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.938 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3393648093, 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511629349, 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-46.xml" 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3426118592, 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628938, 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-14.xml" 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3441492304, 2023-10-06T22:25:05.939 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628643, 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-77.xml" 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3448388280, 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511630180, 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-72.xml" 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3450381015, 2023-10-06T22:25:05.940 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628765, 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-2.jpg" 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3469945263, 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628655, 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-48.jpg" 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3484306937, 2023-10-06T22:25:05.941 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628737, 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-19.jpg" 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3504930395, 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628934, 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-18.xml" 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3540137632, 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628730, 2023-10-06T22:25:05.942 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-23.jpg" 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3566559751, 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628673, 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-42.jpg" 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3692862045, 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628659, 2023-10-06T22:25:05.943 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-47.jpg" 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3715006780, 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628728, 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-24.jpg" 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3721442921, 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628757, 2023-10-06T22:25:05.944 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-8.jpg" 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3739543453, 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628937, 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-15.xml" 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3802707189, 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628944, 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-8.xml" 2023-10-06T22:25:05.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3849277082, 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628811, 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-88.jpg" 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3855485259, 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628888, 2023-10-06T22:25:05.946 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-45.xml" 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3863049269, 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628897, 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-36.xml" 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3867894884, 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628901, 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-34.xml" 2023-10-06T22:25:05.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3895753824, 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628912, 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-28.xml" 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3957736444, 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628747, 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-16.jpg" 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.948 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3997041793, 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628663, 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-45.jpg" 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4027616028, 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628754, 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-11.jpg" 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.949 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4035095970, 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628941, 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-11.xml" 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4161509339, 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628930, 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-21.xml" 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.950 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4169361677, 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628890, 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-43.xml" 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4226131987, 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628908, 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-29.xml" 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.951 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4228047532, 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628759, 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-6.jpg" 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4228331599, 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628692, 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-38.jpg" 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4259558712, 2023-10-06T22:25:05.952 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628686, 2023-10-06T22:25:05.953 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-39.jpg" 2023-10-06T22:25:05.953 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:25:05.953 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0cdc0a1670 msgr2=0x7f0cdc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0cdc0a1670 0x7f0cdc0a3b30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0cec016fd0 tx=0x7f0cec016ee0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf41083f0 msgr2=0x7f0cf41a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf41083f0 0x7f0cf41a5f10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9a0 tx=0x7f0ce800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 shutdown_connections 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f0cdc0a1670 0x7f0cdc0a3b30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0cf411a690 0x7f0cf41aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cf41083f0 0x7f0cf41a5f10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 --2- 172.21.15.57:0/3702039378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf410f640 0x7f0cf41a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 >> 172.21.15.57:0/3702039378 conn(0x7f0cf406d1a0 msgr2=0x7f0cf4118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 shutdown_connections 2023-10-06T22:25:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:05.891+0000 7f0cfadff700 1 -- 172.21.15.57:0/3702039378 wait complete. 2023-10-06T22:25:05.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:05 smithi057 ceph-mon[98492]: pgmap v321: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2023-10-06T22:25:05.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:05 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3249243662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:25:05.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-mon[134358]: pgmap v321: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2023-10-06T22:25:05.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3249243662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:25:05.956 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: *** Caught signal (Segmentation fault) ** 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: in thread 7fa508420700 thread_name:md_log_replay 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fa518fcfcf0] 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55e846c9b11f] 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e846c9d353] 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 4: (MDLog::_replay_thread()+0x647) [0x55e846cf8bf7] 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e846d02141] 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa518fc51ca] 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 7: clone() 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2023-10-06T22:25:05.534+0000 7fa508420700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T22:25:05.957 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: in thread 7fa508420700 thread_name:md_log_replay 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fa518fcfcf0] 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55e846c9b11f] 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e846c9d353] 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 4: (MDLog::_replay_thread()+0x647) [0x55e846cf8bf7] 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e846d02141] 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa518fc51ca] 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 7: clone() 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2023-10-06T22:25:05.958 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 0> 2023-10-06T22:25:05.534+0000 7fa508420700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: in thread 7fa508420700 thread_name:md_log_replay 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fa518fcfcf0] 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55e846c9b11f] 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e846c9d353] 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 4: (MDLog::_replay_thread()+0x647) [0x55e846cf8bf7] 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e846d02141] 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa518fc51ca] 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 7: clone() 2023-10-06T22:25:05.959 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 0> 2023-10-06T22:25:05.534+0000 7fa508420700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: in thread 7fa508420700 thread_name:md_log_replay 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fa518fcfcf0] 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55e846c9b11f] 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e846c9d353] 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 4: (MDLog::_replay_thread()+0x647) [0x55e846cf8bf7] 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e846d02141] 2023-10-06T22:25:05.960 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa518fc51ca] 2023-10-06T22:25:05.961 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 7: clone() 2023-10-06T22:25:05.961 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T22:25:05.961 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[174948]: 2023-10-06T22:25:05.961 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:05 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Main process exited, code=exited, status=139/n/a 2023-10-06T22:25:05.961 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-06T22:25:06.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:05 smithi190 ceph-mon[134891]: pgmap v321: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 240 op/s 2023-10-06T22:25:06.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:05 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3249243662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 -- 172.21.15.57:0/2553811097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1144101a60 msgr2=0x7f1144105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 --2- 172.21.15.57:0/2553811097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1144101a60 0x7f1144105aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1138009990 tx=0x7f1138009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 -- 172.21.15.57:0/2553811097 shutdown_connections 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 --2- 172.21.15.57:0/2553811097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11441061d0 0x7f114410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 --2- 172.21.15.57:0/2553811097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11441010b0 0x7f1144101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 --2- 172.21.15.57:0/2553811097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1144101a60 0x7f1144105aa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 -- 172.21.15.57:0/2553811097 >> 172.21.15.57:0/2553811097 conn(0x7f11440fc950 msgr2=0x7f11440fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.173+0000 7f1148dff700 1 -- 172.21.15.57:0/2553811097 shutdown_connections 2023-10-06T22:25:06.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.174+0000 7f1148dff700 1 -- 172.21.15.57:0/2553811097 wait complete. 2023-10-06T22:25:06.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.174+0000 7f1148dff700 1 Processor -- start 2023-10-06T22:25:06.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1148dff700 1 -- start start 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1148dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11441010b0 0x7f1144199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1148dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11441061d0 0x7f11441997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1148dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f114419db80 0x7f114419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1148dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114419e600 con 0x7f114419db80 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1148dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114419e770 con 0x7f11441061d0 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1148dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114419e930 con 0x7f11441010b0 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f114259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11441010b0 0x7f1144199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f114259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11441010b0 0x7f1144199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56404/0 (socket says 172.21.15.57:56404) 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1142d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f114419db80 0x7f114419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f114259c700 1 -- 172.21.15.57:0/1613866101 learned_addr learned my addr 172.21.15.57:0/1613866101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.175+0000 7f1141d9b700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11441061d0 0x7f11441997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f1141d9b700 1 -- 172.21.15.57:0/1613866101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11441010b0 msgr2=0x7f1144199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f1141d9b700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11441010b0 0x7f1144199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f1141d9b700 1 -- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f114419db80 msgr2=0x7f114419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f1141d9b700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f114419db80 0x7f114419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f1141d9b700 1 -- 172.21.15.57:0/1613866101 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11380095f0 con 0x7f11441061d0 2023-10-06T22:25:06.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f114259c700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11441010b0 0x7f1144199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:06.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f1141d9b700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11441061d0 0x7f11441997c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1138000c00 tx=0x7f1138005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f11337fe700 1 -- 172.21.15.57:0/1613866101 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113801d070 con 0x7f11441061d0 2023-10-06T22:25:06.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f11337fe700 1 -- 172.21.15.57:0/1613866101 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1138021410 con 0x7f11441061d0 2023-10-06T22:25:06.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11441a5e30 con 0x7f11441061d0 2023-10-06T22:25:06.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.176+0000 7f11337fe700 1 -- 172.21.15.57:0/1613866101 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138021ea0 con 0x7f11441061d0 2023-10-06T22:25:06.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.177+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11441a61f0 con 0x7f11441061d0 2023-10-06T22:25:06.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.180+0000 7f11337fe700 1 -- 172.21.15.57:0/1613866101 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1138021580 con 0x7f11441061d0 2023-10-06T22:25:06.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.181+0000 7f115c6d3b80 1 Processor -- start 2023-10-06T22:25:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f115c6d3b80 1 -- start start 2023-10-06T22:25:06.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f11337fe700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f112c0a15f0 0x7f112c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f11337fe700 1 -- 172.21.15.57:0/1613866101 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f11380ce130 con 0x7f11441061d0 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f1b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f1dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f1142d9d700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f112c0a15f0 0x7f112c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e79460 con 0x558661f1b890 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e6ac20 con 0x558661f100c0 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.182+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e68540 con 0x558661f109e0 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f1141d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f1b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38344/0 (socket says 172.21.15.57:38344) 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 -- 172.21.15.57:0/2458566587 learned_addr learned my addr 172.21.15.57:0/2458566587 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f1142d9d700 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f1dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 -- 172.21.15.57:0/2458566587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 msgr2=0x558661f1b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f1b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 -- 172.21.15.57:0/2458566587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 msgr2=0x558661f1dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f1dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 -- 172.21.15.57:0/2458566587 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558661c8ede0 con 0x558661f100c0 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f1142d9d700 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f1dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f114259c700 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f104a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11340117e0 tx=0x7f1134011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=413d27f2a71fed1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f1142d9d700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f112c0a15f0 0x7f112c0a3ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f113c009ec0 tx=0x7f113c009920 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.183+0000 7f1130ff9700 1 -- 172.21.15.57:0/2458566587 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400f480 con 0x558661f100c0 2023-10-06T22:25:06.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f1130ff9700 1 -- 172.21.15.57:0/2458566587 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f113400faa0 con 0x558661f100c0 2023-10-06T22:25:06.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f1130ff9700 1 -- 172.21.15.57:0/2458566587 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134019a20 con 0x558661f100c0 2023-10-06T22:25:06.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 -- 172.21.15.57:0/2458566587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 msgr2=0x558661f104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f104a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11340117e0 tx=0x7f1134011af0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 -- 172.21.15.57:0/2458566587 shutdown_connections 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f1dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f104a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/2458566587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f1b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 -- 172.21.15.57:0/2458566587 >> 172.21.15.57:0/2458566587 conn(0x558661e85260 msgr2=0x558661f0c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.184+0000 7f115c6d3b80 1 -- 172.21.15.57:0/2458566587 shutdown_connections 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.185+0000 7f115c6d3b80 1 -- 172.21.15.57:0/2458566587 wait complete. 2023-10-06T22:25:06.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.185+0000 7f115c6d3b80 1 -- start start 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.185+0000 7f115c6d3b80 1 Processor -- start 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f100c0 0x558661f1b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f109e0 0x558661f184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 0x558661f18ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e63e30 con 0x558661f100c0 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e00710 con 0x558661f109e0 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661e003b0 con 0x558661f1b890 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f115c6d3b80 10 client.0 _ll_register_callbacks cb 0x558661c5d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.186+0000 7f114259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f100c0 0x558661f1b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f114259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f100c0 0x558661f1b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42962/0 (socket says 172.21.15.57:42962) 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1142d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 0x558661f18ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1141d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f109e0 0x558661f184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f114259c700 1 -- 172.21.15.57:0/3332948249 learned_addr learned my addr 172.21.15.57:0/3332948249 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1141d9b700 1 -- 172.21.15.57:0/3332948249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 msgr2=0x558661f18ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1141d9b700 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 0x558661f18ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1141d9b700 1 -- 172.21.15.57:0/3332948249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f100c0 msgr2=0x558661f1b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1141d9b700 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f100c0 0x558661f1b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1141d9b700 1 -- 172.21.15.57:0/3332948249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1134011440 con 0x558661f109e0 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f114259c700 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f100c0 0x558661f1b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1141d9b700 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f109e0 0x558661f184a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11380cb840 tx=0x7f113809ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f42bd289816b04d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1129ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1129ffb700 1 -- 172.21.15.57:0/3332948249 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113809b160 con 0x558661f109e0 2023-10-06T22:25:06.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f1129ffb700 1 -- 172.21.15.57:0/3332948249 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f113809f2b0 con 0x558661f109e0 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.187+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558661f03f10 con 0x558661f109e0 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f1129ffb700 1 -- 172.21.15.57:0/3332948249 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11380a3460 con 0x558661f109e0 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 10 client.27071 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x558661c77c60 con 0x558661f109e0 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f1129ffb700 1 -- 172.21.15.57:0/3332948249 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558661f03f10 con 0x558661f109e0 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f1129ffb700 1 -- 172.21.15.57:0/3332948249 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f11380a3120 con 0x558661f109e0 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 10 client.27071 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 10 client.27071 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 20 client.27071 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f2af90 0x558661f2d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 4 client.27071 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:06.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558661e5ce10 con 0x558661f2af90 2023-10-06T22:25:06.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.188+0000 7f114259c700 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f2af90 0x558661f2d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.189+0000 7f114259c700 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f2af90 0x558661f2d3b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11214a3bfc9c7c91 server_cookie=be66453fdf0294fe in_seq=0 out_seq=0 2023-10-06T22:25:06.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.189+0000 7f1129ffb700 10 client.27071 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:25:06.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f1129ffb700 1 -- 172.21.15.57:0/3332948249 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558661e5ce10 con 0x558661f2af90 2023-10-06T22:25:06.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f1129ffb700 10 client.27071 handle_command_reply: tid=0 2023-10-06T22:25:06.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 client.27071 shutdown 2023-10-06T22:25:06.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f109e0 msgr2=0x558661f184a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f109e0 0x558661f184a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11380cb840 tx=0x7f113809ad90 comp rx=0 tx=0).stop 2023-10-06T22:25:06.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 shutdown_connections 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f2af90 0x558661f2d3b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 0x558661f18ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f100c0 0x558661f1b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f1129ffb700 0 client.27071 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/3332948249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f109e0 0x558661f184a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 >> 172.21.15.57:0/3332948249 conn(0x558661e85260 msgr2=0x558661f0bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 shutdown_connections 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.190+0000 7f115c6d3b80 1 -- 172.21.15.57:0/3332948249 wait complete. 2023-10-06T22:25:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.191+0000 7f115c6d3b80 20 client.27071 trim_cache size 0 max 16384 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.204+0000 7f115c6d3b80 1 Processor -- start 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.204+0000 7f115c6d3b80 1 -- start start 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.204+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.204+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f13df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661ce4920 con 0x558661f1b890 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661f0dd40 con 0x558661f100c0 2023-10-06T22:25:06.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661f118b0 con 0x558661f109e0 2023-10-06T22:25:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f1141d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f114259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f1141d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56432/0 (socket says 172.21.15.57:56432) 2023-10-06T22:25:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f1141d9b700 1 -- 172.21.15.57:0/501133119 learned_addr learned my addr 172.21.15.57:0/501133119 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f1142d9d700 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f13df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f114259c700 1 -- 172.21.15.57:0/501133119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 msgr2=0x558661f138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f114259c700 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f114259c700 1 -- 172.21.15.57:0/501133119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 msgr2=0x558661f13df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f114259c700 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f13df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f114259c700 1 -- 172.21.15.57:0/501133119 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558661cb5250 con 0x558661f100c0 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f1141d9b700 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.205+0000 7f114259c700 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f1adc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x558661f2dc60 tx=0x7f1134009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a64b7552aef2289 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f11317fa700 1 -- 172.21.15.57:0/501133119 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134019410 con 0x558661f100c0 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f11317fa700 1 -- 172.21.15.57:0/501133119 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1134029070 con 0x558661f100c0 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f11317fa700 1 -- 172.21.15.57:0/501133119 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134003680 con 0x558661f100c0 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 -- 172.21.15.57:0/501133119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 msgr2=0x558661f1adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f1adc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x558661f2dc60 tx=0x7f1134009810 comp rx=0 tx=0).stop 2023-10-06T22:25:06.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 -- 172.21.15.57:0/501133119 shutdown_connections 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f1b890 0x558661f13df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f1adc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/501133119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f109e0 0x558661f138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 -- 172.21.15.57:0/501133119 >> 172.21.15.57:0/501133119 conn(0x558661e85260 msgr2=0x558661f0abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.206+0000 7f115c6d3b80 1 -- 172.21.15.57:0/501133119 shutdown_connections 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.207+0000 7f115c6d3b80 1 -- 172.21.15.57:0/501133119 wait complete. 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.207+0000 7f115c6d3b80 1 -- start start 2023-10-06T22:25:06.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.207+0000 7f115c6d3b80 1 Processor -- start 2023-10-06T22:25:06.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f109e0 0x558661f22260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f115c6d3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 0x558661f203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661f209d0 con 0x558661f109e0 2023-10-06T22:25:06.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661f20b40 con 0x558661f100c0 2023-10-06T22:25:06.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f115c6d3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661f20d00 con 0x558661f1b890 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f115c6d3b80 10 client.0 _ll_register_callbacks cb 0x7f113809af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f1141d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f109e0 0x558661f22260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f1141d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f109e0 0x558661f22260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42972/0 (socket says 172.21.15.57:42972) 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f1141d9b700 1 -- 172.21.15.57:0/571493548 learned_addr learned my addr 172.21.15.57:0/571493548 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.208+0000 7f114259c700 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1141d9b700 1 -- 172.21.15.57:0/571493548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 msgr2=0x558661f203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1141d9b700 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 0x558661f203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1141d9b700 1 -- 172.21.15.57:0/571493548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 msgr2=0x558661f21d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1141d9b700 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f21d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1141d9b700 1 -- 172.21.15.57:0/571493548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558661c77c60 con 0x558661f109e0 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f114259c700 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:06.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1141d9b700 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f109e0 0x558661f22260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1138009ce0 tx=0x7f1138021b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bab33dbfb19145 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1129ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1129ffb700 1 -- 172.21.15.57:0/571493548 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113809f2b0 con 0x558661f109e0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1129ffb700 1 -- 172.21.15.57:0/571493548 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f11380b8060 con 0x558661f109e0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f113400faa0 con 0x558661f109e0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f1129ffb700 1 -- 172.21.15.57:0/571493548 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f113809b160 con 0x558661f109e0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f115c6d3b80 10 client.17061 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.209+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x558661c8ede0 con 0x558661f109e0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f1129ffb700 1 -- 172.21.15.57:0/571493548 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f113400faa0 con 0x558661f109e0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f1129ffb700 1 -- 172.21.15.57:0/571493548 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f11380b3b90 con 0x558661f109e0 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f115c6d3b80 10 client.17061 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f115c6d3b80 10 client.17061 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-06T22:25:06.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f115c6d3b80 20 client.17061 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f7fc80 0x558661f820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f115c6d3b80 4 client.17061 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 --> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558661e5cb30 con 0x558661f7fc80 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.210+0000 7f114259c700 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f7fc80 0x558661f820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f114259c700 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f7fc80 0x558661f820a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4a79ea9bea845c3 server_cookie=eeb451e3c60b1ee5 in_seq=0 out_seq=0 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f1129ffb700 10 client.17061 ms_handle_connect on v2:172.21.15.190:6834/871306260 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f1129ffb700 1 -- 172.21.15.57:0/571493548 <== mds.3 v2:172.21.15.190:6834/871306260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+4482 (crc 0 0 0) 0x558661e5cb30 con 0x558661f7fc80 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f1129ffb700 10 client.17061 handle_command_reply: tid=0 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 client.17061 shutdown 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f109e0 msgr2=0x558661f22260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f109e0 0x558661f22260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1138009ce0 tx=0x7f1138021b40 comp rx=0 tx=0).stop 2023-10-06T22:25:06.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 shutdown_connections 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.190:6834/871306260,v1:172.21.15.190:6835/871306260] conn(0x558661f7fc80 0x558661f820a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558661f1b890 0x558661f203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558661f100c0 0x558661f21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 --2- 172.21.15.57:0/571493548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558661f109e0 0x558661f22260 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f1129ffb700 0 client.17061 ms_handle_reset on v2:172.21.15.190:6834/871306260 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 >> 172.21.15.57:0/571493548 conn(0x558661e85260 msgr2=0x558661f138b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 shutdown_connections 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.211+0000 7f115c6d3b80 1 -- 172.21.15.57:0/571493548 wait complete. 2023-10-06T22:25:06.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f115c6d3b80 20 client.17061 trim_cache size 0 max 16384 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 119042157, 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633218, 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-50.xml" 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 859134248, 2023-10-06T22:25:06.220 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633198, 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-69.xml" 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 921649095, 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633216, 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-52.xml" 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1011791416, 2023-10-06T22:25:06.221 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633222, 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-47.xml" 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1142437002, 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633201, 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-66.xml" 2023-10-06T22:25:06.222 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1382557549, 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633193, 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-74.xml" 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1501651205, 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633195, 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-72.xml" 2023-10-06T22:25:06.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2178890934, 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633209, 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-59.xml" 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.224 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2264349643, 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633197, 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-70.xml" 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2378627981, 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633203, 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-64.xml" 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.225 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2381279710, 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633914, 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/picture-25.jpg" 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2576460873, 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633200, 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-67.xml" 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.226 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 2607808755, 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633191, 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-76.xml" 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3101419895, 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633190, 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-77.xml" 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.227 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3644703633, 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633205, 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-62.xml" 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 3725683812, 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633225, 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-44.xml" 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.228 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4050290929, 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633223, 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-46.xml" 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4099676203, 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511633221, 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-48.xml" 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:25:06.229 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-06T22:25:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f112c0a15f0 msgr2=0x7f112c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f112c0a15f0 0x7f112c0a3ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f113c009ec0 tx=0x7f113c009920 comp rx=0 tx=0).stop 2023-10-06T22:25:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11441061d0 msgr2=0x7f11441997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11441061d0 0x7f11441997c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1138000c00 tx=0x7f1138005740 comp rx=0 tx=0).stop 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 shutdown_connections 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f112c0a15f0 0x7f112c0a3ab0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f114419db80 0x7f114419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11441010b0 0x7f1144199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 --2- 172.21.15.57:0/1613866101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11441061d0 0x7f11441997c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 >> 172.21.15.57:0/1613866101 conn(0x7f11440fc950 msgr2=0x7f11440fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.212+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 shutdown_connections 2023-10-06T22:25:06.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.213+0000 7f1148dff700 1 -- 172.21.15.57:0/1613866101 wait complete. 2023-10-06T22:25:06.233 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T22:25:06.254 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:06 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Failed with result 'exit-code'. 2023-10-06T22:25:06.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.445+0000 7f956ebff700 1 -- 172.21.15.57:0/2418565058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9568101a60 msgr2=0x7f9568105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.445+0000 7f956ebff700 1 --2- 172.21.15.57:0/2418565058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9568101a60 0x7f9568105aa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9554009990 tx=0x7f9554009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.446+0000 7f956ebff700 1 -- 172.21.15.57:0/2418565058 shutdown_connections 2023-10-06T22:25:06.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.446+0000 7f956ebff700 1 --2- 172.21.15.57:0/2418565058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95681061d0 0x7f956810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.446+0000 7f956ebff700 1 --2- 172.21.15.57:0/2418565058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95681010b0 0x7f9568101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.446+0000 7f956ebff700 1 --2- 172.21.15.57:0/2418565058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9568101a60 0x7f9568105aa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.446+0000 7f956ebff700 1 -- 172.21.15.57:0/2418565058 >> 172.21.15.57:0/2418565058 conn(0x7f95680fc950 msgr2=0x7f95680fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.446+0000 7f956ebff700 1 -- 172.21.15.57:0/2418565058 shutdown_connections 2023-10-06T22:25:06.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.446+0000 7f956ebff700 1 -- 172.21.15.57:0/2418565058 wait complete. 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 Processor -- start 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 -- start start 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95681010b0 0x7f9568199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95681061d0 0x7f9568199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956819dba0 0x7f956819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956819e620 con 0x7f95681061d0 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956819e790 con 0x7f956819dba0 2023-10-06T22:25:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956ebff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956819e950 con 0x7f95681010b0 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95681010b0 0x7f9568199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.447+0000 7f956d19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956819dba0 0x7f956819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956819dba0 0x7f956819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38390/0 (socket says 172.21.15.57:38390) 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95681010b0 0x7f9568199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56456/0 (socket says 172.21.15.57:56456) 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 -- 172.21.15.57:0/3362227950 learned_addr learned my addr 172.21.15.57:0/3362227950 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f9567fff700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95681061d0 0x7f9568199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 -- 172.21.15.57:0/3362227950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95681010b0 msgr2=0x7f9568199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95681010b0 0x7f9568199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 -- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95681061d0 msgr2=0x7f9568199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95681061d0 0x7f9568199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 -- 172.21.15.57:0/3362227950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c009c40 con 0x7f956819dba0 2023-10-06T22:25:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956d19c700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956819dba0 0x7f956819e020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f956000ea80 tx=0x7f956000ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f9565ffb700 1 -- 172.21.15.57:0/3362227950 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95600098a0 con 0x7f956819dba0 2023-10-06T22:25:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9560009520 con 0x7f956819dba0 2023-10-06T22:25:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f9565ffb700 1 -- 172.21.15.57:0/3362227950 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f956000ce10 con 0x7f956819dba0 2023-10-06T22:25:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.448+0000 7f9565ffb700 1 -- 172.21.15.57:0/3362227950 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560003dc0 con 0x7f956819dba0 2023-10-06T22:25:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.449+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95540095f0 con 0x7f956819dba0 2023-10-06T22:25:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.452+0000 7f9565ffb700 1 -- 172.21.15.57:0/3362227950 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f956000c950 con 0x7f956819dba0 2023-10-06T22:25:06.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.453+0000 7f958248cb80 1 Processor -- start 2023-10-06T22:25:06.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f958248cb80 1 -- start start 2023-10-06T22:25:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f958248cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a14260c0 0x5627a14264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f958248cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a1431350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f958248cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a1431890 0x5627a1433c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f958248cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a138f460 con 0x5627a1431890 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f958248cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a1380c20 con 0x5627a14260c0 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f958248cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a137e540 con 0x5627a14269e0 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f956c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a1431350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f956c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a1431350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56468/0 (socket says 172.21.15.57:56468) 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f956c99b700 1 -- 172.21.15.57:0/2723040596 learned_addr learned my addr 172.21.15.57:0/2723040596 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f9567fff700 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a14260c0 0x5627a14264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.454+0000 7f9565ffb700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f95580a15f0 0x7f95580a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f9567fff700 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a1431890 0x5627a1433c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f9565ffb700 1 -- 172.21.15.57:0/3362227950 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f9560014070 con 0x7f956819dba0 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956d19c700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f95580a15f0 0x7f95580a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956c99b700 1 -- 172.21.15.57:0/2723040596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a14260c0 msgr2=0x5627a14264a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956c99b700 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a14260c0 0x5627a14264a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956c99b700 1 -- 172.21.15.57:0/2723040596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a1431890 msgr2=0x5627a1433c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956c99b700 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a1431890 0x5627a1433c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956c99b700 1 -- 172.21.15.57:0/2723040596 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a11a4de0 con 0x5627a14269e0 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f9567fff700 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a1431890 0x5627a1433c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956c99b700 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a1431350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e750 tx=0x7f955c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80aae6b6a57863c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f956d19c700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f95580a15f0 0x7f95580a3ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9560009230 tx=0x7f9560003820 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f9552ffd700 1 -- 172.21.15.57:0/2723040596 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c00e040 con 0x5627a14269e0 2023-10-06T22:25:06.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f9552ffd700 1 -- 172.21.15.57:0/2723040596 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f955c00f040 con 0x5627a14269e0 2023-10-06T22:25:06.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.455+0000 7f9552ffd700 1 -- 172.21.15.57:0/2723040596 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0158f0 con 0x5627a14269e0 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 -- 172.21.15.57:0/2723040596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 msgr2=0x5627a1431350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a1431350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e750 tx=0x7f955c009920 comp rx=0 tx=0).stop 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 -- 172.21.15.57:0/2723040596 shutdown_connections 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a1431890 0x5627a1433c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a14260c0 0x5627a14264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 --2- 172.21.15.57:0/2723040596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a1431350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 -- 172.21.15.57:0/2723040596 >> 172.21.15.57:0/2723040596 conn(0x5627a139b260 msgr2=0x5627a1422450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 -- 172.21.15.57:0/2723040596 shutdown_connections 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.456+0000 7f958248cb80 1 -- 172.21.15.57:0/2723040596 wait complete. 2023-10-06T22:25:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.457+0000 7f958248cb80 1 -- start start 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.457+0000 7f958248cb80 1 Processor -- start 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.457+0000 7f958248cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14260c0 0x5627a1431300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.457+0000 7f958248cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a142e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.457+0000 7f958248cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a142ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f958248cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a1379e30 con 0x5627a14260c0 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f958248cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a1316710 con 0x5627a1431890 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f958248cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a13163b0 con 0x5627a14269e0 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f958248cb80 10 client.0 _ll_register_callbacks cb 0x5627a1173800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f9567fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14260c0 0x5627a1431300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a142e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a142e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56480/0 (socket says 172.21.15.57:56480) 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f9567fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14260c0 0x5627a1431300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43010/0 (socket says 172.21.15.57:43010) 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 -- 172.21.15.57:0/3915875116 learned_addr learned my addr 172.21.15.57:0/3915875116 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f9567fff700 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a142ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 -- 172.21.15.57:0/3915875116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 msgr2=0x5627a142ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a142ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 -- 172.21.15.57:0/3915875116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14260c0 msgr2=0x5627a1431300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14260c0 0x5627a1431300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f956c99b700 1 -- 172.21.15.57:0/3915875116 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c00e430 con 0x5627a14269e0 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.458+0000 7f9567fff700 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a142ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f956c99b700 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a142e510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f955c009f60 tx=0x7f955c005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a86ebb4d59f3530 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f9547fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:25:06.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f9547fff700 1 -- 172.21.15.57:0/3915875116 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c004450 con 0x5627a14269e0 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627a1419f10 con 0x5627a14269e0 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f9547fff700 1 -- 172.21.15.57:0/3915875116 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f955c025070 con 0x5627a14269e0 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f9547fff700 1 -- 172.21.15.57:0/3915875116 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955c0045b0 con 0x5627a14269e0 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f958248cb80 10 client.27112 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.459+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5627a118dc60 con 0x5627a14269e0 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f9547fff700 1 -- 172.21.15.57:0/3915875116 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627a1419f10 con 0x5627a14269e0 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f9547fff700 1 -- 172.21.15.57:0/3915875116 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f955c007500 con 0x5627a14269e0 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f958248cb80 10 client.27112 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f958248cb80 10 client.27112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f958248cb80 20 client.27112 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:06.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f958248cb80 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a14412c0 0x5627a14436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f958248cb80 4 client.27112 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:06.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627a1372e10 con 0x5627a14412c0 2023-10-06T22:25:06.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.460+0000 7f9567fff700 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a14412c0 0x5627a14436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.461+0000 7f9567fff700 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a14412c0 0x5627a14436e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=986c8c8e1b4fb2f0 server_cookie=bd3c30017ff9743b in_seq=0 out_seq=0 2023-10-06T22:25:06.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.461+0000 7f9547fff700 10 client.27112 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.461+0000 7f9547fff700 1 -- 172.21.15.57:0/3915875116 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5627a1372e10 con 0x5627a14412c0 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.461+0000 7f9547fff700 10 client.27112 handle_command_reply: tid=0 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.461+0000 7f958248cb80 1 client.27112 shutdown 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 msgr2=0x5627a142e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a142e510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f955c009f60 tx=0x7f955c005ae0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 shutdown_connections 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a14412c0 0x5627a14436e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a142ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14260c0 0x5627a1431300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f9547fff700 0 client.27112 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:25:06.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 --2- 172.21.15.57:0/3915875116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14269e0 0x5627a142e510 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 >> 172.21.15.57:0/3915875116 conn(0x5627a139b260 msgr2=0x5627a1421fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 shutdown_connections 2023-10-06T22:25:06.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 1 -- 172.21.15.57:0/3915875116 wait complete. 2023-10-06T22:25:06.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.462+0000 7f958248cb80 20 client.27112 trim_cache size 0 max 16384 2023-10-06T22:25:06.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 Processor -- start 2023-10-06T22:25:06.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 -- start start 2023-10-06T22:25:06.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a142e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1428f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a1430e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a11fa920 con 0x5627a14269e0 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a143e970 con 0x5627a1431890 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f958248cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a1423020 con 0x5627a14260c0 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f9567fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a142e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f9567fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a142e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56492/0 (socket says 172.21.15.57:56492) 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f9567fff700 1 -- 172.21.15.57:0/1312129608 learned_addr learned my addr 172.21.15.57:0/1312129608 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.477+0000 7f9567fff700 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a1430e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f956c99b700 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1428f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f9567fff700 1 -- 172.21.15.57:0/1312129608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 msgr2=0x5627a1430e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f9567fff700 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a1430e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f9567fff700 1 -- 172.21.15.57:0/1312129608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 msgr2=0x5627a1428f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f9567fff700 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1428f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f9567fff700 1 -- 172.21.15.57:0/1312129608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a11cb250 con 0x5627a14260c0 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f956c99b700 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1428f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f9567fff700 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a142e4c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f956806a470 tx=0x7f9554009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48de855eee937bce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f95537fe700 1 -- 172.21.15.57:0/1312129608 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955401d040 con 0x5627a14260c0 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f95537fe700 1 -- 172.21.15.57:0/1312129608 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9554035020 con 0x5627a14260c0 2023-10-06T22:25:06.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.478+0000 7f95537fe700 1 -- 172.21.15.57:0/1312129608 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955402e3c0 con 0x5627a14260c0 2023-10-06T22:25:06.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 -- 172.21.15.57:0/1312129608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 msgr2=0x5627a142e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a142e4c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f956806a470 tx=0x7f9554009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 -- 172.21.15.57:0/1312129608 shutdown_connections 2023-10-06T22:25:06.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a1430e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a142e4c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 --2- 172.21.15.57:0/1312129608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1428f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 -- 172.21.15.57:0/1312129608 >> 172.21.15.57:0/1312129608 conn(0x5627a139b260 msgr2=0x5627a142ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 -- 172.21.15.57:0/1312129608 shutdown_connections 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 -- 172.21.15.57:0/1312129608 wait complete. 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.479+0000 7f958248cb80 1 -- start start 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 1 Processor -- start 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a1437fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1438510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a14364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a1436bf0 con 0x5627a14269e0 2023-10-06T22:25:06.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a1436d60 con 0x5627a1431890 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a1436f20 con 0x5627a14260c0 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.480+0000 7f958248cb80 10 client.0 _ll_register_callbacks cb 0x7f9560005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1438510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1438510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43026/0 (socket says 172.21.15.57:43026) 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 -- 172.21.15.57:0/1413966436 learned_addr learned my addr 172.21.15.57:0/1413966436 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f9567fff700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a1437fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f9567fff700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a14364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 -- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 msgr2=0x5627a1437fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a1437fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 -- 172.21.15.57:0/1413966436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 msgr2=0x5627a14364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a14364b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 -- 172.21.15.57:0/1413966436 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9554020040 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f9567fff700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a1437fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f956c99b700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1438510 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f955c009960 tx=0x7f955c007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2b929f9aa25cf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.481+0000 7f9547fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f9547fff700 1 -- 172.21.15.57:0/1413966436 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c004660 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f955c00f040 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f9547fff700 1 -- 172.21.15.57:0/1413966436 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f955c025040 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f9547fff700 1 -- 172.21.15.57:0/1413966436 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955c00ed40 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 10 client.17079 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5627a11a32b0 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f9547fff700 1 -- 172.21.15.57:0/1413966436 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f955c03e080 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f9547fff700 1 -- 172.21.15.57:0/1413966436 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f955c01f430 con 0x5627a14269e0 2023-10-06T22:25:06.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 10 client.17079 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 10 client.17079 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 20 client.17079 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a1496210 0x5627a1498630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 4 client.17079 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.482+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 --> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627a1372b30 con 0x5627a1496210 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f9567fff700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a1496210 0x5627a1498630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f9567fff700 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a1496210 0x5627a1498630 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c698d5be9d91a0c4 server_cookie=8b339640a2ceca79 in_seq=0 out_seq=0 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f9547fff700 10 client.17079 ms_handle_connect on v2:172.21.15.190:6836/3737066182 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f9547fff700 1 -- 172.21.15.57:0/1413966436 <== mds.2 v2:172.21.15.190:6836/3737066182 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627a1372b30 con 0x5627a1496210 2023-10-06T22:25:06.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f9547fff700 10 client.17079 handle_command_reply: tid=0 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f958248cb80 1 client.17079 shutdown 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 msgr2=0x5627a1438510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.483+0000 7f958248cb80 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1438510 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f955c009960 tx=0x7f955c007a30 comp rx=0 tx=0).stop 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 shutdown_connections 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:6836/3737066182,v1:172.21.15.190:6837/3737066182] conn(0x5627a1496210 0x5627a1498630 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627a1431890 0x5627a14364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627a14260c0 0x5627a1437fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 --2- 172.21.15.57:0/1413966436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5627a14269e0 0x5627a1438510 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 >> 172.21.15.57:0/1413966436 conn(0x5627a139b260 msgr2=0x5627a142cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f9547fff700 0 client.17079 ms_handle_reset on v2:172.21.15.190:6836/3737066182 2023-10-06T22:25:06.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 shutdown_connections 2023-10-06T22:25:06.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 1 -- 172.21.15.57:0/1413966436 wait complete. 2023-10-06T22:25:06.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f958248cb80 20 client.17079 trim_cache size 0 max 16384 2023-10-06T22:25:06.492 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:25:06.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f95580a15f0 msgr2=0x7f95580a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f956ebff700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f95580a15f0 0x7f95580a3ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9560009230 tx=0x7f9560003820 comp rx=0 tx=0).stop 2023-10-06T22:25:06.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956819dba0 msgr2=0x7f956819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.484+0000 7f956ebff700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956819dba0 0x7f956819e020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f956000ea80 tx=0x7f956000ed90 comp rx=0 tx=0).stop 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 shutdown_connections 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f95580a15f0 0x7f95580a3ab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f956819dba0 0x7f956819e020 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95681010b0 0x7f9568199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 --2- 172.21.15.57:0/3362227950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95681061d0 0x7f9568199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 >> 172.21.15.57:0/3362227950 conn(0x7f95680fc950 msgr2=0x7f95680fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 shutdown_connections 2023-10-06T22:25:06.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.485+0000 7f956ebff700 1 -- 172.21.15.57:0/3362227950 wait complete. 2023-10-06T22:25:06.506 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T22:25:06.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 -- 172.21.15.57:0/3292418900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 msgr2=0x7f44701036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 --2- 172.21.15.57:0/3292418900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f44701036e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f446c009990 tx=0x7f446c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 -- 172.21.15.57:0/3292418900 shutdown_connections 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 --2- 172.21.15.57:0/3292418900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4470108420 0x7f447010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 --2- 172.21.15.57:0/3292418900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f44701036e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 --2- 172.21.15.57:0/3292418900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4470103cb0 0x7f4470107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 -- 172.21.15.57:0/3292418900 >> 172.21.15.57:0/3292418900 conn(0x7f44700feb40 msgr2=0x7f4470100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 -- 172.21.15.57:0/3292418900 shutdown_connections 2023-10-06T22:25:06.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.714+0000 7f44781ff700 1 -- 172.21.15.57:0/3292418900 wait complete. 2023-10-06T22:25:06.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.715+0000 7f44781ff700 1 Processor -- start 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.715+0000 7f44781ff700 1 -- start start 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f44781ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f4470199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f44781ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4470108420 0x7f44701996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f44781ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447019db20 0x7f447019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f44781ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447019e5a0 con 0x7f4470108420 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f44781ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447019e710 con 0x7f4470103300 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f44781ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447019e8d0 con 0x7f447019db20 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f4470199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f4470199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38442/0 (socket says 172.21.15.57:38442) 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f447679c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447019db20 0x7f447019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 -- 172.21.15.57:0/3009563162 learned_addr learned my addr 172.21.15.57:0/3009563162 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f447579a700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4470108420 0x7f44701996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 -- 172.21.15.57:0/3009563162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447019db20 msgr2=0x7f447019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447019db20 0x7f447019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 -- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4470108420 msgr2=0x7f44701996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4470108420 0x7f44701996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.716+0000 7f4475f9b700 1 -- 172.21.15.57:0/3009563162 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0095f0 con 0x7f4470103300 2023-10-06T22:25:06.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.717+0000 7f4475f9b700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f4470199180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f446c005910 tx=0x7f446c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.717+0000 7f4466ffd700 1 -- 172.21.15.57:0/3009563162 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c01d070 con 0x7f4470103300 2023-10-06T22:25:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.717+0000 7f4466ffd700 1 -- 172.21.15.57:0/3009563162 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f446c022470 con 0x7f4470103300 2023-10-06T22:25:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.717+0000 7f4466ffd700 1 -- 172.21.15.57:0/3009563162 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c017650 con 0x7f4470103300 2023-10-06T22:25:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.717+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44701a5dd0 con 0x7f4470103300 2023-10-06T22:25:06.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.717+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44701a61a0 con 0x7f4470103300 2023-10-06T22:25:06.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.720+0000 7f4466ffd700 1 -- 172.21.15.57:0/3009563162 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f446c00fad0 con 0x7f4470103300 2023-10-06T22:25:06.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.721+0000 7f448bb2cb80 1 Processor -- start 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f448bb2cb80 1 -- start start 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd84fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8508350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd850ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8466460 con 0x561bd8508890 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8457c20 con 0x561bd84fd0c0 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8455540 con 0x561bd84fd9e0 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f4466ffd700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f445c0a15f0 0x7f445c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8508350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8508350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56520/0 (socket says 172.21.15.57:56520) 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f4475f9b700 1 -- 172.21.15.57:0/1250368061 learned_addr learned my addr 172.21.15.57:0/1250368061 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f4466ffd700 1 -- 172.21.15.57:0/3009563162 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f446c0ce5a0 con 0x7f4470103300 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.722+0000 7f447579a700 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd850ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f447679c700 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd84fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f4475f9b700 1 -- 172.21.15.57:0/1250368061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 msgr2=0x561bd84fd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f4475f9b700 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd84fd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f4475f9b700 1 -- 172.21.15.57:0/1250368061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 msgr2=0x561bd850ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f4475f9b700 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd850ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f4475f9b700 1 -- 172.21.15.57:0/1250368061 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd827bde0 con 0x561bd84fd9e0 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f447579a700 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd850ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f447679c700 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd84fd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f447679c700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f445c0a15f0 0x7f445c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f4475f9b700 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8508350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f446c006010 tx=0x7f446c066930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9061920429eea0cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f445bfff700 1 -- 172.21.15.57:0/1250368061 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c086040 con 0x561bd84fd9e0 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f445bfff700 1 -- 172.21.15.57:0/1250368061 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f446c09bcd0 con 0x561bd84fd9e0 2023-10-06T22:25:06.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.723+0000 7f445bfff700 1 -- 172.21.15.57:0/1250368061 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0225e0 con 0x561bd84fd9e0 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 -- 172.21.15.57:0/1250368061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 msgr2=0x561bd8508350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8508350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f446c006010 tx=0x7f446c066930 comp rx=0 tx=0).stop 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f447679c700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f445c0a15f0 0x7f445c0a3ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4468003ef0 tx=0x7f4468004b40 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 -- 172.21.15.57:0/1250368061 shutdown_connections 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd850ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd84fd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/1250368061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8508350 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 -- 172.21.15.57:0/1250368061 >> 172.21.15.57:0/1250368061 conn(0x561bd8472260 msgr2=0x561bd84f9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 -- 172.21.15.57:0/1250368061 shutdown_connections 2023-10-06T22:25:06.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.724+0000 7f448bb2cb80 1 -- 172.21.15.57:0/1250368061 wait complete. 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.725+0000 7f448bb2cb80 1 -- start start 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.725+0000 7f448bb2cb80 1 Processor -- start 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.725+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd8508c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.725+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd85091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.725+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd8513cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.725+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8450e30 con 0x561bd8508890 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd83ed710 con 0x561bd84fd0c0 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd83ed3b0 con 0x561bd84fd9e0 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f448bb2cb80 10 client.0 _ll_register_callbacks cb 0x561bd824a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f447679c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd8513cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f447579a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd85091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd8508c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f447579a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd85091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56534/0 (socket says 172.21.15.57:56534) 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f447679c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd8513cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43070/0 (socket says 172.21.15.57:43070) 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f447579a700 1 -- 172.21.15.57:0/178398597 learned_addr learned my addr 172.21.15.57:0/178398597 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f4475f9b700 1 -- 172.21.15.57:0/178398597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 msgr2=0x561bd85091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f4475f9b700 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd85091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f4475f9b700 1 -- 172.21.15.57:0/178398597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 msgr2=0x561bd8513cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f4475f9b700 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd8513cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f4475f9b700 1 -- 172.21.15.57:0/178398597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44701a61a0 con 0x561bd84fd0c0 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f447579a700 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd85091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.726+0000 7f4475f9b700 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd8508c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f446c09b560 tx=0x7f446c09b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=405e8c267d0d3d9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f44597fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:25:06.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f44597fa700 1 -- 172.21.15.57:0/178398597 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c09b5d0 con 0x561bd84fd0c0 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f44597fa700 1 -- 172.21.15.57:0/178398597 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f446c09f720 con 0x561bd84fd0c0 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bd84f0f10 con 0x561bd84fd0c0 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f44597fa700 1 -- 172.21.15.57:0/178398597 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f446c066e40 con 0x561bd84fd0c0 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 10 client.27104 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x561bd8264c60 con 0x561bd84fd0c0 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f44597fa700 1 -- 172.21.15.57:0/178398597 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bd84f0f10 con 0x561bd84fd0c0 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f44597fa700 1 -- 172.21.15.57:0/178398597 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f446c09aac0 con 0x561bd84fd0c0 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 10 client.27104 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 10 client.27104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 20 client.27104 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd8517a70 0x561bd8519e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 4 client.27104 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.727+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bd8449e10 con 0x561bd8517a70 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.728+0000 7f447579a700 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd8517a70 0x561bd8519e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.728+0000 7f447579a700 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd8517a70 0x561bd8519e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7542da789b2090db server_cookie=df4d9c54c372251d in_seq=0 out_seq=0 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.728+0000 7f44597fa700 10 client.27104 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.729+0000 7f44597fa700 1 -- 172.21.15.57:0/178398597 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561bd8449e10 con 0x561bd8517a70 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.729+0000 7f44597fa700 10 client.27104 handle_command_reply: tid=0 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.729+0000 7f448bb2cb80 1 client.27104 shutdown 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.729+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 msgr2=0x561bd8508c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.729+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd8508c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f446c09b560 tx=0x7f446c09b4d0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.729+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 shutdown_connections 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.729+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd8517a70 0x561bd8519e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd8508890 0x561bd8513cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd84fd0c0 0x561bd8508c70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/178398597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd85091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 >> 172.21.15.57:0/178398597 conn(0x561bd8472260 msgr2=0x561bd84f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f44597fa700 0 client.27104 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:25:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 shutdown_connections 2023-10-06T22:25:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f448bb2cb80 1 -- 172.21.15.57:0/178398597 wait complete. 2023-10-06T22:25:06.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.730+0000 7f448bb2cb80 20 client.27104 trim_cache size 0 max 16384 2023-10-06T22:25:06.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 Processor -- start 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 -- start start 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd8506d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8502da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd85032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd82d1920 con 0x561bd84fd0c0 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8449e10 con 0x561bd8508890 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8509520 con 0x561bd84fd9e0 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd8506d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd8506d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43076/0 (socket says 172.21.15.57:43076) 2023-10-06T22:25:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.744+0000 7f447579a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8502da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f4475f9b700 1 -- 172.21.15.57:0/3885385618 learned_addr learned my addr 172.21.15.57:0/3885385618 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f447679c700 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd85032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f447579a700 1 -- 172.21.15.57:0/3885385618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 msgr2=0x561bd85032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f447579a700 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd85032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f447579a700 1 -- 172.21.15.57:0/3885385618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 msgr2=0x561bd8506d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f447579a700 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd8506d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f447579a700 1 -- 172.21.15.57:0/3885385618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd8508330 con 0x561bd84fd9e0 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f4475f9b700 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd8506d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f447579a700 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8502da0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x561bd851a700 tx=0x7f4460009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76d6556f07ae413a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f4464ff9700 1 -- 172.21.15.57:0/3885385618 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460017040 con 0x561bd84fd9e0 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f4464ff9700 1 -- 172.21.15.57:0/3885385618 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4460025070 con 0x561bd84fd9e0 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.745+0000 7f4464ff9700 1 -- 172.21.15.57:0/3885385618 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460012af0 con 0x561bd84fd9e0 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 -- 172.21.15.57:0/3885385618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 msgr2=0x561bd8502da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8502da0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x561bd851a700 tx=0x7f4460009260 comp rx=0 tx=0).stop 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 -- 172.21.15.57:0/3885385618 shutdown_connections 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd85032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd8506d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/3885385618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd8502da0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 -- 172.21.15.57:0/3885385618 >> 172.21.15.57:0/3885385618 conn(0x561bd8472260 msgr2=0x561bd8514290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 -- 172.21.15.57:0/3885385618 shutdown_connections 2023-10-06T22:25:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.746+0000 7f448bb2cb80 1 -- 172.21.15.57:0/3885385618 wait complete. 2023-10-06T22:25:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.747+0000 7f448bb2cb80 1 -- start start 2023-10-06T22:25:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.747+0000 7f448bb2cb80 1 Processor -- start 2023-10-06T22:25:06.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd850bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd850c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f448bb2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd8564830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8564d70 con 0x561bd84fd0c0 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8564eb0 con 0x561bd8508890 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f448bb2cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd8565070 con 0x561bd84fd9e0 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f448bb2cb80 10 client.0 _ll_register_callbacks cb 0x561bd824a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd850bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd850bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43086/0 (socket says 172.21.15.57:43086) 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 -- 172.21.15.57:0/969805843 learned_addr learned my addr 172.21.15.57:0/969805843 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f447579a700 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd850c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f447679c700 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd8564830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 -- 172.21.15.57:0/969805843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 msgr2=0x561bd850c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd850c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 -- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 msgr2=0x561bd8564830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:06.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd8564830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.748+0000 7f4475f9b700 1 -- 172.21.15.57:0/969805843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bd8264c60 con 0x561bd84fd0c0 2023-10-06T22:25:06.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f4475f9b700 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd850bcb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f446c09ae20 tx=0x7f446c022ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89948ff2ca9d0c43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f44597fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f44597fa700 1 -- 172.21.15.57:0/969805843 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c09f720 con 0x561bd84fd0c0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f446c09bcd0 con 0x561bd84fd0c0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f44597fa700 1 -- 172.21.15.57:0/969805843 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f446c09b5d0 con 0x561bd84fd0c0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f44597fa700 1 -- 172.21.15.57:0/969805843 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f446c0a38d0 con 0x561bd84fd0c0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f448bb2cb80 10 client.17097 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x561bd82a2250 con 0x561bd84fd0c0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f44597fa700 1 -- 172.21.15.57:0/969805843 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f446c09bcd0 con 0x561bd84fd0c0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.749+0000 7f44597fa700 1 -- 172.21.15.57:0/969805843 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f446c0a3560 con 0x561bd84fd0c0 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.750+0000 7f448bb2cb80 10 client.17097 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:06.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.750+0000 7f448bb2cb80 10 client.17097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.750+0000 7f448bb2cb80 20 client.17097 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.750+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd856c480 0x561bd856e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.750+0000 7f448bb2cb80 4 client.17097 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.750+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 --> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561bd8449b30 con 0x561bd856c480 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.750+0000 7f447579a700 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd856c480 0x561bd856e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f447579a700 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd856c480 0x561bd856e8a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60e3a7848088e56e server_cookie=ac5fb2994c10a93a in_seq=0 out_seq=0 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f44597fa700 10 client.17097 ms_handle_connect on v2:172.21.15.156:6838/1657784543 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f44597fa700 1 -- 172.21.15.57:0/969805843 <== mds.1 v2:172.21.15.156:6838/1657784543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561bd8449b30 con 0x561bd856c480 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f44597fa700 10 client.17097 handle_command_reply: tid=0 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 client.17097 shutdown 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 msgr2=0x561bd850bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd850bcb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f446c09ae20 tx=0x7f446c022ac0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 shutdown_connections 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:6838/1657784543,v1:172.21.15.156:6839/1657784543] conn(0x561bd856c480 0x561bd856e8a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561bd8508890 0x561bd8564830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd84fd0c0 0x561bd850bcb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f44597fa700 0 client.17097 ms_handle_reset on v2:172.21.15.156:6838/1657784543 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 --2- 172.21.15.57:0/969805843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bd84fd9e0 0x561bd850c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 >> 172.21.15.57:0/969805843 conn(0x561bd8472260 msgr2=0x561bd84f8660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 shutdown_connections 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.751+0000 7f448bb2cb80 1 -- 172.21.15.57:0/969805843 wait complete. 2023-10-06T22:25:06.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f448bb2cb80 20 client.17097 trim_cache size 0 max 16384 2023-10-06T22:25:06.760 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:25:06.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f445c0a15f0 msgr2=0x7f445c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f445c0a15f0 0x7f445c0a3ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4468003ef0 tx=0x7f4468004b40 comp rx=0 tx=0).stop 2023-10-06T22:25:06.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 msgr2=0x7f4470199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f4470199180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f446c005910 tx=0x7f446c004730 comp rx=0 tx=0).stop 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 shutdown_connections 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f445c0a15f0 0x7f445c0a3ab0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447019db20 0x7f447019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4470103300 0x7f4470199180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 --2- 172.21.15.57:0/3009563162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4470108420 0x7f44701996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 >> 172.21.15.57:0/3009563162 conn(0x7f44700feb40 msgr2=0x7f4470106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 shutdown_connections 2023-10-06T22:25:06.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:06.752+0000 7f44781ff700 1 -- 172.21.15.57:0/3009563162 wait complete. 2023-10-06T22:25:06.773 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T22:25:07.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 -- 172.21.15.57:0/1471997347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 msgr2=0x7ff80810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/1471997347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff80810dd10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff80000b0a0 tx=0x7ff80000b3b0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 -- 172.21.15.57:0/1471997347 shutdown_connections 2023-10-06T22:25:07.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/1471997347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff80810dd10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/1471997347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff808101070 0x7ff808101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/1471997347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff808101a20 0x7ff808105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 -- 172.21.15.57:0/1471997347 >> 172.21.15.57:0/1471997347 conn(0x7ff8080fc910 msgr2=0x7ff8080fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:07.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 -- 172.21.15.57:0/1471997347 shutdown_connections 2023-10-06T22:25:07.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.032+0000 7ff80d7ff700 1 -- 172.21.15.57:0/1471997347 wait complete. 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 Processor -- start 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 -- start start 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff808101070 0x7ff8080731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff8080736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff808073c30 0x7ff8081a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081a5950 con 0x7ff808101070 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081a5ac0 con 0x7ff808073c30 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.033+0000 7ff80d7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081a5c80 con 0x7ff808106190 2023-10-06T22:25:07.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff8080736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff806ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff808101070 0x7ff8080731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff8080736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56568/0 (socket says 172.21.15.57:56568) 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 -- 172.21.15.57:0/2397745371 learned_addr learned my addr 172.21.15.57:0/2397745371 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8077fe700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff808073c30 0x7ff8081a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 -- 172.21.15.57:0/2397745371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff808073c30 msgr2=0x7ff8081a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff808073c30 0x7ff8081a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 -- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff808101070 msgr2=0x7ff8080731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff808101070 0x7ff8080731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 -- 172.21.15.57:0/2397745371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f8009520 con 0x7ff808106190 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8077fe700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff808073c30 0x7ff8081a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff806ffd700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff808101070 0x7ff8080731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:07.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff8067fc700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff8080736f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e970 tx=0x7ff7fc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:07.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.034+0000 7ff7effff700 1 -- 172.21.15.57:0/2397745371 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc009790 con 0x7ff808106190 2023-10-06T22:25:07.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.035+0000 7ff7effff700 1 -- 172.21.15.57:0/2397745371 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7fc00eec0 con 0x7ff808106190 2023-10-06T22:25:07.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.035+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff800009d00 con 0x7ff808106190 2023-10-06T22:25:07.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.035+0000 7ff7effff700 1 -- 172.21.15.57:0/2397745371 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc005590 con 0x7ff808106190 2023-10-06T22:25:07.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.035+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8081a6280 con 0x7ff808106190 2023-10-06T22:25:07.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.038+0000 7ff7effff700 1 -- 172.21.15.57:0/2397745371 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff7fc00cbd0 con 0x7ff808106190 2023-10-06T22:25:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.039+0000 7ff821202b80 1 Processor -- start 2023-10-06T22:25:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.039+0000 7ff821202b80 1 -- start start 2023-10-06T22:25:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.039+0000 7ff821202b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113310c0 0x55bf113314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff821202b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113319e0 0x55bf1133c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff821202b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf1133ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff821202b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1129a460 con 0x55bf113319e0 2023-10-06T22:25:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff821202b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1128bc20 con 0x55bf113310c0 2023-10-06T22:25:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff821202b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11289540 con 0x55bf1133c890 2023-10-06T22:25:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113310c0 0x55bf113314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff806ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113319e0 0x55bf1133c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113310c0 0x55bf113314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:38522/0 (socket says 172.21.15.57:38522) 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 -- 172.21.15.57:0/2672246831 learned_addr learned my addr 172.21.15.57:0/2672246831 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8067fc700 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf1133ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 -- 172.21.15.57:0/2672246831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 msgr2=0x55bf1133ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf1133ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 -- 172.21.15.57:0/2672246831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113319e0 msgr2=0x55bf1133c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113319e0 0x55bf1133c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 -- 172.21.15.57:0/2672246831 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf110afde0 con 0x55bf113310c0 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff7effff700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff7f00a15f0 0x7ff7f00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.040+0000 7ff8077fe700 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113310c0 0x55bf113314a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff800015000 tx=0x7ff8000092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f67417cea62b04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.041+0000 7ff7effff700 1 -- 172.21.15.57:0/2397745371 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ff7fc014070 con 0x7ff808106190 2023-10-06T22:25:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.041+0000 7ff806ffd700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff7f00a15f0 0x7ff7f00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.041+0000 7ff7ed7fa700 1 -- 172.21.15.57:0/2672246831 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000e050 con 0x55bf113310c0 2023-10-06T22:25:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.041+0000 7ff7ed7fa700 1 -- 172.21.15.57:0/2672246831 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff800007dc0 con 0x55bf113310c0 2023-10-06T22:25:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.041+0000 7ff7ed7fa700 1 -- 172.21.15.57:0/2672246831 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800004e70 con 0x55bf113310c0 2023-10-06T22:25:07.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 -- 172.21.15.57:0/2672246831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113310c0 msgr2=0x55bf113314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff806ffd700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff7f00a15f0 0x7ff7f00a3ab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8012800 tx=0x7ff7f8009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:07.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113310c0 0x55bf113314a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff800015000 tx=0x7ff8000092e0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 -- 172.21.15.57:0/2672246831 shutdown_connections 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf1133ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113310c0 0x55bf113314a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 --2- 172.21.15.57:0/2672246831 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113319e0 0x55bf1133c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 -- 172.21.15.57:0/2672246831 >> 172.21.15.57:0/2672246831 conn(0x55bf112a6260 msgr2=0x55bf1132d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 -- 172.21.15.57:0/2672246831 shutdown_connections 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 -- 172.21.15.57:0/2672246831 wait complete. 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.042+0000 7ff821202b80 1 -- start start 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.043+0000 7ff821202b80 1 Processor -- start 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff821202b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf1133c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff821202b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113319e0 0x55bf113394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff821202b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf1133c890 0x55bf11339ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff821202b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11284e30 con 0x55bf113310c0 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff821202b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11221710 con 0x55bf1133c890 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff821202b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf112213b0 con 0x55bf113319e0 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff821202b80 10 client.0 _ll_register_callbacks cb 0x55bf1107e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf1133c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff8067fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113319e0 0x55bf113394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf1133c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43120/0 (socket says 172.21.15.57:43120) 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 -- 172.21.15.57:0/2981419569 learned_addr learned my addr 172.21.15.57:0/2981419569 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff8077fe700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf1133c890 0x55bf11339ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 -- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113319e0 msgr2=0x55bf113394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113319e0 0x55bf113394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 -- 172.21.15.57:0/2981419569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf1133c890 msgr2=0x55bf11339ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf1133c890 0x55bf11339ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff806ffd700 1 -- 172.21.15.57:0/2981419569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80810e580 con 0x55bf113310c0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff8067fc700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113319e0 0x55bf113394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.044+0000 7ff8077fe700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf1133c890 0x55bf11339ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff806ffd700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf1133c290 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800fc10 tx=0x7ff7f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb8a09e2592ff4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff7eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff7eaff5700 1 -- 172.21.15.57:0/2981419569 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8022070 con 0x55bf113310c0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf11324f10 con 0x55bf113310c0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff7eaff5700 1 -- 172.21.15.57:0/2981419569 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ff7f801d820 con 0x55bf113310c0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff7eaff5700 1 -- 172.21.15.57:0/2981419569 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f8017070 con 0x55bf113310c0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff821202b80 10 client.17109 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.045+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55bf11098c60 con 0x55bf113310c0 2023-10-06T22:25:07.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff7eaff5700 1 -- 172.21.15.57:0/2981419569 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f8035020 con 0x55bf113310c0 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff7eaff5700 1 -- 172.21.15.57:0/2981419569 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7ff7f801cb60 con 0x55bf113310c0 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff821202b80 10 client.17109 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff821202b80 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff821202b80 20 client.17109 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff821202b80 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf1134c300 0x55bf1134e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff821202b80 4 client.17109 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf1127de10 con 0x55bf1134c300 2023-10-06T22:25:07.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.046+0000 7ff8067fc700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf1134c300 0x55bf1134e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.047+0000 7ff8067fc700 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf1134c300 0x55bf1134e720 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8770d96ce5e3f6fb server_cookie=beca4b2baaf40f9d in_seq=0 out_seq=0 2023-10-06T22:25:07.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.047+0000 7ff7eaff5700 10 client.17109 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:07.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff7eaff5700 1 -- 172.21.15.57:0/2981419569 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bf1127de10 con 0x55bf1134c300 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff7eaff5700 10 client.17109 handle_command_reply: tid=0 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 client.17109 shutdown 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 msgr2=0x55bf1133c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf1133c290 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800fc10 tx=0x7ff7f80047c0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 shutdown_connections 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf1134c300 0x55bf1134e720 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf1133c890 0x55bf11339ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf1133c290 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 --2- 172.21.15.57:0/2981419569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113319e0 0x55bf113394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff7eaff5700 0 client.17109 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.048+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 >> 172.21.15.57:0/2981419569 conn(0x55bf112a6260 msgr2=0x55bf1132cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:07.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.049+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 shutdown_connections 2023-10-06T22:25:07.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.049+0000 7ff821202b80 1 -- 172.21.15.57:0/2981419569 wait complete. 2023-10-06T22:25:07.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.049+0000 7ff821202b80 20 client.17109 trim_cache size 0 max 16384 2023-10-06T22:25:07.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 Processor -- start 2023-10-06T22:25:07.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 -- start start 2023-10-06T22:25:07.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf11343370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf11338d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11105920 con 0x55bf113310c0 2023-10-06T22:25:07.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1133a0e0 con 0x55bf113319e0 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff821202b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf113438e0 con 0x55bf1133c890 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff8077fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf11338d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff806ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf11343370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff8077fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf11338d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56586/0 (socket says 172.21.15.57:56586) 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.064+0000 7ff8077fe700 1 -- 172.21.15.57:0/1072002722 learned_addr learned my addr 172.21.15.57:0/1072002722 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff8067fc700 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff8077fe700 1 -- 172.21.15.57:0/1072002722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 msgr2=0x55bf1133bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff8077fe700 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff8077fe700 1 -- 172.21.15.57:0/1072002722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 msgr2=0x55bf11343370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff8077fe700 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf11343370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff8077fe700 1 -- 172.21.15.57:0/1072002722 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf110d6250 con 0x55bf1133c890 2023-10-06T22:25:07.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff806ffd700 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf11343370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:07.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff8077fe700 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf11338d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff800022000 tx=0x7ff800009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e96fc821af026e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff7edffb700 1 -- 172.21.15.57:0/1072002722 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000bc50 con 0x55bf1133c890 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff7edffb700 1 -- 172.21.15.57:0/1072002722 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff80000bdb0 con 0x55bf1133c890 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.065+0000 7ff7edffb700 1 -- 172.21.15.57:0/1072002722 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800005320 con 0x55bf1133c890 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 -- 172.21.15.57:0/1072002722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 msgr2=0x55bf11338d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf11338d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff800022000 tx=0x7ff800009570 comp rx=0 tx=0).stop 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 -- 172.21.15.57:0/1072002722 shutdown_connections 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf1133c890 0x55bf11338d40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf113310c0 0x55bf11343370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 --2- 172.21.15.57:0/1072002722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 -- 172.21.15.57:0/1072002722 >> 172.21.15.57:0/1072002722 conn(0x55bf112a6260 msgr2=0x55bf113347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:07.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 -- 172.21.15.57:0/1072002722 shutdown_connections 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.066+0000 7ff821202b80 1 -- 172.21.15.57:0/1072002722 wait complete. 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 -- start start 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 Processor -- start 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113310c0 0x55bf1133f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf1133c890 0x55bf1133fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1139f100 con 0x55bf1133c890 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1139f240 con 0x55bf113319e0 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.067+0000 7ff821202b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf1139f380 con 0x55bf113310c0 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff821202b80 10 client.0 _ll_register_callbacks cb 0x7ff7f800f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113310c0 0x55bf1133f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113310c0 0x55bf1133f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:56594/0 (socket says 172.21.15.57:56594) 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 -- 172.21.15.57:0/1325278091 learned_addr learned my addr 172.21.15.57:0/1325278091 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff8077fe700 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf1133c890 0x55bf1133fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff8067fc700 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 -- 172.21.15.57:0/1325278091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 msgr2=0x55bf1133f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 -- 172.21.15.57:0/1325278091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf1133c890 msgr2=0x55bf1133fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf1133c890 0x55bf1133fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 -- 172.21.15.57:0/1325278091 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf110afde0 con 0x55bf113310c0 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff806ffd700 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113310c0 0x55bf1133f350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800ffd0 tx=0x7ff7f8012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf6ff672e1b4f00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.068+0000 7ff7eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff7eaff5700 1 -- 172.21.15.57:0/1325278091 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8022040 con 0x55bf113310c0 2023-10-06T22:25:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff800007dc0 con 0x55bf113310c0 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff7eaff5700 1 -- 172.21.15.57:0/1325278091 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ff7f8036080 con 0x55bf113310c0 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff7eaff5700 1 -- 172.21.15.57:0/1325278091 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f8030ac0 con 0x55bf113310c0 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 10 client.27154 fetch_fsmap learned FSMap version 58 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55bf110ae2b0 con 0x55bf113310c0 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff7eaff5700 1 -- 172.21.15.57:0/1325278091 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff800007dc0 con 0x55bf113310c0 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff7eaff5700 1 -- 172.21.15.57:0/1325278091 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 58) v1 ==== 7962+0+0 (secure 0 0 0) 0x7ff7f801c4c0 con 0x55bf113310c0 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 10 client.27154 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 10 client.27154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 20 client.27154 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf113a0b90 0x55bf113a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 4 client.27154 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.069+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bf1127db30 con 0x55bf113a0b90 2023-10-06T22:25:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.070+0000 7ff8067fc700 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf113a0b90 0x55bf113a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.070+0000 7ff8067fc700 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf113a0b90 0x55bf113a2fb0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583802e2a9ac5fb5 server_cookie=b3777aea84724c90 in_seq=0 out_seq=0 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.070+0000 7ff7eaff5700 10 client.27154 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff7eaff5700 1 -- 172.21.15.57:0/1325278091 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bf1127db30 con 0x55bf113a0b90 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff7eaff5700 10 client.27154 handle_command_reply: tid=0 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 client.27154 shutdown 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113310c0 msgr2=0x55bf1133f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113310c0 0x55bf1133f350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800ffd0 tx=0x7ff7f8012680 comp rx=0 tx=0).stop 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 shutdown_connections 2023-10-06T22:25:07.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55bf113a0b90 0x55bf113a2fb0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf1133c890 0x55bf1133fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf113310c0 0x55bf1133f350 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff7eaff5700 0 client.27154 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 --2- 172.21.15.57:0/1325278091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf113319e0 0x55bf1133f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 >> 172.21.15.57:0/1325278091 conn(0x55bf112a6260 msgr2=0x55bf11333000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 shutdown_connections 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 1 -- 172.21.15.57:0/1325278091 wait complete. 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.072+0000 7ff821202b80 20 client.27154 trim_cache size 0 max 16384 2023-10-06T22:25:07.078 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-06T22:25:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff7f00a15f0 msgr2=0x7ff7f00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff7f00a15f0 0x7ff7f00a3ab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8012800 tx=0x7ff7f8009260 comp rx=0 tx=0).stop 2023-10-06T22:25:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 msgr2=0x7ff8080736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff8080736f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e970 tx=0x7ff7fc00ec80 comp rx=0 tx=0).stop 2023-10-06T22:25:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 shutdown_connections 2023-10-06T22:25:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff7f00a15f0 0x7ff7f00a3ab0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff808073c30 0x7ff8081a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff808101070 0x7ff8080731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 --2- 172.21.15.57:0/2397745371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff808106190 0x7ff8080736f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:07.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 >> 172.21.15.57:0/2397745371 conn(0x7ff8080fc910 msgr2=0x7ff8080fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:07.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 shutdown_connections 2023-10-06T22:25:07.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:07.073+0000 7ff80d7ff700 1 -- 172.21.15.57:0/2397745371 wait complete. 2023-10-06T22:25:07.094 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 88351292, 'ino': 1099511628711, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-33.jpg'}, {'damage_type': 'backtrace', 'id': 167578459, 'ino': 1099511628766, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 176735633, 'ino': 1099511628891, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-42.xml'}, {'damage_type': 'backtrace', 'id': 196937144, 'ino': 1099511628946, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-6.xml'}, {'damage_type': 'backtrace', 'id': 219643474, 'ino': 1099511628756, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 265176836, 'ino': 1099511628933, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-19.xml'}, {'damage_type': 'backtrace', 'id': 291956509, 'ino': 1099511633886, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-55.xml'}, {'damage_type': 'backtrace', 'id': 293788785, 'ino': 1099511628764, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 349792818, 'ino': 1099511628749, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 370026347, 'ino': 1099511631259, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-69.xml'}, {'damage_type': 'backtrace', 'id': 408144989, 'ino': 1099511628921, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-23.xml'}, {'damage_type': 'backtrace', 'id': 414889664, 'ino': 2199023261418, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-29'}, {'damage_type': 'backtrace', 'id': 434022498, 'ino': 1099511629226, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-73.xml'}, {'damage_type': 'backtrace', 'id': 484947972, 'ino': 1099511628652, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-49.jpg'}, {'damage_type': 'backtrace', 'id': 486007126, 'ino': 1099511628892, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-41.xml'}, {'damage_type': 'backtrace', 'id': 502811322, 'ino': 1099511628704, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-35.jpg'}, {'damage_type': 'backtrace', 'id': 531962698, 'ino': 1099511628721, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 549721860, 'ino': 1099511628753, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 566944881, 'ino': 1099511628651, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-32.xml'}, {'damage_type': 'backtrace', 'id': 602925643, 'ino': 1099511628947, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-5.xml'}, {'damage_type': 'backtrace', 'id': 612988522, 'ino': 1099511628945, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-7.xml'}, {'damage_type': 'backtrace', 'id': 818880005, 'ino': 1099511634190, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-5.xml'}, {'damage_type': 'backtrace', 'id': 821284007, 'ino': 1099511628713, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-32.jpg'}, {'damage_type': 'backtrace', 'id': 830024297, 'ino': 1099511628725, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 840360994, 'ino': 1099511628939, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-13.xml'}, {'damage_type': 'backtrace', 'id': 910737369, 'ino': 1099511628640, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-52.jpg'}, {'damage_type': 'backtrace', 'id': 980390732, 'ino': 1099511628668, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-43.jpg'}, {'damage_type': 'backtrace', 'id': 1016860047, 'ino': 1099511628732, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 1061577863, 'ino': 1099511628936, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-16.xml'}, {'damage_type': 'backtrace', 'id': 1066223658, 'ino': 1099511628683, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-40.jpg'}, {'damage_type': 'backtrace', 'id': 1213486864, 'ino': 1099511635225, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-16.xml'}, {'damage_type': 'backtrace', 'id': 1292070347, 'ino': 1099511628894, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-39.xml'}, {'damage_type': 'backtrace', 'id': 1359819109, 'ino': 1099511628763, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 1414374689, 'ino': 1099511628889, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-44.xml'}, {'damage_type': 'backtrace', 'id': 1481516929, 'ino': 1099511629348, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-59.jpg'}, {'damage_type': 'backtrace', 'id': 1506011126, 'ino': 1099511628902, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-33.xml'}, {'damage_type': 'backtrace', 'id': 1528919897, 'ino': 1099511628815, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-97.xml'}, {'damage_type': 'backtrace', 'id': 1551034609, 'ino': 1099511628896, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-37.xml'}, {'damage_type': 'backtrace', 'id': 1580449016, 'ino': 1099511628943, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-9.xml'}, {'damage_type': 'backtrace', 'id': 1655569438, 'ino': 1099511628718, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-29.jpg'}, {'damage_type': 'backtrace', 'id': 1669385892, 'ino': 1099511629249, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-23.xml'}, {'damage_type': 'backtrace', 'id': 1689057025, 'ino': 1099511628917, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-25.xml'}, {'damage_type': 'backtrace', 'id': 1732660726, 'ino': 1099511628949, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-3.xml'}, {'damage_type': 'backtrace', 'id': 1774782357, 'ino': 1099511628707, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-34.jpg'}, {'damage_type': 'backtrace', 'id': 1775769023, 'ino': 1099511628716, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-30.jpg'}, {'damage_type': 'backtrace', 'id': 1782606662, 'ino': 1099511628679, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-41.jpg'}, {'damage_type': 'backtrace', 'id': 1795987958, 'ino': 1099511628918, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-24.xml'}, {'damage_type': 'backtrace', 'id': 1828692975, 'ino': 1099511628666, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-44.jpg'}, {'damage_type': 'backtrace', 'id': 1832269330, 'ino': 1099511628940, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-12.xml'}, {'damage_type': 'backtrace', 'id': 1874300046, 'ino': 1099511628714, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-31.jpg'}, {'damage_type': 'backtrace', 'id': 1893176122, 'ino': 1099511628932, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-20.xml'}, {'damage_type': 'backtrace', 'id': 1971576019, 'ino': 1099511628951, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-1.xml'}, {'damage_type': 'backtrace', 'id': 1981320787, 'ino': 1099511628727, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 2081386983, 'ino': 1099511628647, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-51.jpg'}, {'damage_type': 'backtrace', 'id': 2156522129, 'ino': 1099511628758, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 2259722656, 'ino': 1099511628723, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 2266873419, 'ino': 1099511628948, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-4.xml'}, {'damage_type': 'backtrace', 'id': 2307909573, 'ino': 1099511628742, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 2311000192, 'ino': 1099511628635, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-53.jpg'}, {'damage_type': 'backtrace', 'id': 2311036627, 'ino': 1099511628895, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-38.xml'}, {'damage_type': 'backtrace', 'id': 2314070114, 'ino': 1099511628694, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-37.jpg'}, {'damage_type': 'backtrace', 'id': 2359530033, 'ino': 1099511628935, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-17.xml'}, {'damage_type': 'backtrace', 'id': 2387525055, 'ino': 1099511628928, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-22.xml'}, {'damage_type': 'backtrace', 'id': 2411999478, 'ino': 1099511628893, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-40.xml'}, {'damage_type': 'backtrace', 'id': 2435604883, 'ino': 1099511628950, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-2.xml'}, {'damage_type': 'backtrace', 'id': 2444331978, 'ino': 1099511628698, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-36.jpg'}, {'damage_type': 'backtrace', 'id': 2464837917, 'ino': 1099511628740, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 2486115379, 'ino': 1099511628649, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-50.jpg'}, {'damage_type': 'backtrace', 'id': 2492435275, 'ino': 1099511628750, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 2536779642, 'ino': 1099511628735, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 2620082748, 'ino': 1099511628915, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-26.xml'}, {'damage_type': 'backtrace', 'id': 2685034247, 'ino': 1099511628905, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-31.xml'}, {'damage_type': 'backtrace', 'id': 2733946414, 'ino': 1099511628733, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 2852451909, 'ino': 1099511628761, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 2859144552, 'ino': 1099511628942, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-10.xml'}, {'damage_type': 'backtrace', 'id': 2942141425, 'ino': 1099511634189, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-75.jpg'}, {'damage_type': 'backtrace', 'id': 3054403192, 'ino': 1099511628899, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-35.xml'}, {'damage_type': 'backtrace', 'id': 3114963693, 'ino': 1099511628661, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-46.jpg'}, {'damage_type': 'backtrace', 'id': 3162312234, 'ino': 1099511628913, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-27.xml'}, {'damage_type': 'backtrace', 'id': 3207878774, 'ino': 1099511628907, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-30.xml'}, {'damage_type': 'backtrace', 'id': 3296114892, 'ino': 1099511628752, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 3359903584, 'ino': 1099511628903, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-32.xml'}, {'damage_type': 'backtrace', 'id': 3367843313, 'ino': 1099511628755, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 3393648093, 'ino': 1099511629349, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-46.xml'}, {'damage_type': 'backtrace', 'id': 3426118592, 'ino': 1099511628938, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-14.xml'}, {'damage_type': 'backtrace', 'id': 3441492304, 'ino': 1099511628643, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-77.xml'}, {'damage_type': 'backtrace', 'id': 3448388280, 'ino': 1099511630180, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/article-72.xml'}, {'damage_type': 'backtrace', 'id': 3450381015, 'ino': 1099511628765, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 3469945263, 'ino': 1099511628655, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-48.jpg'}, {'damage_type': 'backtrace', 'id': 3484306937, 'ino': 1099511628737, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 3504930395, 'ino': 1099511628934, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-18.xml'}, {'damage_type': 'backtrace', 'id': 3540137632, 'ino': 1099511628730, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 3566559751, 'ino': 1099511628673, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-42.jpg'}, {'damage_type': 'backtrace', 'id': 3692862045, 'ino': 1099511628659, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-47.jpg'}, {'damage_type': 'backtrace', 'id': 3715006780, 'ino': 1099511628728, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 3721442921, 'ino': 1099511628757, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 3739543453, 'ino': 1099511628937, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-15.xml'}, {'damage_type': 'backtrace', 'id': 3802707189, 'ino': 1099511628944, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-8.xml'}, {'damage_type': 'backtrace', 'id': 3849277082, 'ino': 1099511628811, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-88.jpg'}, {'damage_type': 'backtrace', 'id': 3855485259, 'ino': 1099511628888, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-45.xml'}, {'damage_type': 'backtrace', 'id': 3863049269, 'ino': 1099511628897, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-36.xml'}, {'damage_type': 'backtrace', 'id': 3867894884, 'ino': 1099511628901, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-34.xml'}, {'damage_type': 'backtrace', 'id': 3895753824, 'ino': 1099511628912, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-28.xml'}, {'damage_type': 'backtrace', 'id': 3957736444, 'ino': 1099511628747, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 3997041793, 'ino': 1099511628663, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-45.jpg'}, {'damage_type': 'backtrace', 'id': 4027616028, 'ino': 1099511628754, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 4035095970, 'ino': 1099511628941, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-11.xml'}, {'damage_type': 'backtrace', 'id': 4161509339, 'ino': 1099511628930, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-21.xml'}, {'damage_type': 'backtrace', 'id': 4169361677, 'ino': 1099511628890, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-43.xml'}, {'damage_type': 'backtrace', 'id': 4226131987, 'ino': 1099511628908, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/comment-29.xml'}, {'damage_type': 'backtrace', 'id': 4228047532, 'ino': 1099511628759, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 4228331599, 'ino': 1099511628692, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-38.jpg'}, {'damage_type': 'backtrace', 'id': 4259558712, 'ino': 1099511628686, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-26/picture-39.jpg'}] 2023-10-06T22:25:07.095 ERROR:tasks.fwd_scrub:rank 3 damaged: [{'damage_type': 'backtrace', 'id': 119042157, 'ino': 1099511633218, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-50.xml'}, {'damage_type': 'backtrace', 'id': 859134248, 'ino': 1099511633198, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-69.xml'}, {'damage_type': 'backtrace', 'id': 921649095, 'ino': 1099511633216, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-52.xml'}, {'damage_type': 'backtrace', 'id': 1011791416, 'ino': 1099511633222, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-47.xml'}, {'damage_type': 'backtrace', 'id': 1142437002, 'ino': 1099511633201, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-66.xml'}, {'damage_type': 'backtrace', 'id': 1382557549, 'ino': 1099511633193, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-74.xml'}, {'damage_type': 'backtrace', 'id': 1501651205, 'ino': 1099511633195, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-72.xml'}, {'damage_type': 'backtrace', 'id': 2178890934, 'ino': 1099511633209, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-59.xml'}, {'damage_type': 'backtrace', 'id': 2264349643, 'ino': 1099511633197, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-70.xml'}, {'damage_type': 'backtrace', 'id': 2378627981, 'ino': 1099511633203, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-64.xml'}, {'damage_type': 'backtrace', 'id': 2381279710, 'ino': 1099511633914, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 2576460873, 'ino': 1099511633200, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-67.xml'}, {'damage_type': 'backtrace', 'id': 2607808755, 'ino': 1099511633191, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-76.xml'}, {'damage_type': 'backtrace', 'id': 3101419895, 'ino': 1099511633190, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-77.xml'}, {'damage_type': 'backtrace', 'id': 3644703633, 'ino': 1099511633205, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-62.xml'}, {'damage_type': 'backtrace', 'id': 3725683812, 'ino': 1099511633225, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-44.xml'}, {'damage_type': 'backtrace', 'id': 4050290929, 'ino': 1099511633223, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-46.xml'}, {'damage_type': 'backtrace', 'id': 4099676203, 'ino': 1099511633221, 'path': '/volumes/_nogroup/sv_1/064cd3dd-8455-4835-b2cc-d301e4708adc/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/comment-48.xml'}] 2023-10-06T22:25:07.096 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-10-06T22:25:07.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:07 smithi057 ceph-mon[98492]: pgmap v322: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2023-10-06T22:25:07.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:07 smithi156 ceph-mon[134358]: pgmap v322: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2023-10-06T22:25:07.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:07 smithi190 ceph-mon[134891]: pgmap v322: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2023-10-06T22:25:09.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:09 smithi190 ceph-mon[134891]: pgmap v323: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 218 op/s 2023-10-06T22:25:09.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:09 smithi057 ceph-mon[98492]: pgmap v323: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 218 op/s 2023-10-06T22:25:09.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:09 smithi156 ceph-mon[134358]: pgmap v323: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 218 op/s 2023-10-06T22:25:11.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:11 smithi190 ceph-mon[134891]: pgmap v324: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2023-10-06T22:25:11.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:11 smithi190 ceph-mon[134891]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T22:25:11.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:11 smithi057 ceph-mon[98492]: pgmap v324: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2023-10-06T22:25:11.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:11 smithi057 ceph-mon[98492]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T22:25:11.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:11 smithi156 ceph-mon[134358]: pgmap v324: 97 pgs: 97 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2023-10-06T22:25:11.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:11 smithi156 ceph-mon[134358]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T22:25:13.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:13 smithi057 ceph-mon[98492]: pgmap v326: 97 pgs: 35 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 43 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-10-06T22:25:13.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:13 smithi156 ceph-mon[134358]: pgmap v326: 97 pgs: 35 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 43 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-10-06T22:25:13.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:13 smithi190 ceph-mon[134891]: pgmap v326: 97 pgs: 35 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 43 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-10-06T22:25:15.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:15 smithi057 ceph-mon[98492]: pgmap v327: 97 pgs: 37 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 39 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2023-10-06T22:25:15.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:15 smithi156 ceph-mon[134358]: pgmap v327: 97 pgs: 37 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 39 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2023-10-06T22:25:15.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:15 smithi190 ceph-mon[134891]: pgmap v327: 97 pgs: 37 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 39 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2023-10-06T22:25:16.753 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:16 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Service RestartSec=10s expired, scheduling restart. 2023-10-06T22:25:16.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:16 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service: Scheduled restart job, restart counter is at 2. 2023-10-06T22:25:16.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:16 smithi156 systemd[1]: Stopped Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:25:16.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:16 smithi156 systemd[1]: Starting Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:25:16.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:16 smithi156 podman[177371]: 2023-10-06T22:25:17.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:17 smithi057 ceph-mon[98492]: pgmap v328: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2023-10-06T22:25:17.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:17 smithi057 ceph-mon[98492]: daemon mds.e restarted 2023-10-06T22:25:17.503 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:17 smithi156 bash[177371]: 27ec6b6af460b83151631085c8d6b2440f1e36a1401cee66770dc1081f1749fe 2023-10-06T22:25:17.504 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[177384]: starting mds.e at 2023-10-06T22:25:17.504 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:25:17 smithi156 systemd[1]: Started Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:25:17.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:17 smithi156 ceph-mon[134358]: pgmap v328: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2023-10-06T22:25:17.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:17 smithi156 ceph-mon[134358]: daemon mds.e restarted 2023-10-06T22:25:17.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:17 smithi190 ceph-mon[134891]: pgmap v328: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2023-10-06T22:25:17.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:17 smithi190 ceph-mon[134891]: daemon mds.e restarted 2023-10-06T22:25:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 -- 172.21.15.57:0/4027420850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd811a690 msgr2=0x7f2dd811ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/4027420850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd811a690 0x7f2dd811ca80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009f90 tx=0x7f2dd000b330 comp rx=0 tx=0).stop 2023-10-06T22:25:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 -- 172.21.15.57:0/4027420850 shutdown_connections 2023-10-06T22:25:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/4027420850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd811a690 0x7f2dd811ca80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/4027420850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd810f620 0x7f2dd8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/4027420850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dd810fa90 0x7f2dd81083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 -- 172.21.15.57:0/4027420850 >> 172.21.15.57:0/4027420850 conn(0x7f2dd806cfa0 msgr2=0x7f2dd806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 -- 172.21.15.57:0/4027420850 shutdown_connections 2023-10-06T22:25:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.934+0000 7f2dde3ff700 1 -- 172.21.15.57:0/4027420850 wait complete. 2023-10-06T22:25:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 Processor -- start 2023-10-06T22:25:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 -- start start 2023-10-06T22:25:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd810f620 0x7f2dd8118b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dd810fa90 0x7f2dd8119080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd811a690 0x7f2dd8111a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8112240 con 0x7f2dd810fa90 2023-10-06T22:25:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd81123b0 con 0x7f2dd810f620 2023-10-06T22:25:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.936+0000 7f2dde3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8112570 con 0x7f2dd811a690 2023-10-06T22:25:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.937+0000 7f2ddc99c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd811a690 0x7f2dd8111a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.937+0000 7f2ddc99c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd811a690 0x7f2dd8111a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:35720/0 (socket says 172.21.15.57:35720) 2023-10-06T22:25:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.937+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1529928823 learned_addr learned my addr 172.21.15.57:0/1529928823 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.938+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1529928823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd810f620 msgr2=0x7f2dd8118b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.938+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd810f620 0x7f2dd8118b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.938+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dd810fa90 msgr2=0x7f2dd8119080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.938+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dd810fa90 0x7f2dd8119080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.938+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1529928823 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80095f0 con 0x7f2dd811a690 2023-10-06T22:25:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.939+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd811a690 0x7f2dd8111a70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0007960 tx=0x7f2dd0004ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.942+0000 7f2dd57fa700 1 -- 172.21.15.57:0/1529928823 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000e040 con 0x7f2dd811a690 2023-10-06T22:25:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.942+0000 7f2dd57fa700 1 -- 172.21.15.57:0/1529928823 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dd000bbf0 con 0x7f2dd811a690 2023-10-06T22:25:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.942+0000 7f2dd57fa700 1 -- 172.21.15.57:0/1529928823 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0005330 con 0x7f2dd811a690 2023-10-06T22:25:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.944+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd0009c40 con 0x7f2dd811a690 2023-10-06T22:25:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.944+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd8116870 con 0x7f2dd811a690 2023-10-06T22:25:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.949+0000 7f2df1e18b80 1 Processor -- start 2023-10-06T22:25:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.951+0000 7f2df1e18b80 1 -- start start 2023-10-06T22:25:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.954+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.954+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.954+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.954+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626def9400 con 0x55626df9b3d0 2023-10-06T22:25:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.954+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626deeabc0 con 0x55626df90520 2023-10-06T22:25:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.954+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626dee84e0 con 0x55626df8fc00 2023-10-06T22:25:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.955+0000 7f2dd57fa700 1 -- 172.21.15.57:0/1529928823 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2dd0005490 con 0x7f2dd811a690 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:37630/0 (socket says 172.21.15.57:37630) 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 -- 172.21.15.57:0/3168280848 learned_addr learned my addr 172.21.15.57:0/3168280848 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 -- 172.21.15.57:0/3168280848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 msgr2=0x55626df8ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df8ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 -- 172.21.15.57:0/3168280848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 msgr2=0x55626df9d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df9d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.956+0000 7f2dd7fff700 1 -- 172.21.15.57:0/3168280848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55626dd0ed80 con 0x55626df90520 2023-10-06T22:25:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.957+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9ae90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098a0 tx=0x7f2dcc009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7cbb31c5d8d2c0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.957+0000 7f2dbe7fc700 1 -- 172.21.15.57:0/3168280848 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc011830 con 0x55626df90520 2023-10-06T22:25:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.958+0000 7f2dbe7fc700 1 -- 172.21.15.57:0/3168280848 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dcc011e50 con 0x55626df90520 2023-10-06T22:25:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.958+0000 7f2dbe7fc700 1 -- 172.21.15.57:0/3168280848 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc010380 con 0x55626df90520 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.958+0000 7f2dd57fa700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2dc00a17c0 0x7f2dc00a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.958+0000 7f2dd57fa700 1 -- 172.21.15.57:0/1529928823 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f2dd00cdff0 con 0x7f2dd811a690 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2dc00a17c0 0x7f2dc00a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2df1e18b80 1 -- 172.21.15.57:0/3168280848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 msgr2=0x55626df9ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9ae90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098a0 tx=0x7f2dcc009bb0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2df1e18b80 1 -- 172.21.15.57:0/3168280848 shutdown_connections 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df9d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df8ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/3168280848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9ae90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.959+0000 7f2df1e18b80 1 -- 172.21.15.57:0/3168280848 >> 172.21.15.57:0/3168280848 conn(0x55626df05200 msgr2=0x55626df8c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.960+0000 7f2df1e18b80 1 -- 172.21.15.57:0/3168280848 shutdown_connections 2023-10-06T22:25:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.960+0000 7f2df1e18b80 1 -- 172.21.15.57:0/3168280848 wait complete. 2023-10-06T22:25:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.960+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2dc00a17c0 0x7f2dc00a3c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80095c0 tx=0x7f2dc8015040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.961+0000 7f2df1e18b80 1 -- start start 2023-10-06T22:25:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.961+0000 7f2df1e18b80 1 Processor -- start 2023-10-06T22:25:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.962+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.962+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df97fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.962+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.962+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626dee3dd0 con 0x55626df9b3d0 2023-10-06T22:25:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.962+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626de806b0 con 0x55626df90520 2023-10-06T22:25:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.962+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626de80350 con 0x55626df8fc00 2023-10-06T22:25:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.962+0000 7f2df1e18b80 10 client.0 _ll_register_callbacks cb 0x55626dcdd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47708/0 (socket says 172.21.15.57:47708) 2023-10-06T22:25:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1821490074 learned_addr learned my addr 172.21.15.57:0/1821490074 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 msgr2=0x55626df9ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df9ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1821490074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 msgr2=0x55626df97fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T22:25:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df97fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.963+0000 7f2ddc99c700 1 -- 172.21.15.57:0/1821490074 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc0094d0 con 0x55626df9b3d0 2023-10-06T22:25:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.964+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df985f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00a2df0 tx=0x7f2dd009a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db6385a158462db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.964+0000 7f2db77fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.965+0000 7f2db77fe700 1 -- 172.21.15.57:0/1821490074 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd009b480 con 0x55626df9b3d0 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.965+0000 7f2db77fe700 1 -- 172.21.15.57:0/1821490074 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f2dd009f170 con 0x55626df9b3d0 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.965+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55626df83d50 con 0x55626df9b3d0 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.965+0000 7f2db77fe700 1 -- 172.21.15.57:0/1821490074 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dd00a3320 con 0x55626df9b3d0 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.965+0000 7f2df1e18b80 10 client.17127 fetch_fsmap learned FSMap version 60 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.965+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55626dcf7c00 con 0x55626df9b3d0 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.966+0000 7f2db77fe700 1 -- 172.21.15.57:0/1821490074 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55626df83d50 con 0x55626df9b3d0 2023-10-06T22:25:17.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.966+0000 7f2db77fe700 1 -- 172.21.15.57:0/1821490074 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 60) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f2dd00a2e90 con 0x55626df9b3d0 2023-10-06T22:25:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.967+0000 7f2df1e18b80 10 client.17127 fetch_fsmap finished waiting for FSMap version 60 2023-10-06T22:25:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.967+0000 7f2df1e18b80 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.967+0000 7f2df1e18b80 20 client.17127 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.967+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfaad20 0x55626dfad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.967+0000 7f2df1e18b80 4 client.17127 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.967+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55626dedcdb0 con 0x55626dfaad20 2023-10-06T22:25:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.971+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfaad20 0x55626dfad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.979+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfaad20 0x55626dfad140 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09009fd3ec364ac server_cookie=3ef8158726bb538f in_seq=0 out_seq=0 2023-10-06T22:25:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.979+0000 7f2db77fe700 10 client.17127 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.982+0000 7f2db77fe700 1 -- 172.21.15.57:0/1821490074 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55626dedcdb0 con 0x55626dfaad20 2023-10-06T22:25:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.982+0000 7f2db77fe700 10 client.17127 handle_command_reply: tid=0 2023-10-06T22:25:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.983+0000 7f2df1e18b80 1 client.17127 shutdown 2023-10-06T22:25:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.984+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 msgr2=0x55626df985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.984+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df985f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00a2df0 tx=0x7f2dd009a980 comp rx=0 tx=0).stop 2023-10-06T22:25:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.986+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 shutdown_connections 2023-10-06T22:25:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.986+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfaad20 0x55626dfad140 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.986+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df9b3d0 0x55626df985f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.986+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df8fc00 0x55626df9ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.986+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/1821490074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df97fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.986+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 >> 172.21.15.57:0/1821490074 conn(0x55626df05200 msgr2=0x55626df8bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.986+0000 7f2db77fe700 0 client.17127 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.987+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 shutdown_connections 2023-10-06T22:25:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.987+0000 7f2df1e18b80 1 -- 172.21.15.57:0/1821490074 wait complete. 2023-10-06T22:25:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:17.987+0000 7f2df1e18b80 20 client.17127 trim_cache size 0 max 16384 2023-10-06T22:25:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 Processor -- start 2023-10-06T22:25:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 -- start start 2023-10-06T22:25:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626dfa1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df93390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626df977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626dd648c0 con 0x55626df8fc00 2023-10-06T22:25:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626dfa24b0 con 0x55626df90520 2023-10-06T22:25:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.015+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626df96540 con 0x55626df9b3d0 2023-10-06T22:25:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.016+0000 7f2ddc99c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626df977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.016+0000 7f2ddc99c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626df977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:35734/0 (socket says 172.21.15.57:35734) 2023-10-06T22:25:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.016+0000 7f2ddc99c700 1 -- 172.21.15.57:0/2041371465 learned_addr learned my addr 172.21.15.57:0/2041371465 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.017+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626dfa1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.017+0000 7f2ddc99c700 1 -- 172.21.15.57:0/2041371465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 msgr2=0x55626df93390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.017+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df93390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.017+0000 7f2ddc99c700 1 -- 172.21.15.57:0/2041371465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 msgr2=0x55626dfa1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.017+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626dfa1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.017+0000 7f2ddc99c700 1 -- 172.21.15.57:0/2041371465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55626dd351f0 con 0x55626df9b3d0 2023-10-06T22:25:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.017+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626df977a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2dd009afb0 tx=0x7f2dd000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0fd260753cff7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.018+0000 7f2dbeffd700 1 -- 172.21.15.57:0/2041371465 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00b9240 con 0x55626df9b3d0 2023-10-06T22:25:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.018+0000 7f2dbeffd700 1 -- 172.21.15.57:0/2041371465 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dd009f170 con 0x55626df9b3d0 2023-10-06T22:25:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.018+0000 7f2dbeffd700 1 -- 172.21.15.57:0/2041371465 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00a33b0 con 0x55626df9b3d0 2023-10-06T22:25:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.019+0000 7f2df1e18b80 1 -- 172.21.15.57:0/2041371465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 msgr2=0x55626df977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.019+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626df977a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2dd009afb0 tx=0x7f2dd000bd60 comp rx=0 tx=0).stop 2023-10-06T22:25:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.020+0000 7f2df1e18b80 1 -- 172.21.15.57:0/2041371465 shutdown_connections 2023-10-06T22:25:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.020+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626df977a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.020+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626dfa1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.020+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/2041371465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df93390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.020+0000 7f2df1e18b80 1 -- 172.21.15.57:0/2041371465 >> 172.21.15.57:0/2041371465 conn(0x55626df05200 msgr2=0x55626dfa66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.021+0000 7f2df1e18b80 1 -- 172.21.15.57:0/2041371465 shutdown_connections 2023-10-06T22:25:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.021+0000 7f2df1e18b80 1 -- 172.21.15.57:0/2041371465 wait complete. 2023-10-06T22:25:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.022+0000 7f2df1e18b80 1 -- start start 2023-10-06T22:25:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.022+0000 7f2df1e18b80 1 Processor -- start 2023-10-06T22:25:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626df9df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2df1e18b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626dffdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626df9eda0 con 0x55626df8fc00 2023-10-06T22:25:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626dffe080 con 0x55626df90520 2023-10-06T22:25:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2df1e18b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626dffe1c0 con 0x55626df9b3d0 2023-10-06T22:25:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2df1e18b80 10 client.0 _ll_register_callbacks cb 0x55626dcdd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626df9df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626df9df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47730/0 (socket says 172.21.15.57:47730) 2023-10-06T22:25:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.023+0000 7f2dd7fff700 1 -- 172.21.15.57:0/4241320524 learned_addr learned my addr 172.21.15.57:0/4241320524 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.024+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.024+0000 7f2ddc99c700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626dffdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.024+0000 7f2dd7fff700 1 -- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 msgr2=0x55626dffdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.024+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626dffdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.025+0000 7f2dd7fff700 1 -- 172.21.15.57:0/4241320524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 msgr2=0x55626df9e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.025+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.025+0000 7f2dd7fff700 1 -- 172.21.15.57:0/4241320524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55626dcf7c00 con 0x55626df8fc00 2023-10-06T22:25:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.025+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.025+0000 7f2dd7fff700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626df9df50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098a0 tx=0x7f2dcc004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcbfcc75f527e91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.025+0000 7f2db77fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-06T22:25:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.026+0000 7f2db77fe700 1 -- 172.21.15.57:0/4241320524 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00b5e0 con 0x55626df8fc00 2023-10-06T22:25:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.026+0000 7f2db77fe700 1 -- 172.21.15.57:0/4241320524 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f2dcc020040 con 0x55626df8fc00 2023-10-06T22:25:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.026+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dcc011e50 con 0x55626df8fc00 2023-10-06T22:25:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.026+0000 7f2db77fe700 1 -- 172.21.15.57:0/4241320524 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dcc0235a0 con 0x55626df8fc00 2023-10-06T22:25:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.026+0000 7f2df1e18b80 10 client.17133 fetch_fsmap learned FSMap version 60 2023-10-06T22:25:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.026+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55626dd0d250 con 0x55626df8fc00 2023-10-06T22:25:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2db77fe700 1 -- 172.21.15.57:0/4241320524 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dcc011e50 con 0x55626df8fc00 2023-10-06T22:25:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2db77fe700 1 -- 172.21.15.57:0/4241320524 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 60) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f2dcc011630 con 0x55626df8fc00 2023-10-06T22:25:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2df1e18b80 10 client.17133 fetch_fsmap finished waiting for FSMap version 60 2023-10-06T22:25:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2df1e18b80 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2df1e18b80 20 client.17133 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfff950 0x55626e001d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2df1e18b80 4 client.17133 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:25:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.027+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55626dedcad0 con 0x55626dfff950 2023-10-06T22:25:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.029+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfff950 0x55626e001d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.033+0000 7f2dd77fe700 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfff950 0x55626e001d70 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31b0a177bfda5ca server_cookie=52e9e28fa68c47a0 in_seq=0 out_seq=0 2023-10-06T22:25:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:18.033+0000 7f2db77fe700 10 client.17133 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:18.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:18 smithi057 ceph-mon[98492]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T22:25:18.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:18 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6834/1181822965,v1:172.21.15.156:6835/1181822965] up:boot 2023-10-06T22:25:18.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:18 smithi057 ceph-mon[98492]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-10-06T22:25:18.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:18 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:25:18.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:18 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:18.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:18 smithi156 ceph-mon[134358]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T22:25:18.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:18 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6834/1181822965,v1:172.21.15.156:6835/1181822965] up:boot 2023-10-06T22:25:18.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:18 smithi156 ceph-mon[134358]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-10-06T22:25:18.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:18 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:25:18.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:18 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:18.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:18 smithi190 ceph-mon[134891]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T22:25:18.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:18 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6834/1181822965,v1:172.21.15.156:6835/1181822965] up:boot 2023-10-06T22:25:18.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:18 smithi190 ceph-mon[134891]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-10-06T22:25:18.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:18 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:25:18.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:18 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:19.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:19 smithi057 ceph-mon[98492]: pgmap v330: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.1 MiB/s wr, 260 op/s 2023-10-06T22:25:19.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:19 smithi156 ceph-mon[134358]: pgmap v330: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.1 MiB/s wr, 260 op/s 2023-10-06T22:25:19.571 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:19 smithi190 ceph-mon[134891]: pgmap v330: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.1 MiB/s wr, 260 op/s 2023-10-06T22:25:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.855+0000 7f2db77fe700 1 -- 172.21.15.57:0/4241320524 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55626dedcad0 con 0x55626dfff950 2023-10-06T22:25:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.856+0000 7f2db77fe700 10 client.17133 handle_command_reply: tid=0 2023-10-06T22:25:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.856+0000 7f2df1e18b80 1 client.17133 shutdown 2023-10-06T22:25:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.858+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 msgr2=0x55626df9df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.858+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626df9df50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098a0 tx=0x7f2dcc004860 comp rx=0 tx=0).stop 2023-10-06T22:25:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.859+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 shutdown_connections 2023-10-06T22:25:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.859+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x55626dfff950 0x55626e001d70 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.859+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55626df9b3d0 0x55626dffdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.859+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55626df8fc00 0x55626df9df50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.859+0000 7f2df1e18b80 1 --2- 172.21.15.57:0/4241320524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55626df90520 0x55626df9e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.859+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 >> 172.21.15.57:0/4241320524 conn(0x55626df05200 msgr2=0x55626df93390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.860+0000 7f2db77fe700 0 client.17133 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.860+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 shutdown_connections 2023-10-06T22:25:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.860+0000 7f2df1e18b80 1 -- 172.21.15.57:0/4241320524 wait complete. 2023-10-06T22:25:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.861+0000 7f2df1e18b80 20 client.17133 trim_cache size 0 max 16384 2023-10-06T22:25:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:25:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:25:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:25:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:25:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.863+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2dc00a17c0 msgr2=0x7f2dc00a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.863+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2dc00a17c0 0x7f2dc00a3c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80095c0 tx=0x7f2dc8015040 comp rx=0 tx=0).stop 2023-10-06T22:25:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.863+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd811a690 msgr2=0x7f2dd8111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.863+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd811a690 0x7f2dd8111a70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0007960 tx=0x7f2dd0004ae0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 shutdown_connections 2023-10-06T22:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2dc00a17c0 0x7f2dc00a3c80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd811a690 0x7f2dd8111a70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dd810f620 0x7f2dd8118b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 --2- 172.21.15.57:0/1529928823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dd810fa90 0x7f2dd8119080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 >> 172.21.15.57:0/1529928823 conn(0x7f2dd806cfa0 msgr2=0x7f2dd806f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 shutdown_connections 2023-10-06T22:25:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:19.867+0000 7f2dde3ff700 1 -- 172.21.15.57:0/1529928823 wait complete. 2023-10-06T22:25:21.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:21 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:21.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:21 smithi156 ceph-mon[134358]: pgmap v331: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-10-06T22:25:21.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:21 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:21.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:21 smithi190 ceph-mon[134891]: pgmap v331: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-10-06T22:25:21.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:21 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:21.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:21 smithi057 ceph-mon[98492]: pgmap v331: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-10-06T22:25:23.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:23 smithi190 ceph-mon[134891]: pgmap v332: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2023-10-06T22:25:23.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:23 smithi057 ceph-mon[98492]: pgmap v332: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2023-10-06T22:25:23.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:23 smithi156 ceph-mon[134358]: pgmap v332: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2023-10-06T22:25:25.573 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:25 smithi190 ceph-mon[134891]: pgmap v333: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-10-06T22:25:25.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:25 smithi057 ceph-mon[98492]: pgmap v333: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-10-06T22:25:25.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:25 smithi156 ceph-mon[134358]: pgmap v333: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 213 op/s 2023-10-06T22:25:26.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:26 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:26 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:26.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:26 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:27.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:27 smithi190 ceph-mon[134891]: pgmap v334: 97 pgs: 18 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 68 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 207 op/s 2023-10-06T22:25:27.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:27 smithi057 ceph-mon[98492]: pgmap v334: 97 pgs: 18 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 68 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 207 op/s 2023-10-06T22:25:27.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:27 smithi156 ceph-mon[134358]: pgmap v334: 97 pgs: 18 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 68 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 207 op/s 2023-10-06T22:25:29.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:29 smithi190 ceph-mon[134891]: pgmap v335: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2023-10-06T22:25:29.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:29 smithi057 ceph-mon[98492]: pgmap v335: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2023-10-06T22:25:29.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:29 smithi156 ceph-mon[134358]: pgmap v335: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2023-10-06T22:25:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:31 smithi057 ceph-mon[98492]: pgmap v336: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 205 op/s 2023-10-06T22:25:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:31 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:31 smithi156 ceph-mon[134358]: pgmap v336: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 205 op/s 2023-10-06T22:25:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:31 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:31.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:31 smithi190 ceph-mon[134891]: pgmap v336: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 205 op/s 2023-10-06T22:25:31.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:31 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:25:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.035+0000 7f6613fff700 1 -- 172.21.15.57:0/1539176746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 msgr2=0x7f660c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.035+0000 7f6613fff700 1 --2- 172.21.15.57:0/1539176746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c103700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6608009990 tx=0x7f6608009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.035+0000 7f6613fff700 1 -- 172.21.15.57:0/1539176746 shutdown_connections 2023-10-06T22:25:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.035+0000 7f6613fff700 1 --2- 172.21.15.57:0/1539176746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f660c108440 0x7f660c10ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.035+0000 7f6613fff700 1 --2- 172.21.15.57:0/1539176746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c103700 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.035+0000 7f6613fff700 1 --2- 172.21.15.57:0/1539176746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f660c103cd0 0x7f660c107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.035+0000 7f6613fff700 1 -- 172.21.15.57:0/1539176746 >> 172.21.15.57:0/1539176746 conn(0x7f660c0feb60 msgr2=0x7f660c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.036+0000 7f6613fff700 1 -- 172.21.15.57:0/1539176746 shutdown_connections 2023-10-06T22:25:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.036+0000 7f6613fff700 1 -- 172.21.15.57:0/1539176746 wait complete. 2023-10-06T22:25:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 Processor -- start 2023-10-06T22:25:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 -- start start 2023-10-06T22:25:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f660c108440 0x7f660c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f660c19de40 0x7f660c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c19e8c0 con 0x7f660c108440 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c19ea30 con 0x7f660c19de40 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6613fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c19ebf0 con 0x7f660c103320 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6611d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.037+0000 7f6611d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60676/0 (socket says 172.21.15.57:60676) 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f6611d9b700 1 -- 172.21.15.57:0/1594190902 learned_addr learned my addr 172.21.15.57:0/1594190902 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f661159a700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f660c108440 0x7f660c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f661259c700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f660c19de40 0x7f660c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f6611d9b700 1 -- 172.21.15.57:0/1594190902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f660c19de40 msgr2=0x7f660c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f6611d9b700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f660c19de40 0x7f660c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f6611d9b700 1 -- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f660c108440 msgr2=0x7f660c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f6611d9b700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f660c108440 0x7f660c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f6611d9b700 1 -- 172.21.15.57:0/1594190902 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66080095f0 con 0x7f660c103320 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.038+0000 7f6611d9b700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c199290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6608005910 tx=0x7f6608005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.039+0000 7f6602ffd700 1 -- 172.21.15.57:0/1594190902 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660801d070 con 0x7f660c103320 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.039+0000 7f6602ffd700 1 -- 172.21.15.57:0/1594190902 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6608021410 con 0x7f660c103320 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.039+0000 7f6602ffd700 1 -- 172.21.15.57:0/1594190902 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608017650 con 0x7f660c103320 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.039+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c1a60f0 con 0x7f660c103320 2023-10-06T22:25:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.039+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c1a64c0 con 0x7f660c103320 2023-10-06T22:25:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.042+0000 7f6602ffd700 1 -- 172.21.15.57:0/1594190902 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6608021580 con 0x7f660c103320 2023-10-06T22:25:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 Processor -- start 2023-10-06T22:25:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 -- start start 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e994fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e995520 0x56531e99fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e9a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e8fe400 con 0x56531e995520 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e8efbc0 con 0x56531e9a03d0 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.044+0000 7f66278cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e8ed4e0 con 0x56531e994c00 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e994fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e9a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e994fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60680/0 (socket says 172.21.15.57:60680) 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 -- 172.21.15.57:0/725557421 learned_addr learned my addr 172.21.15.57:0/725557421 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e9a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:39502/0 (socket says 172.21.15.57:39502) 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f6611d9b700 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e995520 0x56531e99fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 -- 172.21.15.57:0/725557421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 msgr2=0x56531e9a27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e9a27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 -- 172.21.15.57:0/725557421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e995520 msgr2=0x56531e99fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e995520 0x56531e99fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 -- 172.21.15.57:0/725557421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56531e713d80 con 0x56531e994c00 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f6602ffd700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f65f80a16b0 0x7f65f80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f6602ffd700 1 -- 172.21.15.57:0/1594190902 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f66080ce2f0 con 0x7f660c103320 2023-10-06T22:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.045+0000 7f661259c700 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e994fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6604011680 tx=0x7f6604009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5f048dd2cd6065 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.046+0000 7f6611d9b700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f65f80a16b0 0x7f65f80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.046+0000 7f65f7fff700 1 -- 172.21.15.57:0/725557421 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660400e040 con 0x56531e994c00 2023-10-06T22:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.046+0000 7f65f7fff700 1 -- 172.21.15.57:0/725557421 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660400f040 con 0x56531e994c00 2023-10-06T22:25:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.046+0000 7f65f7fff700 1 -- 172.21.15.57:0/725557421 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040048c0 con 0x56531e994c00 2023-10-06T22:25:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.046+0000 7f6611d9b700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f65f80a16b0 0x7f65f80a3b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66080d1fd0 tx=0x7f6608097040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.046+0000 7f66278cfb80 1 -- 172.21.15.57:0/725557421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 msgr2=0x56531e994fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e994fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6604011680 tx=0x7f6604009920 comp rx=0 tx=0).stop 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 -- 172.21.15.57:0/725557421 shutdown_connections 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e9a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e994fe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 --2- 172.21.15.57:0/725557421 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e995520 0x56531e99fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 -- 172.21.15.57:0/725557421 >> 172.21.15.57:0/725557421 conn(0x56531e90a200 msgr2=0x56531e991080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 -- 172.21.15.57:0/725557421 shutdown_connections 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 -- 172.21.15.57:0/725557421 wait complete. 2023-10-06T22:25:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.047+0000 7f66278cfb80 1 -- start start 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 1 Processor -- start 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e994c00 0x56531e99fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e99d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e99d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e8e8dd0 con 0x56531e9a03d0 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e8856b0 con 0x56531e994c00 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e885350 con 0x56531e995520 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f66278cfb80 10 client.0 _ll_register_callbacks cb 0x56531e6e27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f661159a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e99d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f661159a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e99d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60688/0 (socket says 172.21.15.57:60688) 2023-10-06T22:25:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f661159a700 1 -- 172.21.15.57:0/4241630415 learned_addr learned my addr 172.21.15.57:0/4241630415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f661259c700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e994c00 0x56531e99fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.048+0000 7f661259c700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e99d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f661159a700 1 -- 172.21.15.57:0/4241630415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e994c00 msgr2=0x56531e99fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f661159a700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e994c00 0x56531e99fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f661159a700 1 -- 172.21.15.57:0/4241630415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 msgr2=0x56531e99d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f661159a700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e99d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f661159a700 1 -- 172.21.15.57:0/4241630415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6604011330 con 0x56531e995520 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f661259c700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e99d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f661159a700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e99d0f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65fc011780 tx=0x7f65fc011b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63996f7c4e69214 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f65f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f65f57fa700 1 -- 172.21.15.57:0/4241630415 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc016070 con 0x56531e995520 2023-10-06T22:25:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56531e988d50 con 0x56531e995520 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f65f57fa700 1 -- 172.21.15.57:0/4241630415 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f65fc00f890 con 0x56531e995520 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f65f57fa700 1 -- 172.21.15.57:0/4241630415 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65fc00fa90 con 0x56531e995520 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.049+0000 7f66278cfb80 10 client.27181 fetch_fsmap learned FSMap version 63 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x56531e6fcc00 con 0x56531e995520 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f65f57fa700 1 -- 172.21.15.57:0/4241630415 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65fc027020 con 0x56531e995520 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f65f57fa700 1 -- 172.21.15.57:0/4241630415 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 63) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f65fc021070 con 0x56531e995520 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f66278cfb80 10 client.27181 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f66278cfb80 10 client.27181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f66278cfb80 20 client.27181 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f66278cfb80 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531e9b0130 0x56531e9b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f66278cfb80 4 client.27181 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56531e8e1db0 con 0x56531e9b0130 2023-10-06T22:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.050+0000 7f661259c700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531e9b0130 0x56531e9b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.051+0000 7f65f57fa700 10 client.27181 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.051+0000 7f661259c700 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531e9b0130 0x56531e9b2550 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c576e0e0e98b16c server_cookie=7123b398793aafe1 in_seq=0 out_seq=0 2023-10-06T22:25:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.052+0000 7f65f57fa700 1 -- 172.21.15.57:0/4241630415 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56531e8e1db0 con 0x56531e9b0130 2023-10-06T22:25:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.052+0000 7f65f57fa700 10 client.27181 handle_command_reply: tid=0 2023-10-06T22:25:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.052+0000 7f66278cfb80 1 client.27181 shutdown 2023-10-06T22:25:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 msgr2=0x56531e99d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e99d0f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65fc011780 tx=0x7f65fc011b40 comp rx=0 tx=0).stop 2023-10-06T22:25:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 shutdown_connections 2023-10-06T22:25:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531e9b0130 0x56531e9b2550 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e99d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e994c00 0x56531e99fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 --2- 172.21.15.57:0/4241630415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e99d0f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f65f57fa700 0 client.27181 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 >> 172.21.15.57:0/4241630415 conn(0x56531e90a200 msgr2=0x56531e990c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 shutdown_connections 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 1 -- 172.21.15.57:0/4241630415 wait complete. 2023-10-06T22:25:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.053+0000 7f66278cfb80 20 client.27181 trim_cache size 0 max 16384 2023-10-06T22:25:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 Processor -- start 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 -- start start 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e994c00 0x56531e997b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e9a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e99c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e7698c0 con 0x56531e994c00 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e9ac900 con 0x56531e9a03d0 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.068+0000 7f66278cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e991dc0 con 0x56531e995520 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e9a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e9a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60690/0 (socket says 172.21.15.57:60690) 2023-10-06T22:25:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 -- 172.21.15.57:0/2295853816 learned_addr learned my addr 172.21.15.57:0/2295853816 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661259c700 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e994c00 0x56531e997b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 -- 172.21.15.57:0/2295853816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 msgr2=0x56531e99c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661259c700 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e99c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e99c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 -- 172.21.15.57:0/2295853816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e994c00 msgr2=0x56531e997b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e994c00 0x56531e997b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 -- 172.21.15.57:0/2295853816 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56531e73a1f0 con 0x56531e995520 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661259c700 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e99c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f661159a700 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e9a27c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65fc011750 tx=0x7f65fc009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ba1018a00792a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.069+0000 7f6600ff9700 1 -- 172.21.15.57:0/2295853816 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc016030 con 0x56531e995520 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f6600ff9700 1 -- 172.21.15.57:0/2295853816 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65fc027070 con 0x56531e995520 2023-10-06T22:25:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f6600ff9700 1 -- 172.21.15.57:0/2295853816 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc005340 con 0x56531e995520 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 -- 172.21.15.57:0/2295853816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 msgr2=0x56531e9a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e9a27c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65fc011750 tx=0x7f65fc009750 comp rx=0 tx=0).stop 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 -- 172.21.15.57:0/2295853816 shutdown_connections 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e9a03d0 0x56531e99c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e994c00 0x56531e997b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 --2- 172.21.15.57:0/2295853816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e995520 0x56531e9a27c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 -- 172.21.15.57:0/2295853816 >> 172.21.15.57:0/2295853816 conn(0x56531e90a200 msgr2=0x56531e9ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 -- 172.21.15.57:0/2295853816 shutdown_connections 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.070+0000 7f66278cfb80 1 -- 172.21.15.57:0/2295853816 wait complete. 2023-10-06T22:25:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 -- start start 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 Processor -- start 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e9a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e995520 0x56531e9a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e9fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e9fd470 con 0x56531e9a03d0 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e9fd5e0 con 0x56531e995520 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.071+0000 7f66278cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531e9fd7a0 con 0x56531e994c00 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f66278cfb80 10 client.0 _ll_register_callbacks cb 0x7f65fc009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e9a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e995520 0x56531e9a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e9a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:60700/0 (socket says 172.21.15.57:60700) 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661259c700 1 -- 172.21.15.57:0/3951726153 learned_addr learned my addr 172.21.15.57:0/3951726153 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661259c700 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e9fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661159a700 1 -- 172.21.15.57:0/3951726153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 msgr2=0x56531e9a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661159a700 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e9a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661159a700 1 -- 172.21.15.57:0/3951726153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 msgr2=0x56531e9fcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661159a700 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e9fcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661159a700 1 -- 172.21.15.57:0/3951726153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56531e6fcc00 con 0x56531e995520 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661259c700 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e9fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f661159a700 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e995520 0x56531e9a5630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x56531e9980c0 tx=0x7f65fc009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63dcfce5f34c7b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.072+0000 7f65f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f65f57fa700 1 -- 172.21.15.57:0/3951726153 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc016070 con 0x56531e995520 2023-10-06T22:25:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f660400f040 con 0x56531e995520 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f65f57fa700 1 -- 172.21.15.57:0/3951726153 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f65fc017070 con 0x56531e995520 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f65f57fa700 1 -- 172.21.15.57:0/3951726153 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65fc011300 con 0x56531e995520 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f66278cfb80 10 client.27155 fetch_fsmap learned FSMap version 63 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x56531e712250 con 0x56531e995520 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f65f57fa700 1 -- 172.21.15.57:0/3951726153 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660400f040 con 0x56531e995520 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f65f57fa700 1 -- 172.21.15.57:0/3951726153 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 63) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f65fc011860 con 0x56531e995520 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.073+0000 7f66278cfb80 10 client.27155 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f66278cfb80 10 client.27155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f66278cfb80 20 client.27155 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f66278cfb80 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531ea04b90 0x56531ea06fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f66278cfb80 4 client.27155 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:25:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56531e8e1ad0 con 0x56531ea04b90 2023-10-06T22:25:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f661259c700 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531ea04b90 0x56531ea06fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f65f57fa700 10 client.27155 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:33.074+0000 7f661259c700 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531ea04b90 0x56531ea06fb0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d223f738a5eac7df server_cookie=276eaff10a0aee4 in_seq=0 out_seq=0 2023-10-06T22:25:33.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:33 smithi057 ceph-mon[98492]: pgmap v337: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2023-10-06T22:25:33.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:33 smithi156 ceph-mon[134358]: pgmap v337: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2023-10-06T22:25:33.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:33 smithi190 ceph-mon[134891]: pgmap v337: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2023-10-06T22:25:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.979+0000 7f65f57fa700 1 -- 172.21.15.57:0/3951726153 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56531e8e1ad0 con 0x56531ea04b90 2023-10-06T22:25:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.979+0000 7f65f57fa700 10 client.27155 handle_command_reply: tid=0 2023-10-06T22:25:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.979+0000 7f66278cfb80 1 client.27155 shutdown 2023-10-06T22:25:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e995520 msgr2=0x56531e9a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e995520 0x56531e9a5630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x56531e9980c0 tx=0x7f65fc009240 comp rx=0 tx=0).stop 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 shutdown_connections 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x56531ea04b90 0x56531ea06fb0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56531e9a03d0 0x56531e9fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56531e994c00 0x56531e9a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f65f57fa700 0 client.27155 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 --2- 172.21.15.57:0/3951726153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56531e995520 0x56531e9a5630 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 >> 172.21.15.57:0/3951726153 conn(0x56531e90a200 msgr2=0x56531e9ab670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 shutdown_connections 2023-10-06T22:25:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.980+0000 7f66278cfb80 1 -- 172.21.15.57:0/3951726153 wait complete. 2023-10-06T22:25:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.981+0000 7f66278cfb80 20 client.27155 trim_cache size 0 max 16384 2023-10-06T22:25:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:25:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:25:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:25:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:25:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.981+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f65f80a16b0 msgr2=0x7f65f80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.981+0000 7f6613fff700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f65f80a16b0 0x7f65f80a3b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66080d1fd0 tx=0x7f6608097040 comp rx=0 tx=0).stop 2023-10-06T22:25:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.981+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 msgr2=0x7f660c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.981+0000 7f6613fff700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c199290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6608005910 tx=0x7f6608005740 comp rx=0 tx=0).stop 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 shutdown_connections 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f65f80a16b0 0x7f65f80a3b70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f660c19de40 0x7f660c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f660c103320 0x7f660c199290 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 --2- 172.21.15.57:0/1594190902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f660c108440 0x7f660c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 >> 172.21.15.57:0/1594190902 conn(0x7f660c0feb60 msgr2=0x7f660c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 shutdown_connections 2023-10-06T22:25:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:34.982+0000 7f6613fff700 1 -- 172.21.15.57:0/1594190902 wait complete. 2023-10-06T22:25:35.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:35 smithi057 ceph-mon[98492]: pgmap v338: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2023-10-06T22:25:35.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:35 smithi156 ceph-mon[134358]: pgmap v338: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2023-10-06T22:25:35.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:35 smithi190 ceph-mon[134891]: pgmap v338: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2023-10-06T22:25:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:37 smithi057 ceph-mon[98492]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T22:25:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:37 smithi057 ceph-mon[98492]: pgmap v340: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 312 op/s 2023-10-06T22:25:37.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:37 smithi156 ceph-mon[134358]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T22:25:37.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:37 smithi156 ceph-mon[134358]: pgmap v340: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 312 op/s 2023-10-06T22:25:37.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:37 smithi190 ceph-mon[134891]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T22:25:37.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:37 smithi190 ceph-mon[134891]: pgmap v340: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 312 op/s 2023-10-06T22:25:39.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:39 smithi190 ceph-mon[134891]: pgmap v341: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 451 op/s 2023-10-06T22:25:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:39 smithi057 ceph-mon[98492]: pgmap v341: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 451 op/s 2023-10-06T22:25:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:39 smithi156 ceph-mon[134358]: pgmap v341: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 451 op/s 2023-10-06T22:25:41.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:41 smithi190 ceph-mon[134891]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2023-10-06T22:25:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:41 smithi057 ceph-mon[98492]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2023-10-06T22:25:41.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:41 smithi156 ceph-mon[134358]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2023-10-06T22:25:43.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:43 smithi190 ceph-mon[134891]: pgmap v343: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 461 op/s 2023-10-06T22:25:43.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:43 smithi057 ceph-mon[98492]: pgmap v343: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 461 op/s 2023-10-06T22:25:43.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:43 smithi156 ceph-mon[134358]: pgmap v343: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 461 op/s 2023-10-06T22:25:45.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:45 smithi190 ceph-mon[134891]: pgmap v344: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 450 op/s 2023-10-06T22:25:45.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:45 smithi057 ceph-mon[98492]: pgmap v344: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 450 op/s 2023-10-06T22:25:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:45 smithi156 ceph-mon[134358]: pgmap v344: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 450 op/s 2023-10-06T22:25:47.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:47 smithi057 ceph-mon[98492]: pgmap v345: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 416 op/s 2023-10-06T22:25:47.505 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:47 smithi156 ceph-mon[134358]: pgmap v345: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 416 op/s 2023-10-06T22:25:47.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:47 smithi190 ceph-mon[134891]: pgmap v345: 97 pgs: 97 active+clean; 1.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 416 op/s 2023-10-06T22:25:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.171+0000 7f081cdff700 1 -- 172.21.15.57:0/588970299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 msgr2=0x7f08181036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.171+0000 7f081cdff700 1 --2- 172.21.15.57:0/588970299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 0x7f08181036e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f08080098f0 tx=0x7f0808009c00 comp rx=0 tx=0).stop 2023-10-06T22:25:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.173+0000 7f081cdff700 1 -- 172.21.15.57:0/588970299 shutdown_connections 2023-10-06T22:25:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.173+0000 7f081cdff700 1 --2- 172.21.15.57:0/588970299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 0x7f081810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.173+0000 7f081cdff700 1 --2- 172.21.15.57:0/588970299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 0x7f08181036e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.173+0000 7f081cdff700 1 --2- 172.21.15.57:0/588970299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0818103cb0 0x7f0818107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.173+0000 7f081cdff700 1 -- 172.21.15.57:0/588970299 >> 172.21.15.57:0/588970299 conn(0x7f08180feb60 msgr2=0x7f0818100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.173+0000 7f081cdff700 1 -- 172.21.15.57:0/588970299 shutdown_connections 2023-10-06T22:25:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.174+0000 7f081cdff700 1 -- 172.21.15.57:0/588970299 wait complete. 2023-10-06T22:25:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.175+0000 7f081cdff700 1 Processor -- start 2023-10-06T22:25:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.175+0000 7f081cdff700 1 -- start start 2023-10-06T22:25:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f081cdff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 0x7f081810e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f081cdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 0x7f081810ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f081cdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08181a7570 0x7f08181a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f081cdff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181a9fa0 con 0x7f0818103300 2023-10-06T22:25:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f081cdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181aa110 con 0x7f08181a7570 2023-10-06T22:25:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f081cdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181aa2d0 con 0x7f0818108420 2023-10-06T22:25:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f0815d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 0x7f081810ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f0815d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 0x7f081810ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45356/0 (socket says 172.21.15.57:45356) 2023-10-06T22:25:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.176+0000 7f0815d9b700 1 -- 172.21.15.57:0/1634607532 learned_addr learned my addr 172.21.15.57:0/1634607532 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.177+0000 7f081659c700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 0x7f081810e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.177+0000 7f0815d9b700 1 -- 172.21.15.57:0/1634607532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08181a7570 msgr2=0x7f08181a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.177+0000 7f0815d9b700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08181a7570 0x7f08181a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.177+0000 7f0815d9b700 1 -- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 msgr2=0x7f081810e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.177+0000 7f0815d9b700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 0x7f081810e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.177+0000 7f0815d9b700 1 -- 172.21.15.57:0/1634607532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810009d00 con 0x7f0818108420 2023-10-06T22:25:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.177+0000 7f081659c700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 0x7f081810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:25:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.178+0000 7f0815d9b700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 0x7f081810ec30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f080c00e970 tx=0x7f080c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.178+0000 7f08077fe700 1 -- 172.21.15.57:0/1634607532 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c009790 con 0x7f0818108420 2023-10-06T22:25:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.178+0000 7f08077fe700 1 -- 172.21.15.57:0/1634607532 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c004cf0 con 0x7f0818108420 2023-10-06T22:25:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.178+0000 7f08077fe700 1 -- 172.21.15.57:0/1634607532 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c005660 con 0x7f0818108420 2023-10-06T22:25:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.179+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0808009520 con 0x7f0818108420 2023-10-06T22:25:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.179+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08181aa8e0 con 0x7f0818108420 2023-10-06T22:25:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.185+0000 7f0830712b80 1 Processor -- start 2023-10-06T22:25:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.185+0000 7f08077fe700 1 -- 172.21.15.57:0/1634607532 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f080c00cc60 con 0x7f0818108420 2023-10-06T22:25:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.185+0000 7f0830712b80 1 -- start start 2023-10-06T22:25:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0830712b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf412fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0830712b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0830712b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf4207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0830712b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf37c400 con 0x5650bf41e3d0 2023-10-06T22:25:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0830712b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf36dbc0 con 0x5650bf412c00 2023-10-06T22:25:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0830712b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf36b4e0 con 0x5650bf413520 2023-10-06T22:25:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0815d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf4207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0815d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf4207c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55818/0 (socket says 172.21.15.57:55818) 2023-10-06T22:25:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0815d9b700 1 -- 172.21.15.57:0/3040790217 learned_addr learned my addr 172.21.15.57:0/3040790217 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f0816d9d700 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf412fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.186+0000 7f081659c700 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.187+0000 7f0815d9b700 1 -- 172.21.15.57:0/3040790217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 msgr2=0x5650bf41de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.187+0000 7f0815d9b700 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.187+0000 7f0815d9b700 1 -- 172.21.15.57:0/3040790217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 msgr2=0x5650bf412fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.187+0000 7f0815d9b700 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf412fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.187+0000 7f0815d9b700 1 -- 172.21.15.57:0/3040790217 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf191d80 con 0x5650bf41e3d0 2023-10-06T22:25:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.187+0000 7f0815d9b700 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf4207c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ca8d0 tx=0x7f080c0d0f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190274dc7c82f88b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.188+0000 7f0804ff9700 1 -- 172.21.15.57:0/3040790217 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0d5930 con 0x5650bf41e3d0 2023-10-06T22:25:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.188+0000 7f0804ff9700 1 -- 172.21.15.57:0/3040790217 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c0cf2e0 con 0x5650bf41e3d0 2023-10-06T22:25:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.188+0000 7f0804ff9700 1 -- 172.21.15.57:0/3040790217 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0db400 con 0x5650bf41e3d0 2023-10-06T22:25:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.188+0000 7f08077fe700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f08000a17c0 0x7f08000a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.189+0000 7f08077fe700 1 -- 172.21.15.57:0/1634607532 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f080c014070 con 0x7f0818108420 2023-10-06T22:25:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.189+0000 7f0830712b80 1 -- 172.21.15.57:0/3040790217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 msgr2=0x5650bf4207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.189+0000 7f0830712b80 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf4207c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ca8d0 tx=0x7f080c0d0f50 comp rx=0 tx=0).stop 2023-10-06T22:25:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.190+0000 7f0830712b80 1 -- 172.21.15.57:0/3040790217 shutdown_connections 2023-10-06T22:25:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.190+0000 7f0830712b80 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf4207c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.190+0000 7f0830712b80 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf412fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.190+0000 7f0830712b80 1 --2- 172.21.15.57:0/3040790217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.190+0000 7f0830712b80 1 -- 172.21.15.57:0/3040790217 >> 172.21.15.57:0/3040790217 conn(0x5650bf388200 msgr2=0x5650bf40f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.190+0000 7f0816d9d700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f08000a17c0 0x7f08000a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.191+0000 7f0830712b80 1 -- 172.21.15.57:0/3040790217 shutdown_connections 2023-10-06T22:25:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.191+0000 7f0816d9d700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f08000a17c0 0x7f08000a3c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0810009fd0 tx=0x7f081000d040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.192+0000 7f0830712b80 1 -- 172.21.15.57:0/3040790217 wait complete. 2023-10-06T22:25:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.192+0000 7f0830712b80 1 -- start start 2023-10-06T22:25:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.192+0000 7f0830712b80 1 Processor -- start 2023-10-06T22:25:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0830712b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf41de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0830712b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0830712b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf41b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0830712b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf366dd0 con 0x5650bf41e3d0 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0830712b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf3036b0 con 0x5650bf412c00 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0830712b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf303350 con 0x5650bf413520 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0830712b80 10 client.0 _ll_register_callbacks cb 0x5650bf1607a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0815d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0815d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45380/0 (socket says 172.21.15.57:45380) 2023-10-06T22:25:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.193+0000 7f0815d9b700 1 -- 172.21.15.57:0/342899749 learned_addr learned my addr 172.21.15.57:0/342899749 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f081659c700 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf41de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f081659c700 1 -- 172.21.15.57:0/342899749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 msgr2=0x5650bf41b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f081659c700 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f081659c700 1 -- 172.21.15.57:0/342899749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 msgr2=0x5650bf41b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f081659c700 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf41b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f081659c700 1 -- 172.21.15.57:0/342899749 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08181aa8e0 con 0x5650bf412c00 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f081659c700 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf41de40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0808009810 tx=0x7f080800f6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e1efd085075deb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.194+0000 7f07fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.195+0000 7f07fdffb700 1 -- 172.21.15.57:0/342899749 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808025070 con 0x5650bf412c00 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.195+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650bf406d50 con 0x5650bf412c00 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.195+0000 7f07fdffb700 1 -- 172.21.15.57:0/342899749 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f080800b710 con 0x5650bf412c00 2023-10-06T22:25:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.195+0000 7f07fdffb700 1 -- 172.21.15.57:0/342899749 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080801c040 con 0x5650bf412c00 2023-10-06T22:25:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.196+0000 7f0830712b80 10 client.27164 fetch_fsmap learned FSMap version 63 2023-10-06T22:25:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.196+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5650bf17ac00 con 0x5650bf412c00 2023-10-06T22:25:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.196+0000 7f07fdffb700 1 -- 172.21.15.57:0/342899749 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650bf406d50 con 0x5650bf412c00 2023-10-06T22:25:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.196+0000 7f07fdffb700 1 -- 172.21.15.57:0/342899749 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 63) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f0808028020 con 0x5650bf412c00 2023-10-06T22:25:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.197+0000 7f0830712b80 10 client.27164 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T22:25:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.197+0000 7f0830712b80 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.197+0000 7f0830712b80 20 client.27164 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.197+0000 7f0830712b80 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf43f070 0x5650bf441490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.197+0000 7f0830712b80 4 client.27164 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T22:25:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.197+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650bf35fdb0 con 0x5650bf43f070 2023-10-06T22:25:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.197+0000 7f0815d9b700 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf43f070 0x5650bf441490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.198+0000 7f0815d9b700 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf43f070 0x5650bf441490 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50175f129c694b99 server_cookie=cb3216245dd274f8 in_seq=0 out_seq=0 2023-10-06T22:25:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.198+0000 7f07fdffb700 10 client.27164 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.203+0000 7f07fdffb700 1 -- 172.21.15.57:0/342899749 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5650bf35fdb0 con 0x5650bf43f070 2023-10-06T22:25:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.203+0000 7f07fdffb700 10 client.27164 handle_command_reply: tid=0 2023-10-06T22:25:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.203+0000 7f0830712b80 1 client.27164 shutdown 2023-10-06T22:25:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 msgr2=0x5650bf41de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf41de40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0808009810 tx=0x7f080800f6f0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 shutdown_connections 2023-10-06T22:25:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf43f070 0x5650bf441490 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf41e3d0 0x5650bf41b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf41de40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 --2- 172.21.15.57:0/342899749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf413520 0x5650bf41b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 >> 172.21.15.57:0/342899749 conn(0x5650bf388200 msgr2=0x5650bf40ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f07fdffb700 0 client.27164 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 shutdown_connections 2023-10-06T22:25:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.204+0000 7f0830712b80 1 -- 172.21.15.57:0/342899749 wait complete. 2023-10-06T22:25:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.205+0000 7f0830712b80 20 client.27164 trim_cache size 0 max 16384 2023-10-06T22:25:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 Processor -- start 2023-10-06T22:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 -- start start 2023-10-06T22:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf424f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf413520 0x5650bf43b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf1e78c0 con 0x5650bf413520 2023-10-06T22:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf4254b0 con 0x5650bf412c00 2023-10-06T22:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.228+0000 7f0830712b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf416f10 con 0x5650bf41e3d0 2023-10-06T22:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf424f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf424f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:58780/0 (socket says 172.21.15.57:58780) 2023-10-06T22:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 -- 172.21.15.57:0/3223097727 learned_addr learned my addr 172.21.15.57:0/3223097727 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f0816d9d700 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f0815d9b700 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf413520 0x5650bf43b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 -- 172.21.15.57:0/3223097727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 msgr2=0x5650bf4296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 -- 172.21.15.57:0/3223097727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf413520 msgr2=0x5650bf43b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf413520 0x5650bf43b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.229+0000 7f081659c700 1 -- 172.21.15.57:0/3223097727 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf1b81f0 con 0x5650bf412c00 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.230+0000 7f081659c700 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf424f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f080800fa50 tx=0x7f0808009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95eb0d159da62e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.230+0000 7f08057fa700 1 -- 172.21.15.57:0/3223097727 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808025070 con 0x5650bf412c00 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.230+0000 7f08057fa700 1 -- 172.21.15.57:0/3223097727 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080800f810 con 0x5650bf412c00 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.230+0000 7f08057fa700 1 -- 172.21.15.57:0/3223097727 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808005320 con 0x5650bf412c00 2023-10-06T22:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 -- 172.21.15.57:0/3223097727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 msgr2=0x5650bf424f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf424f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f080800fa50 tx=0x7f0808009fd0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 -- 172.21.15.57:0/3223097727 shutdown_connections 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf412c00 0x5650bf424f70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 --2- 172.21.15.57:0/3223097727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf413520 0x5650bf43b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 -- 172.21.15.57:0/3223097727 >> 172.21.15.57:0/3223097727 conn(0x5650bf388200 msgr2=0x5650bf445df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 -- 172.21.15.57:0/3223097727 shutdown_connections 2023-10-06T22:25:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 -- 172.21.15.57:0/3223097727 wait complete. 2023-10-06T22:25:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.231+0000 7f0830712b80 1 -- start start 2023-10-06T22:25:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 1 Processor -- start 2023-10-06T22:25:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf412c00 0x5650bf420f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf413520 0x5650bf421490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf425460 con 0x5650bf412c00 2023-10-06T22:25:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf491e90 con 0x5650bf413520 2023-10-06T22:25:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf491fd0 con 0x5650bf41e3d0 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.232+0000 7f0830712b80 10 client.0 _ll_register_callbacks cb 0x7f080c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0816d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0816d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4219d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:45402/0 (socket says 172.21.15.57:45402) 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0816d9d700 1 -- 172.21.15.57:0/2548369441 learned_addr learned my addr 172.21.15.57:0/2548369441 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f081659c700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf412c00 0x5650bf420f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0815d9b700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf413520 0x5650bf421490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0815d9b700 1 -- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 msgr2=0x5650bf4219d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0815d9b700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4219d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0815d9b700 1 -- 172.21.15.57:0/2548369441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf412c00 msgr2=0x5650bf420f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0815d9b700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf412c00 0x5650bf420f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0815d9b700 1 -- 172.21.15.57:0/2548369441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650bf17ac00 con 0x5650bf413520 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f0816d9d700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.233+0000 7f081659c700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf412c00 0x5650bf420f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f0815d9b700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf413520 0x5650bf421490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f080c000c00 tx=0x7f080c0d53c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b002462d8a945362 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f07fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f07fdffb700 1 -- 172.21.15.57:0/2548369441 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c096560 con 0x5650bf413520 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f07fdffb700 1 -- 172.21.15.57:0/2548369441 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f080c09a920 con 0x5650bf413520 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650bf35fad0 con 0x5650bf413520 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f07fdffb700 1 -- 172.21.15.57:0/2548369441 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080c099ff0 con 0x5650bf413520 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f0830712b80 10 client.27170 fetch_fsmap learned FSMap version 63 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5650bf190250 con 0x5650bf413520 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.234+0000 7f07fdffb700 1 -- 172.21.15.57:0/2548369441 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650bf35fad0 con 0x5650bf413520 2023-10-06T22:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f07fdffb700 1 -- 172.21.15.57:0/2548369441 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 63) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f080c096930 con 0x5650bf413520 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f0830712b80 10 client.27170 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f0830712b80 10 client.27170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f0830712b80 20 client.27170 populate_metadata read hostname 'smithi057' 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f0830712b80 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf4934f0 0x5650bf495910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f0830712b80 4 client.27170 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 --> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5650bf1b2f10 con 0x5650bf4934f0 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.235+0000 7f081659c700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf4934f0 0x5650bf495910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.236+0000 7f081659c700 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf4934f0 0x5650bf495910 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930bf4e11fc64e6a server_cookie=a58c4fc827651ed0 in_seq=0 out_seq=0 2023-10-06T22:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.236+0000 7f07fdffb700 10 client.27170 ms_handle_connect on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f07fdffb700 1 -- 172.21.15.57:0/2548369441 <== mds.0 v2:172.21.15.190:6838/751138264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5650bf1b2f10 con 0x5650bf4934f0 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f07fdffb700 10 client.27170 handle_command_reply: tid=0 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 client.27170 shutdown 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf413520 msgr2=0x5650bf421490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf413520 0x5650bf421490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f080c000c00 tx=0x7f080c0d53c0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 shutdown_connections 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:6838/751138264,v1:172.21.15.190:6839/751138264] conn(0x5650bf4934f0 0x5650bf495910 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5650bf41e3d0 0x5650bf4219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5650bf412c00 0x5650bf420f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f07fdffb700 0 client.27170 ms_handle_reset on v2:172.21.15.190:6838/751138264 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 --2- 172.21.15.57:0/2548369441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650bf413520 0x5650bf421490 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 >> 172.21.15.57:0/2548369441 conn(0x5650bf388200 msgr2=0x5650bf4296d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 shutdown_connections 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 1 -- 172.21.15.57:0/2548369441 wait complete. 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.274+0000 7f0830712b80 20 client.27170 trim_cache size 0 max 16384 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-06T22:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-06T22:25:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f08000a17c0 msgr2=0x7f08000a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f08000a17c0 0x7f08000a3c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0810009fd0 tx=0x7f081000d040 comp rx=0 tx=0).stop 2023-10-06T22:25:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 msgr2=0x7f081810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 0x7f081810ec30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f080c00e970 tx=0x7f080c00ed30 comp rx=0 tx=0).stop 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 shutdown_connections 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f08000a17c0 0x7f08000a3c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08181a7570 0x7f08181a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0818103300 0x7f081810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 --2- 172.21.15.57:0/1634607532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0818108420 0x7f081810ec30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 >> 172.21.15.57:0/1634607532 conn(0x7f08180feb60 msgr2=0x7f08180fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 shutdown_connections 2023-10-06T22:25:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-06T22:25:48.275+0000 7f081cdff700 1 -- 172.21.15.57:0/1634607532 wait complete. 2023-10-06T22:25:49.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:49 smithi190 ceph-mon[134891]: pgmap v346: 97 pgs: 97 active+clean; 1.8 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 436 op/s 2023-10-06T22:25:49.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:49 smithi057 ceph-mon[98492]: pgmap v346: 97 pgs: 97 active+clean; 1.8 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 436 op/s 2023-10-06T22:25:49.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:49 smithi156 ceph-mon[134358]: pgmap v346: 97 pgs: 97 active+clean; 1.8 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 436 op/s 2023-10-06T22:25:51.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:51 smithi190 ceph-mon[134891]: pgmap v347: 97 pgs: 97 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 301 op/s 2023-10-06T22:25:51.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:51 smithi057 ceph-mon[98492]: pgmap v347: 97 pgs: 97 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 301 op/s 2023-10-06T22:25:51.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:51 smithi156 ceph-mon[134358]: pgmap v347: 97 pgs: 97 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 301 op/s 2023-10-06T22:25:53.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:53 smithi190 ceph-mon[134891]: pgmap v348: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 404 op/s 2023-10-06T22:25:53.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:53 smithi057 ceph-mon[98492]: pgmap v348: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 404 op/s 2023-10-06T22:25:53.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:53 smithi156 ceph-mon[134358]: pgmap v348: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 404 op/s 2023-10-06T22:25:55.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:55 smithi190 ceph-mon[134891]: pgmap v349: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 311 op/s 2023-10-06T22:25:55.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:55 smithi057 ceph-mon[98492]: pgmap v349: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 311 op/s 2023-10-06T22:25:55.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:55 smithi156 ceph-mon[134358]: pgmap v349: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 311 op/s 2023-10-06T22:25:56.599 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-10-06T22:25:56.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-06T22:25:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:57 smithi057 ceph-mon[98492]: pgmap v350: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2023-10-06T22:25:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:57 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:25:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:57 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:25:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:57 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:25:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:57 smithi156 ceph-mon[134358]: pgmap v350: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2023-10-06T22:25:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:57 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:25:57.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:57 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:25:57.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:57 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:25:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:57 smithi190 ceph-mon[134891]: pgmap v350: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2023-10-06T22:25:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:57 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T22:25:57.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:57 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:25:57.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:57 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:25:57.711 DEBUG:teuthology.parallel:result is None 2023-10-06T22:25:57.711 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T22:25:57.790 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T22:25:57.791 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-10-06T22:25:57.801 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:25:57.813 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-10-06T22:25:57.850 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-10-06T22:25:57.850 INFO:teuthology.task.exec:Executing custom commands... 2023-10-06T22:25:57.850 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:25:57.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-10-06T22:25:57.928 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-10-06T22:25:58.008 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-06T22:25:58.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 -- 172.21.15.57:0/816492724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 msgr2=0x7fd760102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 --2- 172.21.15.57:0/816492724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd760102440 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009990 tx=0x7fd75c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 -- 172.21.15.57:0/816492724 shutdown_connections 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 --2- 172.21.15.57:0/816492724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd76010b480 0x7fd76010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 --2- 172.21.15.57:0/816492724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd760102440 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 --2- 172.21.15.57:0/816492724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd760102980 0x7fd76010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 -- 172.21.15.57:0/816492724 >> 172.21.15.57:0/816492724 conn(0x7fd7600fb860 msgr2=0x7fd7600fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 -- 172.21.15.57:0/816492724 shutdown_connections 2023-10-06T22:25:58.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.221+0000 7fd7675ff700 1 -- 172.21.15.57:0/816492724 wait complete. 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.222+0000 7fd7675ff700 1 Processor -- start 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.222+0000 7fd7675ff700 1 -- start start 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd7675ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd7601991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd7675ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd760102980 0x7fd760199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd7675ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76010b480 0x7fd76019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd7675ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76019e3c0 con 0x7fd760102060 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd7675ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76019e530 con 0x7fd76010b480 2023-10-06T22:25:58.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd7675ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76019e6f0 con 0x7fd760102980 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd76539b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd7601991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd76539b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd7601991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46788/0 (socket says 172.21.15.57:46788) 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd76539b700 1 -- 172.21.15.57:0/594229549 learned_addr learned my addr 172.21.15.57:0/594229549 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd765b9c700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76010b480 0x7fd76019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.223+0000 7fd764b9a700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd760102980 0x7fd760199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd76539b700 1 -- 172.21.15.57:0/594229549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd760102980 msgr2=0x7fd760199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd76539b700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd760102980 0x7fd760199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd76539b700 1 -- 172.21.15.57:0/594229549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76010b480 msgr2=0x7fd76019dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd76539b700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76010b480 0x7fd76019dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd76539b700 1 -- 172.21.15.57:0/594229549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd750009520 con 0x7fd760102060 2023-10-06T22:25:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd76539b700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd7601991d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd75c005910 tx=0x7fd75c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:58.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd7567fc700 1 -- 172.21.15.57:0/594229549 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c01d070 con 0x7fd760102060 2023-10-06T22:25:58.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd7567fc700 1 -- 172.21.15.57:0/594229549 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd75c022470 con 0x7fd760102060 2023-10-06T22:25:58.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75c0095f0 con 0x7fd760102060 2023-10-06T22:25:58.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd7567fc700 1 -- 172.21.15.57:0/594229549 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c017650 con 0x7fd760102060 2023-10-06T22:25:58.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.224+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7601a6030 con 0x7fd760102060 2023-10-06T22:25:58.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.226+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7600ff4d0 con 0x7fd760102060 2023-10-06T22:25:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.229+0000 7fd7567fc700 1 -- 172.21.15.57:0/594229549 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd75c0225e0 con 0x7fd760102060 2023-10-06T22:25:58.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.232+0000 7fd7567fc700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.232+0000 7fd7567fc700 1 -- 172.21.15.57:0/594229549 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fd75c0ceb80 con 0x7fd760102060 2023-10-06T22:25:58.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.232+0000 7fd764b9a700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.233+0000 7fd764b9a700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd76019a9f0 tx=0x7fd750009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:58.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.235+0000 7fd7567fc700 1 -- 172.21.15.57:0/594229549 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd75c097ef0 con 0x7fd760102060 2023-10-06T22:25:58.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.489+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 --> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fd760110e30 con 0x7fd74c0a15f0 2023-10-06T22:25:58.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.500+0000 7fd7567fc700 1 -- 172.21.15.57:0/594229549 <== mgr.25414 v2:172.21.15.57:6800/3210017833 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7fd760110e30 con 0x7fd74c0a15f0 2023-10-06T22:25:58.502 INFO:teuthology.orchestra.run.smithi057.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T22:14:46", "first": "2023-10-06T22:15:00", "last": "2023-10-06T22:25:00", "last_pruned": "2023-10-06T22:25:02", "created_count": 11, "pruned_count": 5, "active": true} 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd74c0a15f0 msgr2=0x7fd74c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd76019a9f0 tx=0x7fd750009260 comp rx=0 tx=0).stop 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 msgr2=0x7fd7601991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd7601991d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd75c005910 tx=0x7fd75c005720 comp rx=0 tx=0).stop 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 shutdown_connections 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fd74c0a15f0 0x7fd74c0a3ab0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd76010b480 0x7fd76019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd760102060 0x7fd7601991d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 --2- 172.21.15.57:0/594229549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd760102980 0x7fd760199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.502+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 >> 172.21.15.57:0/594229549 conn(0x7fd7600fb860 msgr2=0x7fd76010d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:58.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.503+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 shutdown_connections 2023-10-06T22:25:58.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.503+0000 7fd7675ff700 1 -- 172.21.15.57:0/594229549 wait complete. 2023-10-06T22:25:58.525 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-06T22:25:58.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 -- 172.21.15.57:0/54084993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501010d0 msgr2=0x7ff5501014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 --2- 172.21.15.57:0/54084993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501010d0 0x7ff5501014b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009990 tx=0x7ff54c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 -- 172.21.15.57:0/54084993 shutdown_connections 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 --2- 172.21.15.57:0/54084993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5501061f0 0x7ff55010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 --2- 172.21.15.57:0/54084993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501010d0 0x7ff5501014b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 --2- 172.21.15.57:0/54084993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff550101a80 0x7ff550105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 -- 172.21.15.57:0/54084993 >> 172.21.15.57:0/54084993 conn(0x7ff5500fc950 msgr2=0x7ff5500fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 -- 172.21.15.57:0/54084993 shutdown_connections 2023-10-06T22:25:58.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.739+0000 7ff5583ff700 1 -- 172.21.15.57:0/54084993 wait complete. 2023-10-06T22:25:58.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.740+0000 7ff5583ff700 1 Processor -- start 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.740+0000 7ff5583ff700 1 -- start start 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff5583ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5501010d0 0x7ff550073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff5583ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501061f0 0x7ff5500738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff5583ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5501a3120 0x7ff5501a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff5583ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501a5a50 con 0x7ff5501a3120 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff5583ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501a5b90 con 0x7ff5501061f0 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff5583ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501a5d00 con 0x7ff5501010d0 2023-10-06T22:25:58.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff55599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501061f0 0x7ff5500738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff55599a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501061f0 0x7ff5500738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:54852/0 (socket says 172.21.15.57:54852) 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff55599a700 1 -- 172.21.15.57:0/2207718825 learned_addr learned my addr 172.21.15.57:0/2207718825 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff55699c700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5501a3120 0x7ff5501a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.741+0000 7ff55619b700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5501010d0 0x7ff550073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff55599a700 1 -- 172.21.15.57:0/2207718825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5501010d0 msgr2=0x7ff550073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff55599a700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5501010d0 0x7ff550073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff55599a700 1 -- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5501a3120 msgr2=0x7ff5501a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff55599a700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5501a3120 0x7ff5501a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:58.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff55599a700 1 -- 172.21.15.57:0/2207718825 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0095f0 con 0x7ff5501061f0 2023-10-06T22:25:58.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff55599a700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501061f0 0x7ff5500738d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff54000e970 tx=0x7ff54000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:58.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff5477fe700 1 -- 172.21.15.57:0/2207718825 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540009790 con 0x7ff5501061f0 2023-10-06T22:25:58.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff5477fe700 1 -- 172.21.15.57:0/2207718825 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54000c7e0 con 0x7ff5501061f0 2023-10-06T22:25:58.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5501a5fe0 con 0x7ff5501061f0 2023-10-06T22:25:58.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.742+0000 7ff5477fe700 1 -- 172.21.15.57:0/2207718825 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400054e0 con 0x7ff5501061f0 2023-10-06T22:25:58.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.743+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5501a6460 con 0x7ff5501061f0 2023-10-06T22:25:58.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.744+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5501005a0 con 0x7ff5501061f0 2023-10-06T22:25:58.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.746+0000 7ff5477fe700 1 -- 172.21.15.57:0/2207718825 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff54000c950 con 0x7ff5501061f0 2023-10-06T22:25:58.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.747+0000 7ff5477fe700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff53c0a15f0 0x7ff53c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:58.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.748+0000 7ff5477fe700 1 -- 172.21.15.57:0/2207718825 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7ff540014070 con 0x7ff5501061f0 2023-10-06T22:25:58.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.748+0000 7ff55619b700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff53c0a15f0 0x7ff53c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:58.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.748+0000 7ff55619b700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff53c0a15f0 0x7ff53c0a3ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00b2e0 tx=0x7ff54c005960 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:58.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:58.749+0000 7ff5477fe700 1 -- 172.21.15.57:0/2207718825 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff540096870 con 0x7ff5501061f0 2023-10-06T22:25:59.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.013+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 --> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff550101b20 con 0x7ff53c0a15f0 2023-10-06T22:25:59.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.022+0000 7ff5477fe700 1 -- 172.21.15.57:0/2207718825 <== mgr.25414 v2:172.21.15.57:6800/3210017833 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ff550101b20 con 0x7ff53c0a15f0 2023-10-06T22:25:59.024 INFO:teuthology.orchestra.run.smithi057.stdout:/ 1m 6m3h 2023-10-06T22:25:59.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff53c0a15f0 msgr2=0x7ff53c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff53c0a15f0 0x7ff53c0a3ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00b2e0 tx=0x7ff54c005960 comp rx=0 tx=0).stop 2023-10-06T22:25:59.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501061f0 msgr2=0x7ff5500738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501061f0 0x7ff5500738d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff54000e970 tx=0x7ff54000ec80 comp rx=0 tx=0).stop 2023-10-06T22:25:59.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 shutdown_connections 2023-10-06T22:25:59.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff53c0a15f0 0x7ff53c0a3ab0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5501a3120 0x7ff5501a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5501010d0 0x7ff550073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 --2- 172.21.15.57:0/2207718825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5501061f0 0x7ff5500738d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 >> 172.21.15.57:0/2207718825 conn(0x7ff5500fc950 msgr2=0x7ff5500fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:59.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 shutdown_connections 2023-10-06T22:25:59.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.024+0000 7ff5583ff700 1 -- 172.21.15.57:0/2207718825 wait complete. 2023-10-06T22:25:59.049 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-06T22:25:59.063 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-06T22:25:59.063 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-10-06T22:25:59.064 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-06T22:25:59.110 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:25:59.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 -- 172.21.15.57:0/1499742169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c1010e0 msgr2=0x7fb44c1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 --2- 172.21.15.57:0/1499742169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c1010e0 0x7fb44c1014c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009990 tx=0x7fb43c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 -- 172.21.15.57:0/1499742169 shutdown_connections 2023-10-06T22:25:59.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 --2- 172.21.15.57:0/1499742169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c106200 0x7fb44c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 --2- 172.21.15.57:0/1499742169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c1010e0 0x7fb44c1014c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 --2- 172.21.15.57:0/1499742169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb44c101a90 0x7fb44c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 -- 172.21.15.57:0/1499742169 >> 172.21.15.57:0/1499742169 conn(0x7fb44c0fc980 msgr2=0x7fb44c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:59.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 -- 172.21.15.57:0/1499742169 shutdown_connections 2023-10-06T22:25:59.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.329+0000 7fb4527ff700 1 -- 172.21.15.57:0/1499742169 wait complete. 2023-10-06T22:25:59.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 Processor -- start 2023-10-06T22:25:59.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 -- start start 2023-10-06T22:25:59.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c1010e0 0x7fb44c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c106200 0x7fb44c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb44c19db80 0x7fb44c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c19e600 con 0x7fb44c106200 2023-10-06T22:25:59.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c19e770 con 0x7fb44c1010e0 2023-10-06T22:25:59.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.330+0000 7fb4527ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c19e930 con 0x7fb44c19db80 2023-10-06T22:25:59.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c1010e0 0x7fb44c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c106200 0x7fb44c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c1010e0 0x7fb44c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:54864/0 (socket says 172.21.15.57:54864) 2023-10-06T22:25:59.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 -- 172.21.15.57:0/321491152 learned_addr learned my addr 172.21.15.57:0/321491152 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:59.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb450d9c700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb44c19db80 0x7fb44c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 -- 172.21.15.57:0/321491152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb44c19db80 msgr2=0x7fb44c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb44c19db80 0x7fb44c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 -- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c106200 msgr2=0x7fb44c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c106200 0x7fb44c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44bfff700 1 -- 172.21.15.57:0/321491152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0095f0 con 0x7fb44c1010e0 2023-10-06T22:25:59.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.331+0000 7fb44b7fe700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c106200 0x7fb44c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:25:59.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.332+0000 7fb44bfff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c1010e0 0x7fb44c1991e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb43c005910 tx=0x7fb43c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:59.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.332+0000 7fb4497fa700 1 -- 172.21.15.57:0/321491152 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c01d070 con 0x7fb44c1010e0 2023-10-06T22:25:59.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.332+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c1a5e30 con 0x7fb44c1010e0 2023-10-06T22:25:59.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.332+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c1a6250 con 0x7fb44c1010e0 2023-10-06T22:25:59.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.332+0000 7fb4497fa700 1 -- 172.21.15.57:0/321491152 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb43c022470 con 0x7fb44c1010e0 2023-10-06T22:25:59.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.332+0000 7fb4497fa700 1 -- 172.21.15.57:0/321491152 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c017610 con 0x7fb44c1010e0 2023-10-06T22:25:59.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.333+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb438005320 con 0x7fb44c1010e0 2023-10-06T22:25:59.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.336+0000 7fb4497fa700 1 -- 172.21.15.57:0/321491152 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb43c00fb10 con 0x7fb44c1010e0 2023-10-06T22:25:59.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.337+0000 7fb4497fa700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4340a1630 0x7fb4340a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.337+0000 7fb4497fa700 1 -- 172.21.15.57:0/321491152 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fb43c0ce230 con 0x7fb44c1010e0 2023-10-06T22:25:59.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.338+0000 7fb44b7fe700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4340a1630 0x7fb4340a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.338+0000 7fb44b7fe700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4340a1630 0x7fb4340a3af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb44c19a7f0 tx=0x7fb440009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:59.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.339+0000 7fb4497fa700 1 -- 172.21.15.57:0/321491152 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb43c0975a0 con 0x7fb44c1010e0 2023-10-06T22:25:59.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.557+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4380058a0 con 0x7fb44c1010e0 2023-10-06T22:25:59.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.559+0000 7fb4497fa700 1 -- 172.21.15.57:0/321491152 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+22978 (secure 0 0 0) 0x7fb43c09abc0 con 0x7fb44c1010e0 2023-10-06T22:25:59.562 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:25:59.562 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":98,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:25:36.118582+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/339414861":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/117215786":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/409950778":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6800/409950778":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2575175349":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3940203258":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:25:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4340a1630 msgr2=0x7fb4340a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4340a1630 0x7fb4340a3af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb44c19a7f0 tx=0x7fb440009260 comp rx=0 tx=0).stop 2023-10-06T22:25:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c1010e0 msgr2=0x7fb44c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c1010e0 0x7fb44c1991e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb43c005910 tx=0x7fb43c004730 comp rx=0 tx=0).stop 2023-10-06T22:25:59.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 shutdown_connections 2023-10-06T22:25:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fb4340a1630 0x7fb4340a3af0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb44c19db80 0x7fb44c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb44c1010e0 0x7fb44c1991e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 --2- 172.21.15.57:0/321491152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb44c106200 0x7fb44c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 >> 172.21.15.57:0/321491152 conn(0x7fb44c0fc980 msgr2=0x7fb44c10d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 shutdown_connections 2023-10-06T22:25:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.562+0000 7fb4527ff700 1 -- 172.21.15.57:0/321491152 wait complete. 2023-10-06T22:25:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:59 smithi190 ceph-mon[134891]: pgmap v351: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 385 op/s 2023-10-06T22:25:59.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:25:59 smithi190 ceph-mon[134891]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:25:59.580 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:59 smithi057 ceph-mon[98492]: pgmap v351: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 385 op/s 2023-10-06T22:25:59.581 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:25:59 smithi057 ceph-mon[98492]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:25:59.583 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:25:59.584 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:25:59.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:59 smithi156 ceph-mon[134358]: pgmap v351: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 385 op/s 2023-10-06T22:25:59.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:25:59 smithi156 ceph-mon[134358]: from='client.17169 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 -- 172.21.15.57:0/2090788155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a81061d0 msgr2=0x7f29a810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 --2- 172.21.15.57:0/2090788155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a81061d0 0x7f29a810dd50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009f90 tx=0x7f29a000b330 comp rx=0 tx=0).stop 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 -- 172.21.15.57:0/2090788155 shutdown_connections 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 --2- 172.21.15.57:0/2090788155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a81061d0 0x7f29a810dd50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 --2- 172.21.15.57:0/2090788155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29a81010b0 0x7f29a8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 --2- 172.21.15.57:0/2090788155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a8101a60 0x7f29a8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 -- 172.21.15.57:0/2090788155 >> 172.21.15.57:0/2090788155 conn(0x7f29a80fc950 msgr2=0x7f29a80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 -- 172.21.15.57:0/2090788155 shutdown_connections 2023-10-06T22:25:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.800+0000 7f29affff700 1 -- 172.21.15.57:0/2090788155 wait complete. 2023-10-06T22:25:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.801+0000 7f29affff700 1 Processor -- start 2023-10-06T22:25:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.801+0000 7f29affff700 1 -- start start 2023-10-06T22:25:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29a81010b0 0x7f29a81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29affff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a81061d0 0x7f29a8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29affff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a819dc70 0x7f29a819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29affff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a819e6f0 con 0x7f29a819dc70 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29affff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a819e860 con 0x7f29a81010b0 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29affff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a819ea20 con 0x7f29a81061d0 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a819dc70 0x7f29a819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a819dc70 0x7f29a819e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46858/0 (socket says 172.21.15.57:46858) 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 -- 172.21.15.57:0/4175311332 learned_addr learned my addr 172.21.15.57:0/4175311332 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29add9b700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29a81010b0 0x7f29a81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ad59a700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a81061d0 0x7f29a8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 -- 172.21.15.57:0/4175311332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a81061d0 msgr2=0x7f29a8199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a81061d0 0x7f29a8199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 -- 172.21.15.57:0/4175311332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29a81010b0 msgr2=0x7f29a81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:25:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29a81010b0 0x7f29a81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.802+0000 7f29ae59c700 1 -- 172.21.15.57:0/4175311332 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009c40 con 0x7f29a819dc70 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.803+0000 7f29ae59c700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a819dc70 0x7f29a819e0f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009f60 tx=0x7f29a0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.803+0000 7f299effd700 1 -- 172.21.15.57:0/4175311332 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000e040 con 0x7f29a819dc70 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.803+0000 7f299effd700 1 -- 172.21.15.57:0/4175311332 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a0008e80 con 0x7f29a819dc70 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.803+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a81a5f20 con 0x7f29a819dc70 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.803+0000 7f299effd700 1 -- 172.21.15.57:0/4175311332 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a00052a0 con 0x7f29a819dc70 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.803+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a81a63a0 con 0x7f29a819dc70 2023-10-06T22:25:59.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.804+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a8067c60 con 0x7f29a819dc70 2023-10-06T22:25:59.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.806+0000 7f299effd700 1 -- 172.21.15.57:0/4175311332 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f29a001d470 con 0x7f29a819dc70 2023-10-06T22:25:59.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.807+0000 7f299effd700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29940a15f0 0x7f29940a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:25:59.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.807+0000 7f299effd700 1 -- 172.21.15.57:0/4175311332 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f29a00ce520 con 0x7f29a819dc70 2023-10-06T22:25:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.808+0000 7f29add9b700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29940a15f0 0x7f29940a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:25:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.808+0000 7f29add9b700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29940a15f0 0x7f29940a3ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29a4005fd0 tx=0x7f29a4009310 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:25:59.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:25:59.811+0000 7f299effd700 1 -- 172.21.15.57:0/4175311332 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f29a0097890 con 0x7f29a819dc70 2023-10-06T22:26:00.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.026+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29a8101ba0 con 0x7f29a819dc70 2023-10-06T22:26:00.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.026+0000 7f299effd700 1 -- 172.21.15.57:0/4175311332 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29a009aeb0 con 0x7f29a819dc70 2023-10-06T22:26:00.028 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:26:00.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29940a15f0 msgr2=0x7f29940a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29940a15f0 0x7f29940a3ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29a4005fd0 tx=0x7f29a4009310 comp rx=0 tx=0).stop 2023-10-06T22:26:00.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a819dc70 msgr2=0x7f29a819e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a819dc70 0x7f29a819e0f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009f60 tx=0x7f29a0004c80 comp rx=0 tx=0).stop 2023-10-06T22:26:00.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 shutdown_connections 2023-10-06T22:26:00.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f29940a15f0 0x7f29940a3ab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a819dc70 0x7f29a819e0f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29a81010b0 0x7f29a81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.028+0000 7f29affff700 1 --2- 172.21.15.57:0/4175311332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a81061d0 0x7f29a8199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.029+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 >> 172.21.15.57:0/4175311332 conn(0x7f29a80fc950 msgr2=0x7f29a80fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:00.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.029+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 shutdown_connections 2023-10-06T22:26:00.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.029+0000 7f29affff700 1 -- 172.21.15.57:0/4175311332 wait complete. 2023-10-06T22:26:00.050 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:26:00.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 -- 172.21.15.57:0/774743915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee8073c00 msgr2=0x7f2ee810d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 --2- 172.21.15.57:0/774743915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee8073c00 0x7f2ee810d140 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098f0 tx=0x7f2edc009c00 comp rx=0 tx=0).stop 2023-10-06T22:26:00.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 -- 172.21.15.57:0/774743915 shutdown_connections 2023-10-06T22:26:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 --2- 172.21.15.57:0/774743915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ee810d680 0x7f2ee810faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 --2- 172.21.15.57:0/774743915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee80732e0 0x7f2ee80736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 --2- 172.21.15.57:0/774743915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee8073c00 0x7f2ee810d140 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 -- 172.21.15.57:0/774743915 >> 172.21.15.57:0/774743915 conn(0x7f2ee80fc950 msgr2=0x7f2ee80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 -- 172.21.15.57:0/774743915 shutdown_connections 2023-10-06T22:26:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.266+0000 7f2eedbff700 1 -- 172.21.15.57:0/774743915 wait complete. 2023-10-06T22:26:00.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.267+0000 7f2eedbff700 1 Processor -- start 2023-10-06T22:26:00.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2eedbff700 1 -- start start 2023-10-06T22:26:00.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2eedbff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee80732e0 0x7f2ee8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2eedbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee8073c00 0x7f2ee8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2eedbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ee810d680 0x7f2ee819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2eedbff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee819e200 con 0x7f2ee80732e0 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2eedbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee819e370 con 0x7f2ee810d680 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2eedbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee819e530 con 0x7f2ee8073c00 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.268+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee80732e0 0x7f2ee8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee80732e0 0x7f2ee8199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46868/0 (socket says 172.21.15.57:46868) 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 -- 172.21.15.57:0/2005195517 learned_addr learned my addr 172.21.15.57:0/2005195517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee6ffd700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee8073c00 0x7f2ee8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee7fff700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ee810d680 0x7f2ee819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 -- 172.21.15.57:0/2005195517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee8073c00 msgr2=0x7f2ee8199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee8073c00 0x7f2ee8199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 -- 172.21.15.57:0/2005195517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ee810d680 msgr2=0x7f2ee819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ee810d680 0x7f2ee819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 -- 172.21.15.57:0/2005195517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc009520 con 0x7f2ee80732e0 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee6ffd700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee8073c00 0x7f2ee8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.269+0000 7f2ee77fe700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee80732e0 0x7f2ee8199220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800e9c0 tx=0x7f2ed800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.270+0000 7f2ee4ff9700 1 -- 172.21.15.57:0/2005195517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8004a30 con 0x7f2ee80732e0 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.270+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee819e810 con 0x7f2ee80732e0 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.270+0000 7f2ee4ff9700 1 -- 172.21.15.57:0/2005195517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ed800cc80 con 0x7f2ee80732e0 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.270+0000 7f2ee4ff9700 1 -- 172.21.15.57:0/2005195517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80054e0 con 0x7f2ee80732e0 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.270+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee81a6050 con 0x7f2ee80732e0 2023-10-06T22:26:00.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.271+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee8067c60 con 0x7f2ee80732e0 2023-10-06T22:26:00.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.275+0000 7f2ee4ff9700 1 -- 172.21.15.57:0/2005195517 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2ed8004b90 con 0x7f2ee80732e0 2023-10-06T22:26:00.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.277+0000 7f2ee4ff9700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2ed00a15f0 0x7f2ed00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.277+0000 7f2ee4ff9700 1 -- 172.21.15.57:0/2005195517 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f2ed8014070 con 0x7f2ee80732e0 2023-10-06T22:26:00.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.277+0000 7f2ee6ffd700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2ed00a15f0 0x7f2ed00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.278+0000 7f2ee6ffd700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2ed00a15f0 0x7f2ed00a3ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005e90 tx=0x7f2edc019040 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:00.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.282+0000 7f2ee4ff9700 1 -- 172.21.15.57:0/2005195517 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ed80964d0 con 0x7f2ee80732e0 2023-10-06T22:26:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.510+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ee80008d0 con 0x7f2ee80732e0 2023-10-06T22:26:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.510+0000 7f2ee4ff9700 1 -- 172.21.15.57:0/2005195517 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ed8099af0 con 0x7f2ee80732e0 2023-10-06T22:26:00.512 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:26:00.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2ed00a15f0 msgr2=0x7f2ed00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2ed00a15f0 0x7f2ed00a3ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005e90 tx=0x7f2edc019040 comp rx=0 tx=0).stop 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee80732e0 msgr2=0x7f2ee8199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee80732e0 0x7f2ee8199220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800e9c0 tx=0x7f2ed800ecd0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 shutdown_connections 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f2ed00a15f0 0x7f2ed00a3ab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ee810d680 0x7f2ee819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ee80732e0 0x7f2ee8199220 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 --2- 172.21.15.57:0/2005195517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee8073c00 0x7f2ee8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 >> 172.21.15.57:0/2005195517 conn(0x7f2ee80fc950 msgr2=0x7f2ee810f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 shutdown_connections 2023-10-06T22:26:00.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.513+0000 7f2eedbff700 1 -- 172.21.15.57:0/2005195517 wait complete. 2023-10-06T22:26:00.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:26:00.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:00 smithi190 ceph-mon[134891]: from='client.27182 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:26:00.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:00 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/321491152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:00.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:00 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/4175311332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:26:00.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:00 smithi057 ceph-mon[98492]: from='client.27182 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:26:00.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:00 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/321491152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:00.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:00 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/4175311332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:26:00.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:00 smithi156 ceph-mon[134358]: from='client.27182 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T22:26:00.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:00 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/321491152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:00.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:00 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/4175311332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:26:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 -- 172.21.15.57:0/2959322180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5980101a70 msgr2=0x7f5980105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 --2- 172.21.15.57:0/2959322180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5980101a70 0x7f5980105ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5974009990 tx=0x7f5974009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 -- 172.21.15.57:0/2959322180 shutdown_connections 2023-10-06T22:26:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 --2- 172.21.15.57:0/2959322180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59801061e0 0x7f598010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 --2- 172.21.15.57:0/2959322180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59801010c0 0x7f59801014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 --2- 172.21.15.57:0/2959322180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5980101a70 0x7f5980105ab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 -- 172.21.15.57:0/2959322180 >> 172.21.15.57:0/2959322180 conn(0x7f59800fc960 msgr2=0x7f59800fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 -- 172.21.15.57:0/2959322180 shutdown_connections 2023-10-06T22:26:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.785+0000 7f59849ff700 1 -- 172.21.15.57:0/2959322180 wait complete. 2023-10-06T22:26:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 Processor -- start 2023-10-06T22:26:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 -- start start 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59801010c0 0x7f5980199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59801061e0 0x7f5980199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f598019daf0 0x7f598019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598019e570 con 0x7f598019daf0 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598019e6e0 con 0x7f59801061e0 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.786+0000 7f59849ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598019e8a0 con 0x7f59801010c0 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59801061e0 0x7f5980199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59801010c0 0x7f5980199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59801061e0 0x7f5980199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:54922/0 (socket says 172.21.15.57:54922) 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 -- 172.21.15.57:0/1592906514 learned_addr learned my addr 172.21.15.57:0/1592906514 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597ed9d700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f598019daf0 0x7f598019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 -- 172.21.15.57:0/1592906514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59801010c0 msgr2=0x7f5980199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59801010c0 0x7f5980199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 -- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f598019daf0 msgr2=0x7f598019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:00.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f598019daf0 0x7f598019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:00.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597dd9b700 1 -- 172.21.15.57:0/1592906514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740095f0 con 0x7f59801061e0 2023-10-06T22:26:00.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.787+0000 7f597e59c700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59801010c0 0x7f5980199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:00.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.788+0000 7f597dd9b700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59801061e0 0x7f5980199690 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5974000c00 tx=0x7f5974005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.788+0000 7f596f7fe700 1 -- 172.21.15.57:0/1592906514 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597401d070 con 0x7f59801061e0 2023-10-06T22:26:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.788+0000 7f596f7fe700 1 -- 172.21.15.57:0/1592906514 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5974021410 con 0x7f59801061e0 2023-10-06T22:26:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.788+0000 7f596f7fe700 1 -- 172.21.15.57:0/1592906514 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974017650 con 0x7f59801061e0 2023-10-06T22:26:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.788+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59801a5da0 con 0x7f59801061e0 2023-10-06T22:26:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.788+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59801a61d0 con 0x7f59801061e0 2023-10-06T22:26:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.789+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59800673d0 con 0x7f59801061e0 2023-10-06T22:26:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.791+0000 7f596f7fe700 1 -- 172.21.15.57:0/1592906514 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5974021580 con 0x7f59801061e0 2023-10-06T22:26:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.793+0000 7f596f7fe700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f59680a15f0 0x7f59680a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.793+0000 7f596f7fe700 1 -- 172.21.15.57:0/1592906514 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f59740ce380 con 0x7f59801061e0 2023-10-06T22:26:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.793+0000 7f597e59c700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f59680a15f0 0x7f59680a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.794+0000 7f597e59c700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f59680a15f0 0x7f59680a3ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5970011490 tx=0x7f5970009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:00.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:00.796+0000 7f596f7fe700 1 -- 172.21.15.57:0/1592906514 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f59740976f0 con 0x7f59801061e0 2023-10-06T22:26:01.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.010+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5980101bf0 con 0x7f59801061e0 2023-10-06T22:26:01.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.011+0000 7f596f7fe700 1 -- 172.21.15.57:0/1592906514 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f597409ad10 con 0x7f59801061e0 2023-10-06T22:26:01.013 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f59680a15f0 msgr2=0x7f59680a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f59680a15f0 0x7f59680a3ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5970011490 tx=0x7f5970009260 comp rx=0 tx=0).stop 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59801061e0 msgr2=0x7f5980199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59801061e0 0x7f5980199690 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5974000c00 tx=0x7f5974005740 comp rx=0 tx=0).stop 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 shutdown_connections 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f59680a15f0 0x7f59680a3ab0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f598019daf0 0x7f598019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59801010c0 0x7f5980199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 --2- 172.21.15.57:0/1592906514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59801061e0 0x7f5980199690 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.013+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 >> 172.21.15.57:0/1592906514 conn(0x7f59800fc960 msgr2=0x7f59800fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.014+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 shutdown_connections 2023-10-06T22:26:01.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.014+0000 7f59849ff700 1 -- 172.21.15.57:0/1592906514 wait complete. 2023-10-06T22:26:01.035 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-06T22:26:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 -- 172.21.15.57:0/527102254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 msgr2=0x7fc5b0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/527102254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 0x7fc5b0101490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009990 tx=0x7fc5a0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 -- 172.21.15.57:0/527102254 shutdown_connections 2023-10-06T22:26:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/527102254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 0x7fc5b010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/527102254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 0x7fc5b0101490 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/527102254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b0101a60 0x7fc5b0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 -- 172.21.15.57:0/527102254 >> 172.21.15.57:0/527102254 conn(0x7fc5b00fc8d0 msgr2=0x7fc5b00fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 -- 172.21.15.57:0/527102254 shutdown_connections 2023-10-06T22:26:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.243+0000 7fc5b49ff700 1 -- 172.21.15.57:0/527102254 wait complete. 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 Processor -- start 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 -- start start 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 0x7fc5b0196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 0x7fc5b0197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b019b8e0 0x7fc5b019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b019c360 con 0x7fc5b01061d0 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b019c4d0 con 0x7fc5b019b8e0 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.244+0000 7fc5b49ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b019c690 con 0x7fc5b01010b0 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 0x7fc5b0197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5ae59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 0x7fc5b0196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 0x7fc5b0197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46902/0 (socket says 172.21.15.57:46902) 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 -- 172.21.15.57:0/1996657946 learned_addr learned my addr 172.21.15.57:0/1996657946 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5aed9d700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b019b8e0 0x7fc5b019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 -- 172.21.15.57:0/1996657946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 msgr2=0x7fc5b0196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 0x7fc5b0196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 -- 172.21.15.57:0/1996657946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b019b8e0 msgr2=0x7fc5b019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b019b8e0 0x7fc5b019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 -- 172.21.15.57:0/1996657946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00095f0 con 0x7fc5b01061d0 2023-10-06T22:26:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5ae59c700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 0x7fc5b0196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.245+0000 7fc5a7fff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 0x7fc5b0197480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc59800e970 tx=0x7fc59800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.246+0000 7fc5a77fe700 1 -- 172.21.15.57:0/1996657946 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598009790 con 0x7fc5b01061d0 2023-10-06T22:26:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.246+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b01a3bf0 con 0x7fc5b01061d0 2023-10-06T22:26:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.246+0000 7fc5a77fe700 1 -- 172.21.15.57:0/1996657946 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc59800c7e0 con 0x7fc5b01061d0 2023-10-06T22:26:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.246+0000 7fc5a77fe700 1 -- 172.21.15.57:0/1996657946 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980054e0 con 0x7fc5b01061d0 2023-10-06T22:26:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.246+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b01a4120 con 0x7fc5b01061d0 2023-10-06T22:26:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.247+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b0067cd0 con 0x7fc5b01061d0 2023-10-06T22:26:01.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.249+0000 7fc5a77fe700 1 -- 172.21.15.57:0/1996657946 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc59800c950 con 0x7fc5b01061d0 2023-10-06T22:26:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.250+0000 7fc5a77fe700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc59c0a1630 0x7fc59c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.250+0000 7fc5a77fe700 1 -- 172.21.15.57:0/1996657946 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8492+0+0 (secure 0 0 0) 0x7fc598014070 con 0x7fc5b01061d0 2023-10-06T22:26:01.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.251+0000 7fc5ae59c700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc59c0a1630 0x7fc59c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:01.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.251+0000 7fc5ae59c700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc59c0a1630 0x7fc59c0a3af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b2e0 tx=0x7fc5a0005960 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:01.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.252+0000 7fc5a77fe700 1 -- 172.21.15.57:0/1996657946 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc598096870 con 0x7fc5b01061d0 2023-10-06T22:26:01.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.544+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fc5b0101cb0 con 0x7fc5b01061d0 2023-10-06T22:26:01.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.561+0000 7fc5a77fe700 1 -- 172.21.15.57:0/1996657946 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) v1 ==== 81+0+734408 (secure 0 0 0) 0x7fc5b0101cb0 con 0x7fc5b01061d0 2023-10-06T22:26:01.573 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:26:01.573 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:26:01.573 INFO:teuthology.orchestra.run.smithi057.stdout: "epoch": 27, 2023-10-06T22:26:01.573 INFO:teuthology.orchestra.run.smithi057.stdout: "active_gid": 25414, 2023-10-06T22:26:01.573 INFO:teuthology.orchestra.run.smithi057.stdout: "active_name": "x", 2023-10-06T22:26:01.573 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addrs": { 2023-10-06T22:26:01.573 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6800", 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3210017833 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6801", 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3210017833 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:01.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addr": "172.21.15.57:6801/3210017833", 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "active_change": "2023-10-06T22:14:45.662123+0000", 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "active_mgr_features": 4540138322906710015, 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "available": true, 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "standbys": [ 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 25394, 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "y", 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540138322906710015, 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2023-10-06T22:26:01.575 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.576 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.577 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.578 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.579 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.580 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.581 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2023-10-06T22:26:01.582 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.583 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.584 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.585 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2023-10-06T22:26:01.586 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.587 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.588 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.589 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.590 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.591 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.592 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.593 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.594 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.595 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.596 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2023-10-06T22:26:01.597 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.598 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.599 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T22:26:01.600 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.601 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.602 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.603 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.604 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.605 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.607 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2023-10-06T22:26:01.608 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap" 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.609 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.610 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.611 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2023-10-06T22:26:01.612 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.613 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.614 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2023-10-06T22:26:01.615 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.616 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2023-10-06T22:26:01.617 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.618 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.619 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T22:26:01.620 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.621 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2023-10-06T22:26:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.624 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.625 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.626 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.627 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T22:26:01.628 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.629 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.630 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.631 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.632 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.633 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.634 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.635 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T22:26:01.636 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.637 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.638 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2023-10-06T22:26:01.639 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.640 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.641 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2023-10-06T22:26:01.642 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.643 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.644 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.645 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2023-10-06T22:26:01.646 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.647 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.648 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T22:26:01.649 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.650 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:01.651 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.652 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.653 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.654 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.655 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.656 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.657 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.658 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.659 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.660 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.661 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2023-10-06T22:26:01.662 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.663 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.664 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.665 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.666 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.667 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.668 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2023-10-06T22:26:01.669 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.670 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.671 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.672 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.673 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.674 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2023-10-06T22:26:01.675 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.676 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T22:26:01.677 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.678 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.679 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.680 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2023-10-06T22:26:01.681 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.682 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.683 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.684 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.685 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.686 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.687 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.688 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.689 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.690 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.692 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.693 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.694 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.695 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T22:26:01.696 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T22:26:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.699 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.700 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.701 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.702 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.703 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T22:26:01.704 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.706 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.707 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:01.708 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.709 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T22:26:01.711 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.712 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.713 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.714 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.715 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.716 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.717 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.718 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.719 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.720 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.721 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.722 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.723 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.724 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T22:26:01.725 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.726 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.727 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.728 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.729 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.730 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.731 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T22:26:01.732 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.733 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.734 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.735 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.736 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.737 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T22:26:01.738 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.740 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.741 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.742 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.744 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.746 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.747 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.748 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.749 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.750 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.751 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.752 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.753 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.754 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2023-10-06T22:26:01.755 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.756 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.757 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.758 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.759 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2023-10-06T22:26:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.762 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2023-10-06T22:26:01.763 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-06T22:26:01.764 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.765 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-06T22:26:01.766 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.767 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.768 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.769 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.770 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.771 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.773 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T22:26:01.774 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.775 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.776 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-06T22:26:01.777 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.778 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2023-10-06T22:26:01.780 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.781 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.782 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.783 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.784 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.785 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.787 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.788 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.789 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.790 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.792 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.793 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.794 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.795 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.796 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.797 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.798 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.799 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.800 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.801 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.802 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.803 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.804 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2023-10-06T22:26:01.806 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2023-10-06T22:26:01.807 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.808 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.809 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.810 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.811 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.812 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.813 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.814 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.815 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.816 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.817 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.819 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.821 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.822 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.823 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.824 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.825 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.826 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.827 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.828 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2023-10-06T22:26:01.830 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.831 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.832 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.833 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.834 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.836 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.837 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.838 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T22:26:01.839 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.840 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2023-10-06T22:26:01.842 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.843 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.844 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2023-10-06T22:26:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.849 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.850 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.851 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.853 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.854 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.855 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.857 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.858 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.859 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.860 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.861 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.862 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.865 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.866 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.867 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.868 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.869 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.870 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.871 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.872 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.873 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.874 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.875 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.876 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.877 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.878 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.879 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.880 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.881 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.882 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.883 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.884 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.885 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.886 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.887 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:01.888 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.889 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.890 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.891 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.892 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.893 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.894 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.895 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.896 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.897 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.898 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.899 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:01.900 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.901 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.902 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.903 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.904 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.905 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.906 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.907 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.908 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.909 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.910 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.914 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.915 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-06T22:26:01.916 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.917 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.918 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.919 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.920 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.921 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.922 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.923 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.924 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.925 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.926 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2023-10-06T22:26:01.927 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.929 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.930 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.932 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.933 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.934 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.935 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.936 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.937 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.938 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.939 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.940 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.941 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.942 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.943 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.944 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "drive_group_interval": { 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "drive_group_interval", 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300.0", 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.946 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.947 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.948 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.949 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.950 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.951 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.954 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.955 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.956 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.957 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.958 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.959 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.960 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.961 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.962 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.963 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.965 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.966 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.967 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.968 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.970 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.971 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.972 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2023-10-06T22:26:01.973 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.974 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.975 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.976 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.977 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.978 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.979 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.980 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.981 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.982 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.983 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.984 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.985 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.986 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.987 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.988 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.989 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.990 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.991 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:01.992 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.993 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.994 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2023-10-06T22:26:01.995 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:01.996 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.997 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:01.998 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:01.999 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.000 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.001 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.004 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.005 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.006 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.007 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.008 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.009 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2023-10-06T22:26:02.010 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2023-10-06T22:26:02.011 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.013 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.014 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.015 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.016 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.017 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.018 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.019 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.020 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.021 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.022 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.023 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.024 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.025 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T22:26:02.026 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.027 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.028 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.029 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.030 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.031 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.032 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-06T22:26:02.033 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.034 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.035 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.036 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.037 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2023-10-06T22:26:02.039 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2023-10-06T22:26:02.040 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.041 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.042 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.043 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2023-10-06T22:26:02.044 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.045 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.046 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.047 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.048 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.049 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.050 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.051 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.052 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.053 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.054 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.055 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 25411, 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "z", 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540138322906710015, 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.056 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.058 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.059 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.060 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.061 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.062 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2023-10-06T22:26:02.064 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.065 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.066 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.067 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2023-10-06T22:26:02.068 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.069 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.070 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.072 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.073 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T22:26:02.074 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-06T22:26:02.075 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2023-10-06T22:26:02.076 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.077 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.078 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2023-10-06T22:26:02.079 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.080 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.081 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.082 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.083 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.084 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.085 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-06T22:26:02.088 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap" 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.089 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.090 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.091 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.092 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.093 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.094 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.095 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.096 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.097 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.098 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2023-10-06T22:26:02.099 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.100 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2023-10-06T22:26:02.102 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.103 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.104 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2023-10-06T22:26:02.105 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.106 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.107 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2023-10-06T22:26:02.109 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.110 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.111 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T22:26:02.112 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.113 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.114 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2023-10-06T22:26:02.115 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.116 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.117 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.118 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2023-10-06T22:26:02.119 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.120 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.121 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T22:26:02.122 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.123 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.124 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T22:26:02.125 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2023-10-06T22:26:02.126 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.127 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.128 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.129 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.131 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.132 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.133 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.134 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T22:26:02.135 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.136 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.137 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.138 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.139 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T22:26:02.140 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.141 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.142 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.143 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.144 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-06T22:26:02.145 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.146 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T22:26:02.147 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.148 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.149 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.150 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.151 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.152 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2023-10-06T22:26:02.153 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.154 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.155 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.156 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.157 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.158 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.160 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.161 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.162 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.163 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.164 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.165 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.166 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.167 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.169 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2023-10-06T22:26:02.170 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.171 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.172 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.173 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.174 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.176 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.177 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T22:26:02.178 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.179 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.180 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.181 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.182 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.183 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.184 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.185 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.186 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.187 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T22:26:02.188 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.189 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.190 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.191 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.192 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.193 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.194 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.195 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.196 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T22:26:02.197 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.198 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.200 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.201 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.202 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.203 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.204 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.205 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T22:26:02.206 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T22:26:02.207 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.209 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.210 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.211 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.212 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.213 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.214 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.215 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.216 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T22:26:02.217 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.218 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.219 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.220 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.221 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.222 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.223 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.224 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-06T22:26:02.225 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.227 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.228 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.229 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.230 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.231 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.232 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.233 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:02.235 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.236 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2023-10-06T22:26:02.239 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-06T22:26:02.240 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.241 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-06T22:26:02.242 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2023-10-06T22:26:02.243 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.244 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.246 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.247 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.248 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2023-10-06T22:26:02.249 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.250 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.251 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.252 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T22:26:02.253 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.254 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.255 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2023-10-06T22:26:02.256 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2023-10-06T22:26:02.257 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.258 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.259 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.260 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.261 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.262 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2023-10-06T22:26:02.263 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2023-10-06T22:26:02.264 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.265 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.267 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.268 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2023-10-06T22:26:02.269 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.270 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.271 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.272 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.273 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.274 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.275 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.276 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.278 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-06T22:26:02.279 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.280 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.281 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.282 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T22:26:02.283 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.284 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.285 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.286 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.287 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.288 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.289 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.290 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.291 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.292 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.293 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.294 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.295 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.296 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.297 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.298 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.299 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.300 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.301 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.302 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.303 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.304 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.305 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.306 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.307 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.308 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.309 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.310 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.311 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.312 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T22:26:02.314 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.315 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.316 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.317 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.319 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.320 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.321 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.322 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.323 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.324 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.325 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.326 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.327 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.328 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.329 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.331 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.332 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.333 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.334 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.335 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.336 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.337 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.338 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.339 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.340 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.341 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2023-10-06T22:26:02.342 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2023-10-06T22:26:02.343 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.344 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.345 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.346 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.347 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.348 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.349 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.350 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.351 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.352 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.353 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.354 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.355 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.357 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.358 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.359 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.360 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.361 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.362 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-06T22:26:02.363 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.364 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.365 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.367 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.368 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.369 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.370 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.371 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.373 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.374 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.375 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.376 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.377 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T22:26:02.378 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.379 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.380 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.381 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.382 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.383 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2023-10-06T22:26:02.384 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T22:26:02.385 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.386 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.387 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.388 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T22:26:02.389 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.390 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.391 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.392 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.393 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.394 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.395 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.396 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.398 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2023-10-06T22:26:02.399 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.400 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.401 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.403 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-06T22:26:02.404 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.405 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.407 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.408 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.409 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.410 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.411 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.412 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2023-10-06T22:26:02.413 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.414 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.415 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.416 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.417 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.418 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.419 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "drive_group_interval": { 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "drive_group_interval", 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300.0", 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.420 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.421 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.422 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.423 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.424 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.425 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.426 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.427 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.429 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.430 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.431 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.432 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.433 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.434 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.435 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.436 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.437 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2023-10-06T22:26:02.438 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.439 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.440 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.441 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.442 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.443 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.444 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.445 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.446 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.447 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.448 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.449 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.450 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.451 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.452 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.453 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.454 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.455 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.456 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.457 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.458 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.459 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.460 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.461 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.462 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.463 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.464 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.465 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.466 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.467 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2023-10-06T22:26:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.469 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.471 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.472 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.473 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.474 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.475 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.476 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.477 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T22:26:02.478 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.479 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.480 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.481 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.482 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.483 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.484 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-06T22:26:02.485 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.486 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.487 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.488 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.489 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.490 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.491 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.492 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.493 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.494 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.495 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.496 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.497 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.498 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.499 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.501 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.502 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.503 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.504 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.505 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.506 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.507 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.508 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.509 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.510 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.511 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.512 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.513 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.514 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.515 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.516 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.517 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.518 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.519 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.520 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.521 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.522 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.523 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.524 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.525 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.526 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: "modules": [ 2023-10-06T22:26:02.528 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard", 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "iostat", 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "nfs", 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "restful", 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "snap_schedule" 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.529 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.530 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.531 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.532 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.533 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.534 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.536 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.537 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2023-10-06T22:26:02.538 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.539 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.541 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.542 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.543 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.544 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.545 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.546 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T22:26:02.547 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.548 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.549 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T22:26:02.550 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.551 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.552 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.553 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.554 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.555 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.556 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.557 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.558 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.559 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.560 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2023-10-06T22:26:02.561 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap" 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.562 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.563 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.564 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T22:26:02.565 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T22:26:02.566 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2023-10-06T22:26:02.567 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2023-10-06T22:26:02.568 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.569 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.570 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T22:26:02.571 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T22:26:02.572 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.573 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.574 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.575 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.577 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.578 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.579 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2023-10-06T22:26:02.581 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2023-10-06T22:26:02.582 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.583 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.584 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2023-10-06T22:26:02.585 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.586 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.587 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T22:26:02.588 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.589 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.590 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.591 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.593 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-06T22:26:02.594 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.595 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.596 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.597 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.598 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.599 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2023-10-06T22:26:02.600 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.601 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.602 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.603 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2023-10-06T22:26:02.604 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.605 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.606 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.607 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.608 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.609 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.611 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.612 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.613 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.614 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2023-10-06T22:26:02.615 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.616 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.617 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2023-10-06T22:26:02.618 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-06T22:26:02.619 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T22:26:02.620 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.621 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.622 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.623 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.624 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.625 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2023-10-06T22:26:02.626 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.627 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.628 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.629 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T22:26:02.630 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.631 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.632 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.633 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.634 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.635 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.636 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.637 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.639 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.640 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2023-10-06T22:26:02.641 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.642 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.644 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.645 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T22:26:02.646 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.647 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.648 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.649 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.650 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.651 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.652 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.653 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.654 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T22:26:02.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.656 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.657 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.658 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.659 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.660 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.661 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.662 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T22:26:02.663 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T22:26:02.664 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.665 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.666 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.667 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.668 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.669 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.670 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.671 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.672 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T22:26:02.673 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.674 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.675 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.676 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.677 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.678 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.679 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.680 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T22:26:02.681 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.682 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.683 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.684 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.685 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.686 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.687 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T22:26:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T22:26:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2023-10-06T22:26:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-06T22:26:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-06T22:26:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T22:26:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2023-10-06T22:26:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2023-10-06T22:26:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2023-10-06T22:26:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2023-10-06T22:26:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2023-10-06T22:26:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T22:26:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2023-10-06T22:26:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T22:26:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2023-10-06T22:26:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.840 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.841 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.842 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.843 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.844 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.846 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.847 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2023-10-06T22:26:02.848 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-06T22:26:02.849 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.850 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.851 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.852 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.853 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.854 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.855 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.856 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.857 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.859 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.860 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.861 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:02.862 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:02.863 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.865 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2023-10-06T22:26:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-06T22:26:02.867 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.868 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.869 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.870 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.871 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.872 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.873 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.874 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T22:26:02.875 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2023-10-06T22:26:02.876 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.877 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.878 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.879 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.880 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.881 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.882 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.883 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.884 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.885 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.886 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.887 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.888 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.889 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.890 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.891 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.892 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.893 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.894 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "drive_group_interval": { 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "drive_group_interval", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300.0", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.895 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.896 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.897 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.898 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.899 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.900 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.901 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.902 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.903 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.904 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.905 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.906 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.907 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.908 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2023-10-06T22:26:02.909 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2023-10-06T22:26:02.910 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.911 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.912 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.913 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.914 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.915 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.916 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.917 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.918 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.919 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2023-10-06T22:26:02.920 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.922 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.923 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.924 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.925 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.926 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.927 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.929 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.930 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.931 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.932 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.933 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.935 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.936 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.937 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.938 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.939 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.940 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.941 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.942 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.943 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.944 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.945 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.946 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.947 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.948 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.949 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.950 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.951 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T22:26:02.952 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.953 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.954 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.955 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.956 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.957 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2023-10-06T22:26:02.958 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-06T22:26:02.959 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.960 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.962 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.963 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.964 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.965 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.966 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.967 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.968 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.969 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.970 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.971 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.972 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.973 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.974 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.975 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.976 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.977 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.977 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.977 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.977 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.977 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.977 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.978 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.979 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.980 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.981 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.982 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.983 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.984 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.985 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.986 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.987 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2023-10-06T22:26:02.988 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2023-10-06T22:26:02.989 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.990 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.991 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.992 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-06T22:26:02.993 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-06T22:26:02.994 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.995 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.996 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.997 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:02.998 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:02.999 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:03.000 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:03.001 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T22:26:03.002 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "services": { 2023-10-06T22:26:03.003 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard": "https://172.21.15.57:8443/" 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "always_on_modules": { 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "octopus": [ 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-06T22:26:03.004 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "pacific": [ 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-06T22:26:03.005 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "quincy": [ 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-06T22:26:03.006 INFO:teuthology.orchestra.run.smithi057.stdout: "reef": [ 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.007 INFO:teuthology.orchestra.run.smithi057.stdout: "last_failure_osd_epoch": 86, 2023-10-06T22:26:03.008 INFO:teuthology.orchestra.run.smithi057.stdout: "active_clients": [ 2023-10-06T22:26:03.008 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.008 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-06T22:26:03.008 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-06T22:26:03.008 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.008 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-06T22:26:03.008 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 397465617 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "libcephsqlite", 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-06T22:26:03.009 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 663557046 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3821737691 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.010 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 4072417097 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.011 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1164051303 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-06T22:26:03.012 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-06T22:26:03.013 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-06T22:26:03.013 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2881140614 2023-10-06T22:26:03.013 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.013 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.013 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:03.013 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-06T22:26:03.013 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:26:03.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.568+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc59c0a1630 msgr2=0x7fc59c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:03.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.568+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc59c0a1630 0x7fc59c0a3af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b2e0 tx=0x7fc5a0005960 comp rx=0 tx=0).stop 2023-10-06T22:26:03.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.568+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 msgr2=0x7fc5b0197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:03.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.568+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 0x7fc5b0197480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc59800e970 tx=0x7fc59800ec80 comp rx=0 tx=0).stop 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 shutdown_connections 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7fc59c0a1630 0x7fc59c0a3af0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b019b8e0 0x7fc5b019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b01010b0 0x7fc5b0196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 --2- 172.21.15.57:0/1996657946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5b01061d0 0x7fc5b0197480 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 >> 172.21.15.57:0/1996657946 conn(0x7fc5b00fc8d0 msgr2=0x7fc5b00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 shutdown_connections 2023-10-06T22:26:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:01.569+0000 7fc5b49ff700 1 -- 172.21.15.57:0/1996657946 wait complete. 2023-10-06T22:26:03.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:01 smithi057 ceph-mon[98492]: pgmap v352: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2023-10-06T22:26:03.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:01 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2005195517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:26:03.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:01 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:01 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:01 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1592906514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:26:03.016 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:01 smithi156 ceph-mon[134358]: pgmap v352: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:01 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2005195517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:01 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:01 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:01 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1592906514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1996657946' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: Adjusting osd_memory_target on smithi057 to 222.0M 2023-10-06T22:26:03.017 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: Unable to set osd_memory_target on smithi057 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:02 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:01 smithi190 ceph-mon[134891]: pgmap v352: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:01 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2005195517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:01 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:01 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:01 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1592906514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1996657946' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: Adjusting osd_memory_target on smithi057 to 222.0M 2023-10-06T22:26:03.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: Unable to set osd_memory_target on smithi057 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T22:26:03.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:26:03.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:26:03.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:02 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.025 DEBUG:tasks.check_counter:Getting stats from a 2023-10-06T22:26:03.025 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1996657946' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: Adjusting osd_memory_target on smithi057 to 222.0M 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: Unable to set osd_memory_target on smithi057 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T22:26:03.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T22:26:03.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T22:26:03.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:02 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' 2023-10-06T22:26:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:26:03.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:03 smithi057 ceph-mon[98492]: pgmap v353: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 321 op/s 2023-10-06T22:26:04.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:03 smithi156 ceph-mon[134358]: pgmap v353: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 321 op/s 2023-10-06T22:26:04.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:03 smithi190 ceph-mon[134891]: pgmap v353: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 321 op/s 2023-10-06T22:26:05.419 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1318, 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 1320, 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 42502389, 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 409899, 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 8, 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.250076638, 2023-10-06T22:26:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.053809168, 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.131200212, 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.046691459, 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1318, 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.355233012, 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000269524 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1222, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 1173, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 86106709, 2023-10-06T22:26:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 424885, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.304567390, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.050701285, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.219006336, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.038897730, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1172, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.557843257, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000475975 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1961648, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 134032 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 440, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 601, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 41734011, 2023-10-06T22:26:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 584270, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 8, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 6, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.114851257, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.028208952, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.067068056, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.013146697, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 599, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.104819775, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000174991 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 449296 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 1, 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-MDSRank": { 2023-10-06T22:26:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 4363, 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.155171230, 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000035565 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "request": 0, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "reply": 0, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "reply_latency": { 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_reply": 0, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "forward": 0, 2023-10-06T22:26:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_complete": 0, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_commit": 0, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_split": 0, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_merge": 0, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes": 25973, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_top": 1346, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_bottom": 23204, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pin_tail": 1423, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pinned": 24049, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_expired": 538, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_with_caps": 0, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": 0, 2023-10-06T22:26:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "subtrees": 1, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_hit": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_forward": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_discover": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_dir_fetch": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_lock": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "load_cent": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "q": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "exported": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "exported_inodes": 0, 2023-10-06T22:26:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "imported": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "imported_inodes": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_backtrace_fetch": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rfiles": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rbytes": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rsnaps": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_inodes": 0, 2023-10-06T22:26:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_file_inodes": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_inode_file_caps": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_grant": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps_dirty": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_cap_release": 0, 2023-10-06T22:26:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "process_request_cap_release": 0 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_cache": { 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update_receipt": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_try_discover": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_send_discover": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_handle_discover": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_created": 0, 2023-10-06T22:26:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_enqueued": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_migrated": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_started": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_completed": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_exportdir": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_flush": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragmentdir": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_inodestats": 0 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_log": { 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "evlrg": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "evadd": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "evex": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "evtrm": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "ev": 47668, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "evexg": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "evexd": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "segadd": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "segex": 0, 2023-10-06T22:26:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "segtrm": 0, 2023-10-06T22:26:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "seg": 70, 2023-10-06T22:26:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "segmjr": 5, 2023-10-06T22:26:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "segexg": 0, 2023-10-06T22:26:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "segexd": 0, 2023-10-06T22:26:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "expos": 4194304, 2023-10-06T22:26:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "wrpos": 171632868, 2023-10-06T22:26:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "rdpos": 171626523, 2023-10-06T22:26:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "jlat": { 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "replayed": 47668 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_mem": { 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 8826, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "ino+": 33242, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "ino-": 24416, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "dir": 295, 2023-10-06T22:26:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "dir+": 1021, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "dir-": 726, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "dn": 25973, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "dn+": 54837, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "dn-": 28864, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "cap": 0, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "cap+": 0, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "cap-": 0, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "rss": 109932, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "heap": 207152 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_server": { 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_client_request": 0, 2023-10-06T22:26:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_server_request": 0, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_request": 0, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_session": 0, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_peer_request": 0, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "req_create_latency": { 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getattr_latency": { 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "req_link_latency": { 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookup_latency": { 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupino_latency": { 2023-10-06T22:26:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupname_latency": { 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lssnap_latency": { 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mkdir_latency": { 2023-10-06T22:26:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mknod_latency": { 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mksnap_latency": { 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "req_open_latency": { 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "req_readdir_latency": { 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rename_latency": { 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmdir_latency": { 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setattr_latency": { 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setlayout_latency": { 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setxattr_latency": { 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "req_symlink_latency": { 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "req_unlink_latency": { 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_sessions": { 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "session_count": 1, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "session_add": 3, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "session_remove": 2, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_open": 1, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_stale": 0, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "total_load": 0, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "average_load": 0, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "avg_session_uptime": 2332, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-06T22:26:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 539239, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 9109, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 7944, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 166, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 38379465, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 374218, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "objecter": { 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "op_active": 0, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "op_laggy": 0, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send": 2687, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send_bytes": 21, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "op_resend": 3, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "op_reply": 2687, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "op_latency": { 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 2687, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 18.650733454, 2023-10-06T22:26:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.006941099 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "op_inflight": 0, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "oplen_avg": { 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 2687, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 2688 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "op": 2687, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "op_r": 2687, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "op_w": 0, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "op_rmw": 0, 2023-10-06T22:26:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "op_pg": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_stat": 1588, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_create": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_read": 1097, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_write": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writefull": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writesame": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_append": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_zero": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_truncate": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_delete": 0, 2023-10-06T22:26:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_mapext": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_getxattr": 1, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_setxattr": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_call": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_watch": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_notify": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_other": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_active": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_send": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_resend": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_ping": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_active": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_send": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_resend": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_active": 0, 2023-10-06T22:26:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_send": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_resend": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_active": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_send": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_resend": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "command_active": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "command_send": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "command_resend": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "map_epoch": 98, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "map_full": 0, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "map_inc": 15, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_sessions": 10, 2023-10-06T22:26:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_open": 10, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_close": 0, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_laggy": 0, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_wr": 0, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_rd": 2, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_del": 0 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "oft": { 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_objs": 0, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_updates": 0, 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_removes": 0 2023-10-06T22:26:05.465 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "osd-slow-ops": { 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_ops_count": 0 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "purge_queue": { 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing": 0, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_high_water": 0, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executed": 0, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 2976, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 170057773, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 2976, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 2976, 2023-10-06T22:26:05.467 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 170057773, 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-06T22:26:05.468 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 2687, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 167630479, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 2687, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 167630479, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.469 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 1024, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:05.470 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 2687, 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 2687, 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 2687, 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 2687, 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:05.471 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:05.472 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle-0x5650abce4a00": { 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:05.473 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:05.474 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:26:05.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:05 smithi057 ceph-mon[98492]: pgmap v354: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-10-06T22:26:05.974 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-10-06T22:26:05.974 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-10-06T22:26:05.974 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-10-06T22:26:05.974 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:05.974 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:05.974 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2023-10-06T22:26:05.974 DEBUG:tasks.check_counter:Getting stats from d 2023-10-06T22:26:05.974 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-06T22:26:06.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:05 smithi156 ceph-mon[134358]: pgmap v354: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-10-06T22:26:06.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:05 smithi190 ceph-mon[134891]: pgmap v354: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2023-10-06T22:26:06.712 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:26:07.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:07 smithi057 ceph-mon[98492]: pgmap v355: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 195 op/s 2023-10-06T22:26:07.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:07 smithi156 ceph-mon[134358]: pgmap v355: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 195 op/s 2023-10-06T22:26:07.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:07 smithi190 ceph-mon[134891]: pgmap v355: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 195 op/s 2023-10-06T22:26:08.378 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:26:08.378 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:08.378 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2023-10-06T22:26:08.378 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2023-10-06T22:26:08.378 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.003777848, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000786791, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.379 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 6, 2023-10-06T22:26:08.380 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 172, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 3312, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 183504, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 3, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.019546441, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.013043476, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.003348841, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 170, 2023-10-06T22:26:08.381 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.047334748, 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000278439 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 183504 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.382 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 235, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 223, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 1700096, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 133680, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.057269779, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.020325316, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.043487315, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.005792610, 2023-10-06T22:26:08.383 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:08.384 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 222, 2023-10-06T22:26:08.384 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.056211238, 2023-10-06T22:26:08.384 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000253203 2023-10-06T22:26:08.384 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.384 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:08.384 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:08.384 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1700096, 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 133680 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.385 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:08.386 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:08.387 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:08.388 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 148977, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 30, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-06T22:26:08.389 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-06T22:26:08.390 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:08.391 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 237, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 1666919, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 237, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 237, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 1666919, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:08.392 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:08.393 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:08.393 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:08.393 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:26:08.907 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-06T22:26:08.907 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-06T22:26:08.907 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-10-06T22:26:08.907 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-10-06T22:26:08.907 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-10-06T22:26:08.907 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-10-06T22:26:08.908 DEBUG:tasks.check_counter:Getting stats from g 2023-10-06T22:26:08.908 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-06T22:26:09.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:09 smithi190 ceph-mon[134891]: pgmap v356: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-10-06T22:26:09.645 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:26:09.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:09 smithi057 ceph-mon[98492]: pgmap v356: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-10-06T22:26:09.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:09 smithi156 ceph-mon[134358]: pgmap v356: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-10-06T22:26:11.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:11 smithi057 ceph-mon[98492]: pgmap v357: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 423 KiB/s wr, 71 op/s 2023-10-06T22:26:11.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:11 smithi057 ceph-mon[98492]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T22:26:11.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:11 smithi156 ceph-mon[134358]: pgmap v357: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 423 KiB/s wr, 71 op/s 2023-10-06T22:26:11.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:11 smithi156 ceph-mon[134358]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T22:26:11.773 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:26:11.773 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:11.773 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1207, 2023-10-06T22:26:11.773 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 1206, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 47963455, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 374271, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 8, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.249595636, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.086202416, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.106211752, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.041145980, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1205, 2023-10-06T22:26:11.774 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.576183782, 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000478160 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:11.775 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1048, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 1213, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 54812142, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 776147, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 8, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 6, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.218256943, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.041619625, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.130614768, 2023-10-06T22:26:11.777 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.034250015, 2023-10-06T22:26:11.778 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:11.778 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1210, 2023-10-06T22:26:11.778 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.543176403, 2023-10-06T22:26:11.778 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000448906 2023-10-06T22:26:11.778 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.778 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:11.778 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 450784 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 770, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 721, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 69211888, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 284747, 2023-10-06T22:26:11.779 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.207657525, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.033178745, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.163421419, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.021390309, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 720, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.237960547, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000330500 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.780 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1963264, 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 135456 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 1, 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-MDSRank": { 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-06T22:26:11.781 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 4463, 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.149266662, 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000033445 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.782 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "request": 0, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "reply": 0, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "reply_latency": { 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_reply": 0, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "forward": 0, 2023-10-06T22:26:11.783 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_complete": 0, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_commit": 0, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_split": 0, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_merge": 0, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes": 32464, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_top": 3698, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_bottom": 25494, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pin_tail": 3272, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pinned": 27180, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_expired": 795, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_with_caps": 0, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": 0, 2023-10-06T22:26:11.784 INFO:teuthology.orchestra.run.smithi057.stdout: "subtrees": 1, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_hit": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_forward": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_discover": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_dir_fetch": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_lock": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "load_cent": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "q": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "exported": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "exported_inodes": 0, 2023-10-06T22:26:11.785 INFO:teuthology.orchestra.run.smithi057.stdout: "imported": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "imported_inodes": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_backtrace_fetch": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rfiles": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rbytes": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rsnaps": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_inodes": 0, 2023-10-06T22:26:11.786 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_file_inodes": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_inode_file_caps": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_grant": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps_dirty": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_cap_release": 0, 2023-10-06T22:26:11.787 INFO:teuthology.orchestra.run.smithi057.stdout: "process_request_cap_release": 0 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_cache": { 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update_receipt": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_try_discover": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_send_discover": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_handle_discover": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_created": 0, 2023-10-06T22:26:11.788 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_enqueued": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_migrated": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_started": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_completed": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_exportdir": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_flush": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragmentdir": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:11.789 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_inodestats": 0 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_log": { 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "evlrg": 0, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "evadd": 0, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "evex": 0, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "evtrm": 0, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "ev": 48867, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "evexg": 0, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "evexd": 0, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "segadd": 0, 2023-10-06T22:26:11.790 INFO:teuthology.orchestra.run.smithi057.stdout: "segex": 0, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "segtrm": 0, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "seg": 70, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "segmjr": 5, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "segexg": 0, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "segexd": 0, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "expos": 4194304, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "wrpos": 173418952, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "rdpos": 173408779, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "jlat": { 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.791 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "replayed": 48867 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_mem": { 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 10002, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "ino+": 32146, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "ino-": 22144, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "dir": 355, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "dir+": 891, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "dir-": 536, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "dn": 32464, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "dn+": 53660, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "dn-": 21196, 2023-10-06T22:26:11.792 INFO:teuthology.orchestra.run.smithi057.stdout: "cap": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "cap+": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "cap-": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "rss": 114560, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "heap": 207152 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_server": { 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_client_request": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_server_request": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_request": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_session": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_peer_request": 0, 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "req_create_latency": { 2023-10-06T22:26:11.793 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getattr_latency": { 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.794 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "req_link_latency": { 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookup_latency": { 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:11.795 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupino_latency": { 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupname_latency": { 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lssnap_latency": { 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.797 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mkdir_latency": { 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mknod_latency": { 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.798 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mksnap_latency": { 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "req_open_latency": { 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.799 INFO:teuthology.orchestra.run.smithi057.stdout: "req_readdir_latency": { 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rename_latency": { 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.800 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmdir_latency": { 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setattr_latency": { 2023-10-06T22:26:11.802 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.803 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setlayout_latency": { 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setxattr_latency": { 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "req_symlink_latency": { 2023-10-06T22:26:11.804 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "req_unlink_latency": { 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:11.805 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_sessions": { 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "session_count": 2, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "session_add": 3, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "session_remove": 1, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_open": 2, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_stale": 0, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "total_load": 0, 2023-10-06T22:26:11.806 INFO:teuthology.orchestra.run.smithi057.stdout: "average_load": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "avg_session_uptime": 3996, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:11.807 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:11.808 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-06T22:26:11.809 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 578737, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 10314, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:11.810 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 7944, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 166, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 45600880, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 449106, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-06T22:26:11.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "objecter": { 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_active": 0, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_laggy": 0, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send": 2728, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send_bytes": 21, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_resend": 3, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_reply": 2728, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_latency": { 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 2728, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 14.463792822, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.005301976 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.812 INFO:teuthology.orchestra.run.smithi057.stdout: "op_inflight": 0, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "oplen_avg": { 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 2728, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 2729 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "op": 2728, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "op_r": 2728, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "op_w": 0, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "op_rmw": 0, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "op_pg": 0, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_stat": 1581, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_create": 0, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_read": 1145, 2023-10-06T22:26:11.813 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_write": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writefull": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writesame": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_append": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_zero": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_truncate": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_delete": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_mapext": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_getxattr": 1, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_setxattr": 0, 2023-10-06T22:26:11.814 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_call": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_watch": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_notify": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_other": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_active": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_send": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_resend": 0, 2023-10-06T22:26:11.815 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_ping": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_active": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_send": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_resend": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_active": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_send": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_resend": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_active": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_send": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_resend": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "command_active": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "command_send": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "command_resend": 0, 2023-10-06T22:26:11.816 INFO:teuthology.orchestra.run.smithi057.stdout: "map_epoch": 99, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "map_full": 0, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "map_inc": 16, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_sessions": 10, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_open": 10, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_close": 0, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_laggy": 0, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_wr": 0, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_rd": 2, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_del": 0 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "oft": { 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_objs": 0, 2023-10-06T22:26:11.817 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_updates": 0, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_removes": 0 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "osd-slow-ops": { 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_ops_count": 0 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "purge_queue": { 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing": 0, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_high_water": 0, 2023-10-06T22:26:11.818 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executed": 0, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 3021, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 171697947, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 3021, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:11.819 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 3021, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 171697947, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:11.820 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 2728, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 169261190, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 2728, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 169261190, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 1024, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 2728, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 2728, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 2728, 2023-10-06T22:26:11.822 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 2728, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:11.823 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:11.824 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle-0x55b0e6bf6a00": { 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-06T22:26:11.825 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:11.826 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:11.826 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:11.826 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:11.826 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.826 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:11.826 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:26:11.826 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:11 smithi190 ceph-mon[134891]: pgmap v357: 97 pgs: 97 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 423 KiB/s wr, 71 op/s 2023-10-06T22:26:11.827 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:11 smithi190 ceph-mon[134891]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T22:26:12.472 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-10-06T22:26:12.472 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-10-06T22:26:12.472 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2023-10-06T22:26:12.472 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:12.472 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:12.472 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2023-10-06T22:26:12.472 DEBUG:tasks.check_counter:Getting stats from j 2023-10-06T22:26:12.473 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-06T22:26:13.576 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T22:26:13.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:13 smithi057 ceph-mon[98492]: pgmap v359: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 31 op/s 2023-10-06T22:26:13.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:13 smithi156 ceph-mon[134358]: pgmap v359: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 31 op/s 2023-10-06T22:26:13.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:13 smithi190 ceph-mon[134891]: pgmap v359: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 31 op/s 2023-10-06T22:26:15.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:15 smithi057 ceph-mon[98492]: pgmap v360: 97 pgs: 21 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 39 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2023-10-06T22:26:15.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:15 smithi156 ceph-mon[134358]: pgmap v360: 97 pgs: 21 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 39 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2023-10-06T22:26:15.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:15 smithi190 ceph-mon[134891]: pgmap v360: 97 pgs: 21 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 39 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2023-10-06T22:26:15.823 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-06T22:26:15.823 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:15.823 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2023-10-06T22:26:15.823 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2023-10-06T22:26:15.823 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2023-10-06T22:26:15.823 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2023-10-06T22:26:15.823 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.003310123, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000597365, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:15.824 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 242, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 228, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 1703904, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 136000, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2023-10-06T22:26:15.825 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.057547979, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.019529887, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.045294924, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.005678124, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 228, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.055213120, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000242162 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1703904, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 136000 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 2, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 172, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 320, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 183408, 2023-10-06T22:26:15.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 3, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.020242200, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.013521531, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.003334281, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 169, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.029290453, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000173316 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:15.828 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 183408 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-06T22:26:15.829 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:15.830 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:15.831 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 148977, 2023-10-06T22:26:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 30, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:15.833 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-06T22:26:15.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 240, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 1667309, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 240, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 240, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 1667309, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-06T22:26:15.835 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-06T22:26:15.836 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-06T22:26:15.836 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:15.836 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-06T22:26:15.836 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-06T22:26:16.517 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-06T22:26:16.517 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-06T22:26:16.517 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-10-06T22:26:16.517 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-10-06T22:26:16.517 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-10-06T22:26:16.517 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-10-06T22:26:16.517 DEBUG:tasks.check_counter:Getting stats from b 2023-10-06T22:26:16.518 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-06T22:26:17.415 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:26:17.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:17 smithi057 ceph-mon[98492]: pgmap v361: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2023-10-06T22:26:17.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:17 smithi156 ceph-mon[134358]: pgmap v361: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2023-10-06T22:26:17.570 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:17 smithi190 ceph-mon[134891]: pgmap v361: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2023-10-06T22:26:18.871 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 745352, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 879964, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 173515256, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 337582408, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 14, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 25.671141637, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 11.338620338, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 17.511963503, 2023-10-06T22:26:18.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.362536152, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 879959, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 35.564379658, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000040415 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1233119, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.148429869, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000120 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1987808, 2023-10-06T22:26:18.873 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 144816 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 542324, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 532363, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 92738461, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 267274682, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 20, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 17.121329495, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 7.112850108, 2023-10-06T22:26:18.874 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 11.653818004, 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.513207142, 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 532360, 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 8.111685798, 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000015237 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 999693, 2023-10-06T22:26:18.875 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.146895156, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000146 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 458816 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2585387, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2581284, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 425387910, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 608089925, 2023-10-06T22:26:18.876 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 14, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 90.107674899, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 42.266854068, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 58.082908482, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.455073076, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2581287, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 37.720483524, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000014613 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:18.877 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4810227, 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.779415667, 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000162 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.878 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7038, 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 95.755928363, 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.013605559 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:18.879 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 33644, 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 8.741114759, 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000259811 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 113738, 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 104447, 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-10-06T22:26:18.880 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 104447, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2320.013823586, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.022212354 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 9290, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 52, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 924, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 100, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 8, 2023-10-06T22:26:18.881 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 39455, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 569, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 25379, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 13507, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 38634, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 58225, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 1, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 1, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 5, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 174609, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 146100, 2023-10-06T22:26:18.882 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 9095, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 850, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 5, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 351, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 20842, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 188, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 43530, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 157, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 32699, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 2947, 2023-10-06T22:26:18.883 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 5279, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 106051378, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 2, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 2947, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 40, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 40, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 2907, 2023-10-06T22:26:18.884 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 105, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 26501, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 50489, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 4, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 1, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 71062, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 27182, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 1593, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 48389 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-10-06T22:26:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 917, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 706, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 106, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 18238, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 18132, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-10-06T22:26:18.886 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 190, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 108, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.887 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 3, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 136463, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 47259, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 36871, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 99592, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 10388, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 188, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 60, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 47, 2023-10-06T22:26:18.888 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 141, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 9, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 13, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 95981778, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 457092740, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3070, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 193.809437908, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.063130110 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.889 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 20118, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 65289, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 45171, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 795, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 2104, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 1309, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 39463, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 118301, 2023-10-06T22:26:18.890 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 78838, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 1, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 44564, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 44563, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 191772, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 198960 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 146199, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 2715, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 113738, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 249, 2023-10-06T22:26:18.891 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 5030, 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 26536, 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 181.612073850, 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006843988 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3158, 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 339.099809844, 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.107378027 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:18.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 26524, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1135.740008734, 2023-10-06T22:26:18.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.042819333 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000130786, 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000130786 2023-10-06T22:26:18.894 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:18.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 190, 2023-10-06T22:26:18.896 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.941058753, 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.020742414 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6764, 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 44.755759799, 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006616759 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 710, 2023-10-06T22:26:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.905483258, 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001275328 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 26527, 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 572.046462523, 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.021564687 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 106, 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 14.343893389, 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.135319748 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-10-06T22:26:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4, 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.436100943, 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.109025235 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13927, 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 27.133041707, 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001948233 2023-10-06T22:26:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-10-06T22:26:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 11068, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 11068, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 689, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 1872892, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 19969, 2023-10-06T22:26:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 169, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 74759567, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 802329, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 26967, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 474455459, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 27, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 26967, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 26967, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 234.966355810, 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.008713106 2023-10-06T22:26:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 26967, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 28458 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 26967, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 3513, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 23454, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 12, 2023-10-06T22:26:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 94, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 514, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 3584, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 295, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 100, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 18336, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 2948, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 188, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 4, 2023-10-06T22:26:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-10-06T22:26:18.916 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 99, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 16, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 2056, 2023-10-06T22:26:18.917 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 104, 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 223 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 8 2023-10-06T22:26:18.918 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 6, 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 18132, 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:18.919 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 3873059, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 346387442, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 3873059, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3873059, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 346387442, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.920 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:18.921 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 26967, 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 476830903, 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 26967, 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 476830903, 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:18.922 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 26967, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 26967, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 26967, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 26967, 2023-10-06T22:26:18.923 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 18132, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 2376276, 2023-10-06T22:26:18.924 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 18132, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 514, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2376276, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:18.925 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55ea9361e6e0": { 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 136463, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 452898436, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 136463, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3070, 2023-10-06T22:26:18.926 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 452898436, 2023-10-06T22:26:18.927 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:18.927 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:18.927 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:18.927 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:18.927 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.927 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:18.927 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-10-06T22:26:19.299 INFO:tasks.check_counter:Daemon mds.b mds.exported=188 2023-10-06T22:26:19.299 INFO:tasks.check_counter:Daemon mds.b mds.imported=157 2023-10-06T22:26:19.300 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=2 2023-10-06T22:26:19.300 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:19.300 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:19.300 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=100 2023-10-06T22:26:19.300 DEBUG:tasks.check_counter:Getting stats from e 2023-10-06T22:26:19.300 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-06T22:26:19.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:19 smithi156 ceph-mon[134358]: pgmap v362: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 613 B/s wr, 12 op/s 2023-10-06T22:26:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:19 smithi057 ceph-mon[98492]: pgmap v362: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 613 B/s wr, 12 op/s 2023-10-06T22:26:19.823 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:19 smithi190 ceph-mon[134891]: pgmap v362: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 613 B/s wr, 12 op/s 2023-10-06T22:26:20.029 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:26:21.507 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:21 smithi156 ceph-mon[134358]: pgmap v363: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 179, 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 175, 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 15070579, 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 58475, 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-10-06T22:26:21.526 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.028231338, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.005321502, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.017105424, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.005172200, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 174, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.030809265, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000177064 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.527 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 181728, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 13376 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 23, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 37, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 6494698, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 113719, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-10-06T22:26:21.528 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.008840741, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000930166, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.007065486, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000557181, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 36, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.034733578, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000964821 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.529 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2816, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 107744 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 104, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 106, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 29390655, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 32351, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-10-06T22:26:21.530 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.023997440, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.001889062, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.017950568, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.003394029, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 104, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.034872328, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000335310 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.531 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:21.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 411, 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.097842372, 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000238059 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.533 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:21.534 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 18233, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 12646, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 5587, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 167, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 211, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-10-06T22:26:21.535 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 2, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-10-06T22:26:21.536 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 4, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 246, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:21.537 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-10-06T22:26:21.538 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-10-06T22:26:21.539 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-10-06T22:26:21.540 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 15989, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-10-06T22:26:21.541 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 2, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 4, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 245044944, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 245526793, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 245524188, 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-10-06T22:26:21.542 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 15989 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 6228, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 12370, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 6142, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 298, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 299, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 1, 2023-10-06T22:26:21.543 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 18233, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 18619, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 386, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 80124, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207152 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-10-06T22:26:21.544 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:21.545 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.546 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.547 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.548 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.549 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.550 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:21.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.553 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-10-06T22:26:21.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.555 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:21.556 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 0, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-10-06T22:26:21.557 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 2660, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:21.558 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:21.559 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 440922, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 6507, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:21.561 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 169, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:21.562 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 27057027, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 215305, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 268, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 26, 2023-10-06T22:26:21.563 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 268, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 268, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.702989470, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002623095 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 268, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 269 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 268, 2023-10-06T22:26:21.564 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 268, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 158, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 107, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-10-06T22:26:21.565 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 2, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-10-06T22:26:21.566 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-10-06T22:26:21.567 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 99, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-10-06T22:26:21.568 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 2, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 9, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 9, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-10-06T22:26:21.569 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-10-06T22:26:21.570 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 303, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 50924301, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 303, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:21.571 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 303, 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 50924301, 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:21.572 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 268, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 50691832, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 268, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 50691832, 2023-10-06T22:26:21.573 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-10-06T22:26:21.574 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 268, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 268, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 268, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 268, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:21.576 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55a93f428960": { 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:21.577 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:21.578 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:21.579 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:21.579 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.579 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-10-06T22:26:21.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:21 smithi057 ceph-mon[98492]: pgmap v363: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-06T22:26:21.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:21 smithi190 ceph-mon[134891]: pgmap v363: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-06T22:26:22.037 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-06T22:26:22.037 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-10-06T22:26:22.037 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2023-10-06T22:26:22.038 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:22.038 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:22.038 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-10-06T22:26:22.038 DEBUG:tasks.check_counter:Getting stats from h 2023-10-06T22:26:22.038 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-06T22:26:22.745 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:26:23.389 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:23 smithi156 ceph-mon[134358]: pgmap v364: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 428 B/s wr, 13 op/s 2023-10-06T22:26:23.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:23 smithi057 ceph-mon[98492]: pgmap v364: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 428 B/s wr, 13 op/s 2023-10-06T22:26:23.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:23 smithi190 ceph-mon[134891]: pgmap v364: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 428 B/s wr, 13 op/s 2023-10-06T22:26:24.374 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 3031980, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 3034986, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 501941427, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 550183381, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 19, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 82.165354159, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 33.635981754, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 56.431390894, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.241300233, 2023-10-06T22:26:24.375 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:24.376 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3034985, 2023-10-06T22:26:24.376 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 42.417265653, 2023-10-06T22:26:24.376 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000013976 2023-10-06T22:26:24.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.376 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:24.376 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5415640, 2023-10-06T22:26:24.376 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.930406961, 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000171 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2105311, 2023-10-06T22:26:24.377 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2106722, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 353244368, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 381635615, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 16, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 77.992130070, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 37.496500449, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 49.527288572, 2023-10-06T22:26:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.176651987, 2023-10-06T22:26:24.379 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:24.379 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2106721, 2023-10-06T22:26:24.379 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 27.634410593, 2023-10-06T22:26:24.379 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000013117 2023-10-06T22:26:24.379 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.379 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:24.379 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3740237, 2023-10-06T22:26:24.380 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.585106847, 2023-10-06T22:26:24.380 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000156 2023-10-06T22:26:24.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.380 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1973824, 2023-10-06T22:26:24.380 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 523680 2023-10-06T22:26:24.381 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.381 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:24.381 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 114680, 2023-10-06T22:26:24.381 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 159210, 2023-10-06T22:26:24.381 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 39713660, 2023-10-06T22:26:24.381 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 136816512, 2023-10-06T22:26:24.381 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 15, 2023-10-06T22:26:24.382 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-10-06T22:26:24.382 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 4.566262976, 2023-10-06T22:26:24.382 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 2.178525479, 2023-10-06T22:26:24.382 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 3.167665676, 2023-10-06T22:26:24.382 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.369384582, 2023-10-06T22:26:24.382 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:24.383 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 159207, 2023-10-06T22:26:24.383 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 41.352374828, 2023-10-06T22:26:24.383 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000259739 2023-10-06T22:26:24.383 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.383 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:24.383 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 63643, 2023-10-06T22:26:24.383 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.021286609, 2023-10-06T22:26:24.384 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000334 2023-10-06T22:26:24.384 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.384 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2784, 2023-10-06T22:26:24.384 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 131456 2023-10-06T22:26:24.384 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.384 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-10-06T22:26:24.384 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-10-06T22:26:24.385 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-10-06T22:26:24.385 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.385 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-10-06T22:26:24.385 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:24.385 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5765, 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 25.513818544, 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004425640 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:24.386 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 18730, 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.210165858, 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000171391 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 47850, 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 47258, 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 47258, 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 773.693050529, 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.016371684 2023-10-06T22:26:24.387 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 592, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 126, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 353, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 38, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 17495, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 371, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 15042, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 2082, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 16962, 2023-10-06T22:26:24.388 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 24979, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 6, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 85949, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 72682, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 377, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 504, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 1, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 642, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 1000, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 1, 2023-10-06T22:26:24.389 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 227, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 18352, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 163, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 24784, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 1182, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 5279, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 106051378, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 2, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 1182, 2023-10-06T22:26:24.390 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 3, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 3, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 1179, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 184, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 4138, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 14922, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 59, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:24.391 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 14938, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 5681, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 510, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 21318 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 6, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 4, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 591, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 929, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 50, 2023-10-06T22:26:24.392 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 9631, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 9581, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 227, 2023-10-06T22:26:24.393 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 39, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 2, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 48867, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 48867, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-10-06T22:26:24.394 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 70, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 70, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 5, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 173418952, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2072, 2023-10-06T22:26:24.395 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 58.696206302, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.028328284 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 9848, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 32620, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 22772, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 386, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 992, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 606, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 17495, 2023-10-06T22:26:24.396 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 55097, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 37602, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 20032, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 20032, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 162208, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 198960 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 55652, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 3105, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 47850, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 308, 2023-10-06T22:26:24.397 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 5060, 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3938, 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 15.233163522, 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003868248 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 64, 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.125668208, 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.048838565 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:24.398 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 28334, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 473.271475987, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.016703306 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.399 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.400 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.401 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5, 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.056378653, 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.011275730 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.402 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2120, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.279252546, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001546817 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.403 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 424, 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.783260271, 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001847311 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3943, 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 256.461678302, 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.065042271 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.404 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 50, 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 6.285359817, 2023-10-06T22:26:24.405 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.125707196 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.407 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.408 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8380, 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 15.196813223, 2023-10-06T22:26:24.409 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001813462 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2023-10-06T22:26:24.410 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 3, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 11653, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 5826, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 682, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:24.411 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:24.412 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:24.413 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:24.414 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 1043914, 2023-10-06T22:26:24.415 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 8660, 2023-10-06T22:26:24.416 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 352, 2023-10-06T22:26:24.416 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 4, 2023-10-06T22:26:24.416 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-10-06T22:26:24.416 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-10-06T22:26:24.416 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:24.416 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:24.417 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:24.417 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:24.417 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:24.417 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 169, 2023-10-06T22:26:24.417 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:24.417 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:24.418 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:24.418 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-10-06T22:26:24.418 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 35150046, 2023-10-06T22:26:24.418 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 343727, 2023-10-06T22:26:24.418 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:24.418 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-10-06T22:26:24.418 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:24.419 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-10-06T22:26:24.419 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.419 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-10-06T22:26:24.419 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-10-06T22:26:24.419 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-10-06T22:26:24.419 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 14855, 2023-10-06T22:26:24.419 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 177894993, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 28, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 14855, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 14855, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 157.716565227, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.010617069 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 14855, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 15562 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.420 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 14855, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 1823, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 13032, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 10, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 515, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 2587, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 308, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-10-06T22:26:24.421 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 38, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 9633, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1184, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 20, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-10-06T22:26:24.422 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-10-06T22:26:24.423 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 99, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 16, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2023-10-06T22:26:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 887, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 252, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 128 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-10-06T22:26:24.425 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 3 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 6, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 9581, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:24.426 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 5251967, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 423896751, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 5251967, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 5251967, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 423896751, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.427 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 14855, 2023-10-06T22:26:24.428 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 179217026, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 14855, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 179217026, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:24.429 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 14855, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 14855, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 14855, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 14855, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 9581, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 1324049, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 9581, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:24.431 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 515, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1324049, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55c40f2fa6e0": { 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 48867, 2023-10-06T22:26:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 169224648, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 48867, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2072, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 169224648, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:24.433 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-10-06T22:26:24.790 INFO:tasks.check_counter:Daemon mds.h mds.exported=227 2023-10-06T22:26:24.790 INFO:tasks.check_counter:Daemon mds.h mds.imported=163 2023-10-06T22:26:24.790 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=2 2023-10-06T22:26:24.790 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:24.790 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:24.791 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=38 2023-10-06T22:26:24.791 DEBUG:tasks.check_counter:Getting stats from k 2023-10-06T22:26:24.791 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-06T22:26:25.567 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T22:26:25.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:25 smithi057 ceph-mon[98492]: pgmap v365: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 340 B/s wr, 10 op/s 2023-10-06T22:26:25.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:25 smithi156 ceph-mon[134358]: pgmap v365: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 340 B/s wr, 10 op/s 2023-10-06T22:26:25.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:25 smithi190 ceph-mon[134891]: pgmap v365: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 340 B/s wr, 10 op/s 2023-10-06T22:26:27.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:27 smithi156 ceph-mon[134358]: pgmap v366: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-10-06T22:26:27.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:27 smithi190 ceph-mon[134891]: pgmap v366: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-10-06T22:26:27.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:27 smithi057 ceph-mon[98492]: pgmap v366: 97 pgs: 22 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 34 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-10-06T22:26:28.794 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-10-06T22:26:28.794 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:28.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 951, 2023-10-06T22:26:28.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 953, 2023-10-06T22:26:28.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 77134419, 2023-10-06T22:26:28.794 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 295825, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.112454238, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.020869385, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.059310114, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.025461602, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 951, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.060778914, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000063910 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.795 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 852, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 801, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 60839655, 2023-10-06T22:26:28.796 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 312561, 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.120314410, 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.021170626, 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.086271064, 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.021736005, 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:28.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 801, 2023-10-06T22:26:28.798 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.042197845, 2023-10-06T22:26:28.798 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000052681 2023-10-06T22:26:28.798 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.798 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1968976, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 142128 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1293, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1466, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 84307235, 2023-10-06T22:26:28.799 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 860029, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.160573735, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.032639234, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.075854930, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.041085031, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1463, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.116638408, 2023-10-06T22:26:28.800 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000079725 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 458528 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-10-06T22:26:28.801 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4634, 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.123395702, 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000026628 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-10-06T22:26:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.803 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 27773, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 313, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 22714, 2023-10-06T22:26:28.804 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 4746, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 27325, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 511, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 1, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-10-06T22:26:28.805 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-10-06T22:26:28.806 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T22:26:28.807 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-10-06T22:26:28.808 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:28.809 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 67659, 2023-10-06T22:26:28.810 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 100, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 7, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 223684206, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 223679553, 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-10-06T22:26:28.811 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 67659 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 8051, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 28144, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 20093, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 320, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 816, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 496, 2023-10-06T22:26:28.812 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 27773, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 49460, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 21687, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 110444, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207152 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-10-06T22:26:28.813 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.814 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-10-06T22:26:28.815 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.816 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:28.817 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.818 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.819 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-10-06T22:26:28.820 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.821 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.822 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.823 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.824 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-10-06T22:26:28.825 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.826 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 4, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 2, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 3, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 3655, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-10-06T22:26:28.827 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:28.828 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:28.829 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:28.830 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 511784, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 8302, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 169, 2023-10-06T22:26:28.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 37945481, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 395320, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-10-06T22:26:28.832 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 2790, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 2790, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2790, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 16.417743860, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.005884496 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2790, 2023-10-06T22:26:28.833 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2791 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 2790, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 2790, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 1644, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 1144, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-10-06T22:26:28.834 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:28.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-10-06T22:26:28.836 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 99, 2023-10-06T22:26:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 16, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 10, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 10, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-10-06T22:26:28.838 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-10-06T22:26:28.839 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 3092, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 221984861, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 3092, 2023-10-06T22:26:28.840 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3092, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 221984861, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-10-06T22:26:28.841 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2790, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 219535561, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2790, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 219535561, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.842 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2790, 2023-10-06T22:26:28.843 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 2790, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2790, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2790, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:28.844 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.845 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5639a08e4960": { 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-10-06T22:26:28.846 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-10-06T22:26:28.847 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-10-06T22:26:29.274 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-10-06T22:26:29.274 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-10-06T22:26:29.274 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-10-06T22:26:29.274 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:29.275 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:29.275 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2023-10-06T22:26:29.275 DEBUG:tasks.check_counter:Getting stats from c 2023-10-06T22:26:29.275 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-06T22:26:29.504 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:29 smithi156 ceph-mon[134358]: pgmap v367: 97 pgs: 18 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 42 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 12 op/s 2023-10-06T22:26:29.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:29 smithi057 ceph-mon[98492]: pgmap v367: 97 pgs: 18 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 42 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 12 op/s 2023-10-06T22:26:29.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:29 smithi190 ceph-mon[134891]: pgmap v367: 97 pgs: 18 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 42 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 12 op/s 2023-10-06T22:26:30.704 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:26:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:31 smithi057 ceph-mon[98492]: pgmap v368: 97 pgs: 16 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 46 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 11 op/s 2023-10-06T22:26:31.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:31 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:26:31.697 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:31 smithi190 ceph-mon[134891]: pgmap v368: 97 pgs: 16 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 46 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 11 op/s 2023-10-06T22:26:31.697 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:31 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:26:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:31 smithi156 ceph-mon[134358]: pgmap v368: 97 pgs: 16 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 46 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 11 op/s 2023-10-06T22:26:31.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:31 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 475429, 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 478165, 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 86765737, 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 155524003, 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 27, 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 9, 2023-10-06T22:26:32.325 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 19.240338422, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 7.097558069, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 14.210574784, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.134471708, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 478162, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 8.955655905, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000018729 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 934996, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.138801156, 2023-10-06T22:26:32.326 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000148 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1995712, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 151264 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 198191, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 195527, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 38362317, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 134220594, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 11, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2023-10-06T22:26:32.327 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 10.339490899, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 4.776817659, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 6.628619553, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.168993349, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 195527, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 6.551545817, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000033507 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 352473, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.064837714, 2023-10-06T22:26:32.328 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000183 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 464016 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 383363, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 432216, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 86576394, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 167639718, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 12, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:32.329 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 16.333641720, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 7.550215573, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 11.368444809, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.222670088, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 432214, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 18.544556834, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000042905 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 567214, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.094267070, 2023-10-06T22:26:32.330 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000166 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 5552, 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 15008 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:32.331 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5248, 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 47.309730309, 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.009014811 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 15126, 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.605926743, 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000304503 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 50036, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 48911, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 48911, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 541.669639045, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.011074597 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 1125, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 36, 2023-10-06T22:26:32.333 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 541, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 59, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 16448, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 4, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 3, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 16441, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 16441, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 26430, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 2, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 2, 2023-10-06T22:26:32.334 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 5, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 80877, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 69973, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 1005, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 332, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 12, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 421, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 129, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 19326, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 136, 2023-10-06T22:26:32.335 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 12670, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 1568, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 5279, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 106051378, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 2, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 1568, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 6, 2023-10-06T22:26:32.336 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 6, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 1562, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 171, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 10866, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 21974, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 95, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 35225, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 12374, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 340, 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 22289 2023-10-06T22:26:32.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 6, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 4, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 398, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 676, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 43, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 8062, 2023-10-06T22:26:32.338 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 8019, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 131, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 59, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:32.339 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 2, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 67659, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 67659, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 100, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2023-10-06T22:26:32.340 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 100, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 7, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 223684206, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1746, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 71.279603097, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.040824514 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.341 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 7956, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 28407, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 20451, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 335, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 870, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 535, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 16448, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 51971, 2023-10-06T22:26:32.342 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 35523, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 2, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 18484, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 18482, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 153400, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 198960 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 62290, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 1970, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 50036, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 202, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 3935, 2023-10-06T22:26:32.343 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 11079, 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 32.884396874, 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002968173 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 181, 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 27.717792099, 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.153136972 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.344 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 16264, 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 277.868323773, 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.017084869 2023-10-06T22:26:32.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-10-06T22:26:32.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.347 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 60, 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.727053176, 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.012117552 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.348 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-10-06T22:26:32.349 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3588, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 10.440257212, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002909770 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 300, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.540745754, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001802485 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 11093, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 165.278028510, 2023-10-06T22:26:32.350 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.014899308 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.351 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 43, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 13.275665097, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.308736397 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.353 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.354 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6303, 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 12.937376550, 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002052574 2023-10-06T22:26:32.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 2, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 4, 2023-10-06T22:26:32.356 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 2, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 2, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 3803, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 1901, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 706, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:32.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:32.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 929729, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 7424, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 264, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 3, 2023-10-06T22:26:32.360 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 169, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 30133978, 2023-10-06T22:26:32.361 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 327770, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 13084, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 231760993, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 27, 2023-10-06T22:26:32.362 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 13084, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 13084, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 170.172142602, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.013006125 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 13084, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 13911 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 13084, 2023-10-06T22:26:32.363 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 1962, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 11122, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 358, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 2104, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 219, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-10-06T22:26:32.364 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 59, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 8083, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1568, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-10-06T22:26:32.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-10-06T22:26:32.366 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 99, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 16, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2023-10-06T22:26:32.367 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 1270, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 72, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 148 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 6, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 8019, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:32.369 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1056979, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 118369653, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1056979, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1056979, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 118369653, 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:32.370 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:32.371 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 13084, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 232801056, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 13084, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 232801056, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:32.372 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 13084, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 13084, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 13084, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 13084, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.373 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 8019, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1040639, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 8019, 2023-10-06T22:26:32.374 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 358, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1040639, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55f8de18a6e0": { 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:32.375 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 67659, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 219489902, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 67659, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1746, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 219489902, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:32.376 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:32.377 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.377 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:32.377 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-10-06T22:26:32.768 INFO:tasks.check_counter:Daemon mds.c mds.exported=129 2023-10-06T22:26:32.768 INFO:tasks.check_counter:Daemon mds.c mds.imported=136 2023-10-06T22:26:32.768 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=2 2023-10-06T22:26:32.768 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:32.768 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:32.768 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=59 2023-10-06T22:26:32.769 DEBUG:tasks.check_counter:Getting stats from f 2023-10-06T22:26:32.769 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-06T22:26:33.405 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:26:33.661 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:33 smithi190 ceph-mon[134891]: pgmap v369: 97 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:33.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:33 smithi057 ceph-mon[98492]: pgmap v369: 97 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:33.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:33 smithi156 ceph-mon[134358]: pgmap v369: 97 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:34.569 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:34 smithi190 ceph-mon[134891]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:26:34.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:34 smithi057 ceph-mon[98492]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:26:34.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:34 smithi156 ceph-mon[134358]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 452090, 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 454349, 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 89263597, 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 135071603, 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 10, 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 22.246203510, 2023-10-06T22:26:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 10.034064335, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 15.302347336, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.293967493, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 454347, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 10.748396246, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000023656 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 884077, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.137750570, 2023-10-06T22:26:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000155 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 628289, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 674970, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 125740788, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 227253379, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 25, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 8, 2023-10-06T22:26:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 22.281832586, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 8.850003393, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 16.200180547, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.174798426, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 674967, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 23.721346720, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000035144 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1111036, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.132431381, 2023-10-06T22:26:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000119 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1072, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 125216 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1602137, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1603222, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 263740020, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 296806937, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 13, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 46.249657238, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 18.352790449, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 32.942522023, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.108892477, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1603222, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 31.319899920, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000019535 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2888009, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.608789453, 2023-10-06T22:26:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000210 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1998192, 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 499200 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5937, 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 29.252493903, 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004927150 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 13284, 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2.910366497, 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000219088 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 47936, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 46440, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 46440, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1722.878771732, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.037099026 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 1495, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 25, 2023-10-06T22:26:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 369, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 40, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 17247, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 9, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 9, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 17229, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 17232, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 31797, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2023-10-06T22:26:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 5, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 84729, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 67466, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 1321, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 703, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 1, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 568, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 179, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 26063, 2023-10-06T22:26:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 203, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 27281, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 1743, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 5279, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 106051378, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 2, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 1743, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 9, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 9, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 1734, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 256, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 6883, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 15211, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 1, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 20888, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 7446, 2023-10-06T22:26:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 333, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 17808 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 6, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 4, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 875, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 852, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 46, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 6866, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 6820, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 180, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 40, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 3, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 47668, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2023-10-06T22:26:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 47668, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 70, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 70, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 5, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 171632868, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2023-10-06T22:26:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2185, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 96.900662850, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.044348129 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 8805, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 33435, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 24630, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 317, 2023-10-06T22:26:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 1149, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 832, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 17247, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 56646, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 39399, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 20490, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 20490, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 149756, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 198960 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-10-06T22:26:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 57470, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 3834, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 47936, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 341, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 5421, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6514, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 27.309904820, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004192493 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 259, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 9.765087292, 2023-10-06T22:26:35.636 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.037703039 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.637 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 24561, 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1387.657631924, 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.056498417 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-10-06T22:26:35.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.640 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 40, 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.126261894, 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003156547 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.641 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2269, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 5.858252148, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002581865 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 266, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.390612227, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001468467 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.642 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6479, 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 272.456754536, 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.042052285 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:35.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 46, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 7.306672185, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.158840699 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:35.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.646 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.647 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6006, 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 12.007594706, 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001999266 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 3, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 2, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 5951, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 5951, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2695, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-10-06T22:26:35.649 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:35.650 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:35.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:35.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 986180, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 9144, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:35.653 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 169, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 32796863, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 365336, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:35.654 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 12336, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 175782026, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 12336, 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-10-06T22:26:35.655 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12336, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 170.193167051, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.013796462 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12336, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 12960 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 12336, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 2139, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 10197, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-10-06T22:26:35.656 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 371, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 2557, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 249, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 40, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 6871, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-10-06T22:26:35.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1743, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-10-06T22:26:35.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-10-06T22:26:35.659 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 99, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 16, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2023-10-06T22:26:35.660 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 921, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 50, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 128 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-10-06T22:26:35.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 37 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-10-06T22:26:35.662 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 6, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 6820, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2682512, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 238788575, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2682512, 2023-10-06T22:26:35.663 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2682512, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 238788575, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:35.664 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 12336, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 176723734, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 12336, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 176723734, 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:35.665 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:35.666 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 12336, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 12336, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 12336, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 12336, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.667 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 6820, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 942108, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 6820, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 372, 2023-10-06T22:26:35.668 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 942108, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55de586ba6e0": { 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 47668, 2023-10-06T22:26:35.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 167438564, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 47668, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2185, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 167438564, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:35.670 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:35.671 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:35.671 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.671 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:35.671 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-10-06T22:26:35.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:35 smithi057 ceph-mon[98492]: pgmap v370: 97 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2023-10-06T22:26:35.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:35 smithi156 ceph-mon[134358]: pgmap v370: 97 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2023-10-06T22:26:35.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:35 smithi190 ceph-mon[134891]: pgmap v370: 97 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2023-10-06T22:26:36.192 INFO:tasks.check_counter:Daemon mds.f mds.exported=179 2023-10-06T22:26:36.192 INFO:tasks.check_counter:Daemon mds.f mds.imported=203 2023-10-06T22:26:36.192 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=2 2023-10-06T22:26:36.192 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:36.192 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:36.192 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=40 2023-10-06T22:26:36.192 DEBUG:tasks.check_counter:Getting stats from i 2023-10-06T22:26:36.193 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-06T22:26:36.873 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:26:37.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:37 smithi190 ceph-mon[134891]: pgmap v371: 97 pgs: 9 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 67 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2023-10-06T22:26:37.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:37 smithi057 ceph-mon[98492]: pgmap v371: 97 pgs: 9 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 67 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2023-10-06T22:26:37.503 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:37 smithi156 ceph-mon[134358]: pgmap v371: 97 pgs: 9 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 67 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2023-10-06T22:26:38.241 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-10-06T22:26:38.241 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:38.241 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 148689, 2023-10-06T22:26:38.241 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 221898, 2023-10-06T22:26:38.241 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 50607067, 2023-10-06T22:26:38.241 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 234906729, 2023-10-06T22:26:38.241 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 20, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 8, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 6.745722660, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 3.153824144, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 4.023881200, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.779705961, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 221888, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 109.393532448, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000493012 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:38.242 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 88292, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.031203027, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000353 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 51744, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 373920 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1711084, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1709090, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 287818068, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 364644882, 2023-10-06T22:26:38.243 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 13, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 55.612181996, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 22.126850155, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 38.784579631, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.777134825, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1709091, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 83.746315074, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000049000 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3180684, 2023-10-06T22:26:38.244 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.628550813, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000197 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 410624, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 237552 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 793506, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 802001, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 153145538, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 203416314, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 169, 2023-10-06T22:26:38.245 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 32.774756388, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 13.560626945, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 22.483588582, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.561878749, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 802005, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 43.926973137, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000054771 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1509162, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.233089321, 2023-10-06T22:26:38.246 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000154 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2058080, 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 162768 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:38.247 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 7144, 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 51.864809616, 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.007259911 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 29273, 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.776807220, 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000163181 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.248 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 68015, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 66355, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 66355, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1307.823894405, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.019709500 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 1652, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 172, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:38.249 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 4246, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 53, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 31936, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 22214, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 9607, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 115, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 200, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 23026, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 3, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 3, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 6, 2023-10-06T22:26:38.250 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 108059, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 93431, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 1519, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 97, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 16, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 220, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 78, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 14709, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 142, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 24546, 2023-10-06T22:26:38.251 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 1880, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 4, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 246, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 6, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 1880, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 66, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 6, 2023-10-06T22:26:38.252 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 60, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 1814, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 118, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 10222, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 28920, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 4, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 13, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 26814, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 10813, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 455, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 29009 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.253 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 6, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 354, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 400, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 110, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 15822, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 15712, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:38.254 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 6, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 78, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 59, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.255 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 3, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 75458, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 75352, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 75352, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 106, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 2028, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 145, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 144, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 144, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 1, 2023-10-06T22:26:38.256 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 1, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 3, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 245044944, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 245526793, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1867, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 68.446792165, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.036661377 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.257 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 17879, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 42973, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 25094, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 676, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 1200, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 524, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 31936, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 76927, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 44991, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 3, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 24671, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 24668, 2023-10-06T22:26:38.258 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 170660, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 207152 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 81375, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 508, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 68015, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 103, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 1483, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 10194, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 35.154984876, 2023-10-06T22:26:38.259 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003448595 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 461, 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 42.416613495, 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.092010007 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-10-06T22:26:38.260 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 27669, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 989.991028328, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.035779790 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000208002, 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000208002 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:38.262 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.004240989, 2023-10-06T22:26:38.263 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000353415 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 70, 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.088424680, 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.015548924 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-10-06T22:26:38.264 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 12.806196027, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 1.067183002 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2990, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 12.339574679, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004126948 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 694, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2.523239780, 2023-10-06T22:26:38.265 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003635792 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 9827, 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 168.068278242, 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.017102704 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.266 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 114, 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 16.267614939, 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.142698376 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6, 2023-10-06T22:26:38.267 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.517641999, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.752940333 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.026739316, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005347863 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.268 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.269 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.017740965, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002956827 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002080093, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000693364 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.270 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 14291, 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 22.599287995, 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001581365 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 2, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 5, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 3, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 2, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 7422, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 3711, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 711, 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:38.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:38.273 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:38.274 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:38.275 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 1775867, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 17360, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:38.276 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 169, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 64133648, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 631516, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.277 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 49017, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 293126471, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 37, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 49017, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 49017, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 10045.656618651, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.204942297 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-10-06T22:26:38.278 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 49017, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 104936 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 49017, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 2432, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 46585, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 3705, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 23124, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 374, 2023-10-06T22:26:38.279 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 2235, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 393, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 53, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 16338, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1892, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 46248, 2023-10-06T22:26:38.280 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 4, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-10-06T22:26:38.281 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-10-06T22:26:38.282 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 99, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 17, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 8193, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 344, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 2033 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-10-06T22:26:38.283 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 23 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 6, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-10-06T22:26:38.284 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 6, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 15712, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2653275, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 254740795, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2653275, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:38.285 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2653275, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 254740795, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 49017, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 295225847, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 49017, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 295225847, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.287 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 49017, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 49017, 2023-10-06T22:26:38.288 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 49017, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 49017, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 15712, 2023-10-06T22:26:38.289 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 2102128, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 15712, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 368, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2102128, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55841fe906e0": { 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 75458, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 241332489, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 75458, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1867, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 241332489, 2023-10-06T22:26:38.291 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:38.292 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:38.292 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:38.292 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:38.292 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.292 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:38.292 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-10-06T22:26:38.618 INFO:tasks.check_counter:Daemon mds.i mds.exported=78 2023-10-06T22:26:38.619 INFO:tasks.check_counter:Daemon mds.i mds.imported=142 2023-10-06T22:26:38.619 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-10-06T22:26:38.619 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=12 2023-10-06T22:26:38.619 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=6 2023-10-06T22:26:38.619 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=53 2023-10-06T22:26:38.620 DEBUG:tasks.check_counter:Getting stats from l 2023-10-06T22:26:38.620 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-06T22:26:39.176 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T22:26:39.586 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:39 smithi190 ceph-mon[134891]: pgmap v372: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:39.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:39 smithi057 ceph-mon[98492]: pgmap v372: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:39.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:39 smithi156 ceph-mon[134358]: pgmap v372: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:40.438 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1934, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1934, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 172563094, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 600685, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 8, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.237257103, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.043208754, 2023-10-06T22:26:40.439 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.119757117, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.060431520, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1933, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.096451124, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000049897 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T22:26:40.440 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 593, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 769, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 130205092, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 646087, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 9, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 7, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.117253667, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.016186989, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.079993730, 2023-10-06T22:26:40.441 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.015444283, 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 766, 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.074650380, 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000097454 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.442 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 462224 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1051, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1000, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 153112316, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 375558, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.263482259, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.026521925, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.133046898, 2023-10-06T22:26:40.443 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.111324198, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 999, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.103214744, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000103318 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1979648, 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 143552 2023-10-06T22:26:40.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5193, 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.169527125, 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000032645 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-10-06T22:26:40.446 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 61326, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 3244, 2023-10-06T22:26:40.447 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 54787, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 3295, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 56691, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 3037, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 1, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-10-06T22:26:40.448 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2023-10-06T22:26:40.449 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T22:26:40.450 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2023-10-06T22:26:40.451 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T22:26:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 136463, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-10-06T22:26:40.453 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 141, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 13, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 95981778, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 457092740, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 457080675, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.454 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 136463 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 20571, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 64835, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 44264, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 768, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 2047, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 1279, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 61326, 2023-10-06T22:26:40.455 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 112393, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 51067, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 157856, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 207152 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2023-10-06T22:26:40.456 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-10-06T22:26:40.457 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.459 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.460 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.461 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-10-06T22:26:40.462 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.464 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.465 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.466 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.467 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.468 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2701, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-10-06T22:26:40.469 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T22:26:40.470 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T22:26:40.471 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-10-06T22:26:40.472 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 953133, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 21091, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8064, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 169, 2023-10-06T22:26:40.473 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 90185315, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 889015, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-10-06T22:26:40.474 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 3267, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 21, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 3, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 3267, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3267, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 11.164681555, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003417410 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-10-06T22:26:40.475 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3267, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3268 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 3267, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 3267, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 1955, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1310, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2023-10-06T22:26:40.476 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-10-06T22:26:40.477 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-10-06T22:26:40.478 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-10-06T22:26:40.479 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 99, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 16, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 11, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 11, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.480 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-10-06T22:26:40.481 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3574, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 455540428, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3574, 2023-10-06T22:26:40.482 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3574, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 455540428, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:40.483 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-10-06T22:26:40.484 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:40.484 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:40.484 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:40.484 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3267, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 452996202, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3267, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 452996202, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.486 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3267, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3267, 2023-10-06T22:26:40.487 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3267, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3267, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:40.488 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.489 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55b292462960": { 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-10-06T22:26:40.490 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-10-06T22:26:40.491 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-10-06T22:26:40.832 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-10-06T22:26:40.833 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-10-06T22:26:40.833 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-10-06T22:26:40.833 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-10-06T22:26:40.833 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T22:26:40.833 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2023-10-06T22:26:40.833 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T22:26:40.844 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T22:26:40.844 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:26:40.844 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T22:26:40.895 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:26:40.895 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T22:26:40.942 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:26:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.190+0000 7f3b31bff700 1 -- 172.21.15.57:0/1529253870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c1061d0 msgr2=0x7f3b2c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.190+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1529253870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c1061d0 0x7f3b2c10dd50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b0a0 tx=0x7f3b2400b3b0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.191+0000 7f3b31bff700 1 -- 172.21.15.57:0/1529253870 shutdown_connections 2023-10-06T22:26:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.191+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1529253870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c1061d0 0x7f3b2c10dd50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.191+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1529253870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 0x7f3b2c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.191+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1529253870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b2c101a60 0x7f3b2c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.191+0000 7f3b31bff700 1 -- 172.21.15.57:0/1529253870 >> 172.21.15.57:0/1529253870 conn(0x7f3b2c0fc8d0 msgr2=0x7f3b2c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.191+0000 7f3b31bff700 1 -- 172.21.15.57:0/1529253870 shutdown_connections 2023-10-06T22:26:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.191+0000 7f3b31bff700 1 -- 172.21.15.57:0/1529253870 wait complete. 2023-10-06T22:26:41.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 Processor -- start 2023-10-06T22:26:41.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 -- start start 2023-10-06T22:26:41.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 0x7f3b2c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b2c1061d0 0x7f3b2c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c19dc80 0x7f3b2c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c19e700 con 0x7f3b2c1010b0 2023-10-06T22:26:41.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c19e870 con 0x7f3b2c1061d0 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b31bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c19ea30 con 0x7f3b2c19dc80 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.192+0000 7f3b2b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 0x7f3b2c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c19dc80 0x7f3b2c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 0x7f3b2c1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55532/0 (socket says 172.21.15.57:55532) 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2b7fe700 1 -- 172.21.15.57:0/1610705150 learned_addr learned my addr 172.21.15.57:0/1610705150 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b23fff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b2c1061d0 0x7f3b2c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2bfff700 1 -- 172.21.15.57:0/1610705150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b2c1061d0 msgr2=0x7f3b2c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2bfff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b2c1061d0 0x7f3b2c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2bfff700 1 -- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 msgr2=0x7f3b2c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2bfff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 0x7f3b2c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2bfff700 1 -- 172.21.15.57:0/1610705150 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24009d00 con 0x7f3b2c19dc80 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2b7fe700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 0x7f3b2c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b2bfff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c19dc80 0x7f3b2c19e100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009fd0 tx=0x7f3b2400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:41.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.193+0000 7f3b297fa700 1 -- 172.21.15.57:0/1610705150 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400e040 con 0x7f3b2c19dc80 2023-10-06T22:26:41.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.194+0000 7f3b297fa700 1 -- 172.21.15.57:0/1610705150 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b240094a0 con 0x7f3b2c19dc80 2023-10-06T22:26:41.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.194+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c1a5f30 con 0x7f3b2c19dc80 2023-10-06T22:26:41.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.194+0000 7f3b297fa700 1 -- 172.21.15.57:0/1610705150 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b240052a0 con 0x7f3b2c19dc80 2023-10-06T22:26:41.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.194+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c1a63b0 con 0x7f3b2c19dc80 2023-10-06T22:26:41.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.195+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2c067cd0 con 0x7f3b2c19dc80 2023-10-06T22:26:41.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.196+0000 7f3b297fa700 1 -- 172.21.15.57:0/1610705150 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3b2401d480 con 0x7f3b2c19dc80 2023-10-06T22:26:41.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.198+0000 7f3b297fa700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3b180a16b0 0x7f3b180a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.198+0000 7f3b297fa700 1 -- 172.21.15.57:0/1610705150 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f3b240ce1b0 con 0x7f3b2c19dc80 2023-10-06T22:26:41.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.198+0000 7f3b2b7fe700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3b180a16b0 0x7f3b180a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.198+0000 7f3b2b7fe700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3b180a16b0 0x7f3b180a3b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c011550 tx=0x7f3b1c009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:41.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.200+0000 7f3b297fa700 1 -- 172.21.15.57:0/1610705150 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3b24097550 con 0x7f3b2c19dc80 2023-10-06T22:26:41.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.412+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b2c101c30 con 0x7f3b2c19dc80 2023-10-06T22:26:41.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.414+0000 7f3b297fa700 1 -- 172.21.15.57:0/1610705150 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+23045 (secure 0 0 0) 0x7f3b2409ab70 con 0x7f3b2c19dc80 2023-10-06T22:26:41.416 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:26:41.416 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":99,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:26:10.301929+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/339414861":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/117215786":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/409950778":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6800/409950778":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2575175349":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3940203258":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:26:41.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3b180a16b0 msgr2=0x7f3b180a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3b180a16b0 0x7f3b180a3b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c011550 tx=0x7f3b1c009260 comp rx=0 tx=0).stop 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c19dc80 msgr2=0x7f3b2c19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c19dc80 0x7f3b2c19e100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009fd0 tx=0x7f3b2400bc80 comp rx=0 tx=0).stop 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 shutdown_connections 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f3b180a16b0 0x7f3b180a3b70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c19dc80 0x7f3b2c19e100 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c1010b0 0x7f3b2c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 --2- 172.21.15.57:0/1610705150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b2c1061d0 0x7f3b2c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 >> 172.21.15.57:0/1610705150 conn(0x7f3b2c0fc8d0 msgr2=0x7f3b2c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 shutdown_connections 2023-10-06T22:26:41.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.417+0000 7f3b31bff700 1 -- 172.21.15.57:0/1610705150 wait complete. 2023-10-06T22:26:41.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T22:08:24.134657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T22:12:41.630708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T22:12:42.661165+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T22:26:41.438 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 -- 172.21.15.57:0/939208472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1010b0 msgr2=0x7f1e5c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 --2- 172.21.15.57:0/939208472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c101490 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009990 tx=0x7f1e50009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 -- 172.21.15.57:0/939208472 shutdown_connections 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 --2- 172.21.15.57:0/939208472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e5c1061d0 0x7f1e5c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 --2- 172.21.15.57:0/939208472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c101490 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 --2- 172.21.15.57:0/939208472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c101a60 0x7f1e5c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 -- 172.21.15.57:0/939208472 >> 172.21.15.57:0/939208472 conn(0x7f1e5c0fc950 msgr2=0x7f1e5c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 -- 172.21.15.57:0/939208472 shutdown_connections 2023-10-06T22:26:41.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.642+0000 7f1e62fff700 1 -- 172.21.15.57:0/939208472 wait complete. 2023-10-06T22:26:41.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.644+0000 7f1e62fff700 1 Processor -- start 2023-10-06T22:26:41.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.644+0000 7f1e62fff700 1 -- start start 2023-10-06T22:26:41.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e62fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e62fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1061d0 0x7f1e5c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e62fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e5c19ba40 0x7f1e5c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e62fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c19c4c0 con 0x7f1e5c1061d0 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e62fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c19c630 con 0x7f1e5c19ba40 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e62fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c19c7f0 con 0x7f1e5c1010b0 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e5bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1061d0 0x7f1e5c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:54384/0 (socket says 172.21.15.57:54384) 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 -- 172.21.15.57:0/2548289047 learned_addr learned my addr 172.21.15.57:0/2548289047 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e6159c700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e5c19ba40 0x7f1e5c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 -- 172.21.15.57:0/2548289047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e5c19ba40 msgr2=0x7f1e5c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e5c19ba40 0x7f1e5c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 -- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1061d0 msgr2=0x7f1e5c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1061d0 0x7f1e5c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e60d9b700 1 -- 172.21.15.57:0/2548289047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500095f0 con 0x7f1e5c1010b0 2023-10-06T22:26:41.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.645+0000 7f1e5bfff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1061d0 0x7f1e5c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:41.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.646+0000 7f1e60d9b700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c1970a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005f50 tx=0x7f1e50004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:41.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.646+0000 7f1e59ffb700 1 -- 172.21.15.57:0/2548289047 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5001d070 con 0x7f1e5c1010b0 2023-10-06T22:26:41.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.646+0000 7f1e59ffb700 1 -- 172.21.15.57:0/2548289047 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e50005a00 con 0x7f1e5c1010b0 2023-10-06T22:26:41.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.646+0000 7f1e59ffb700 1 -- 172.21.15.57:0/2548289047 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50004b60 con 0x7f1e5c1010b0 2023-10-06T22:26:41.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.646+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c1a3cf0 con 0x7f1e5c1010b0 2023-10-06T22:26:41.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.646+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c1a40c0 con 0x7f1e5c1010b0 2023-10-06T22:26:41.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.647+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c067c60 con 0x7f1e5c1010b0 2023-10-06T22:26:41.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.649+0000 7f1e59ffb700 1 -- 172.21.15.57:0/2548289047 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1e5000fb40 con 0x7f1e5c1010b0 2023-10-06T22:26:41.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.651+0000 7f1e59ffb700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1e4c0a15f0 0x7f1e4c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:41.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.651+0000 7f1e59ffb700 1 -- 172.21.15.57:0/2548289047 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f1e500ce6c0 con 0x7f1e5c1010b0 2023-10-06T22:26:41.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.651+0000 7f1e5bfff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1e4c0a15f0 0x7f1e4c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:41.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.652+0000 7f1e5bfff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1e4c0a15f0 0x7f1e4c0a3ab0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c1986b0 tx=0x7f1e48009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:41.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.653+0000 7f1e59ffb700 1 -- 172.21.15.57:0/2548289047 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1e50097b20 con 0x7f1e5c1010b0 2023-10-06T22:26:41.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:41 smithi057 ceph-mon[98492]: pgmap v373: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2023-10-06T22:26:41.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:41 smithi156 ceph-mon[134358]: pgmap v373: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2023-10-06T22:26:41.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:41 smithi190 ceph-mon[134891]: pgmap v373: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2023-10-06T22:26:41.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.865+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e5c110dd0 con 0x7f1e5c1010b0 2023-10-06T22:26:41.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.866+0000 7f1e59ffb700 1 -- 172.21.15.57:0/2548289047 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e5002f020 con 0x7f1e5c1010b0 2023-10-06T22:26:41.867 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-06T22:26:41.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1e4c0a15f0 msgr2=0x7f1e4c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1e4c0a15f0 0x7f1e4c0a3ab0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c1986b0 tx=0x7f1e48009260 comp rx=0 tx=0).stop 2023-10-06T22:26:41.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c1010b0 msgr2=0x7f1e5c1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:41.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c1970a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005f50 tx=0x7f1e50004730 comp rx=0 tx=0).stop 2023-10-06T22:26:41.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 shutdown_connections 2023-10-06T22:26:41.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1e4c0a15f0 0x7f1e4c0a3ab0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e5c19ba40 0x7f1e5c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c1010b0 0x7f1e5c1970a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 --2- 172.21.15.57:0/2548289047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e5c1061d0 0x7f1e5c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:41.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.868+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 >> 172.21.15.57:0/2548289047 conn(0x7f1e5c0fc950 msgr2=0x7f1e5c104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:41.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.869+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 shutdown_connections 2023-10-06T22:26:41.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:41.869+0000 7f1e62fff700 1 -- 172.21.15.57:0/2548289047 wait complete. 2023-10-06T22:26:41.888 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T22:26:42.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 -- 172.21.15.57:0/3377827018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01061d0 msgr2=0x7f45e010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 --2- 172.21.15.57:0/3377827018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01061d0 0x7f45e010dd50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f45d800b0a0 tx=0x7f45d800b3b0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 -- 172.21.15.57:0/3377827018 shutdown_connections 2023-10-06T22:26:42.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 --2- 172.21.15.57:0/3377827018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01061d0 0x7f45e010dd50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 --2- 172.21.15.57:0/3377827018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e01010b0 0x7f45e0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 --2- 172.21.15.57:0/3377827018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e0101a60 0x7f45e0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 -- 172.21.15.57:0/3377827018 >> 172.21.15.57:0/3377827018 conn(0x7f45e00fc930 msgr2=0x7f45e00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:42.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.095+0000 7f45e81ff700 1 -- 172.21.15.57:0/3377827018 shutdown_connections 2023-10-06T22:26:42.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.096+0000 7f45e81ff700 1 -- 172.21.15.57:0/3377827018 wait complete. 2023-10-06T22:26:42.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.096+0000 7f45e81ff700 1 Processor -- start 2023-10-06T22:26:42.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e81ff700 1 -- start start 2023-10-06T22:26:42.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e81ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01010b0 0x7f45e00731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e81ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e01061d0 0x7f45e00736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e81ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e0073c30 0x7f45e01a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e81ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01a58f0 con 0x7f45e01010b0 2023-10-06T22:26:42.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e81ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01a5a60 con 0x7f45e01061d0 2023-10-06T22:26:42.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e81ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01a5c20 con 0x7f45e0073c30 2023-10-06T22:26:42.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e5f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01010b0 0x7f45e00731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e5f9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01010b0 0x7f45e00731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34808/0 (socket says 172.21.15.57:34808) 2023-10-06T22:26:42.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e5f9b700 1 -- 172.21.15.57:0/2073913115 learned_addr learned my addr 172.21.15.57:0/2073913115 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:42.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.097+0000 7f45e579a700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e01061d0 0x7f45e00736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e679c700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e0073c30 0x7f45e01a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e5f9b700 1 -- 172.21.15.57:0/2073913115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e0073c30 msgr2=0x7f45e01a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e5f9b700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e0073c30 0x7f45e01a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e5f9b700 1 -- 172.21.15.57:0/2073913115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e01061d0 msgr2=0x7f45e00736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e5f9b700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e01061d0 0x7f45e00736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e5f9b700 1 -- 172.21.15.57:0/2073913115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8009d00 con 0x7f45e01010b0 2023-10-06T22:26:42.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e679c700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e0073c30 0x7f45e01a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:26:42.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e5f9b700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01010b0 0x7f45e00731b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e9a0 tx=0x7f45dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:42.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45d6ffd700 1 -- 172.21.15.57:0/2073913115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc009790 con 0x7f45e01010b0 2023-10-06T22:26:42.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45d6ffd700 1 -- 172.21.15.57:0/2073913115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45dc00cbd0 con 0x7f45e01010b0 2023-10-06T22:26:42.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45d6ffd700 1 -- 172.21.15.57:0/2073913115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc018680 con 0x7f45e01010b0 2023-10-06T22:26:42.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e01a5f00 con 0x7f45e01010b0 2023-10-06T22:26:42.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.098+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e01a6330 con 0x7f45e01010b0 2023-10-06T22:26:42.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.099+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e0067c60 con 0x7f45e01010b0 2023-10-06T22:26:42.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.102+0000 7f45d6ffd700 1 -- 172.21.15.57:0/2073913115 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f45dc00cd40 con 0x7f45e01010b0 2023-10-06T22:26:42.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.103+0000 7f45d6ffd700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f45cc0a1670 0x7f45cc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.103+0000 7f45d6ffd700 1 -- 172.21.15.57:0/2073913115 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f45dc014070 con 0x7f45e01010b0 2023-10-06T22:26:42.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.103+0000 7f45e579a700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f45cc0a1670 0x7f45cc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.104+0000 7f45e579a700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f45cc0a1670 0x7f45cc0a3b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f45e00749a0 tx=0x7f45d0009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:42.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.105+0000 7f45d6ffd700 1 -- 172.21.15.57:0/2073913115 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f45dc0968c0 con 0x7f45e01010b0 2023-10-06T22:26:42.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.317+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45e0101a60 con 0x7f45e01010b0 2023-10-06T22:26:42.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.317+0000 7f45d6ffd700 1 -- 172.21.15.57:0/2073913115 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45dc099ee0 con 0x7f45e01010b0 2023-10-06T22:26:42.319 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-06T22:26:42.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f45cc0a1670 msgr2=0x7f45cc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f45cc0a1670 0x7f45cc0a3b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f45e00749a0 tx=0x7f45d0009260 comp rx=0 tx=0).stop 2023-10-06T22:26:42.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01010b0 msgr2=0x7f45e00731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01010b0 0x7f45e00731b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e9a0 tx=0x7f45dc00ecb0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 shutdown_connections 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f45cc0a1670 0x7f45cc0a3b30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e0073c30 0x7f45e01a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45e01010b0 0x7f45e00731b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 --2- 172.21.15.57:0/2073913115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e01061d0 0x7f45e00736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 >> 172.21.15.57:0/2073913115 conn(0x7f45e00fc930 msgr2=0x7f45e00fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 shutdown_connections 2023-10-06T22:26:42.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.320+0000 7f45e81ff700 1 -- 172.21.15.57:0/2073913115 wait complete. 2023-10-06T22:26:42.340 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 -- 172.21.15.57:0/3843443637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 msgr2=0x7f3770101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 --2- 172.21.15.57:0/3843443637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770101490 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3760009990 tx=0x7f3760009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 -- 172.21.15.57:0/3843443637 shutdown_connections 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 --2- 172.21.15.57:0/3843443637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37701061d0 0x7f377010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 --2- 172.21.15.57:0/3843443637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770101490 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 --2- 172.21.15.57:0/3843443637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3770101a60 0x7f3770105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 -- 172.21.15.57:0/3843443637 >> 172.21.15.57:0/3843443637 conn(0x7f37700fc950 msgr2=0x7f37700fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 -- 172.21.15.57:0/3843443637 shutdown_connections 2023-10-06T22:26:42.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.545+0000 7f37769ff700 1 -- 172.21.15.57:0/3843443637 wait complete. 2023-10-06T22:26:42.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.546+0000 7f37769ff700 1 Processor -- start 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.546+0000 7f37769ff700 1 -- start start 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f37769ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f37769ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37701061d0 0x7f3770197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f37769ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377019b9e0 0x7f377019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f37769ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377019c460 con 0x7f37701010b0 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f37769ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377019c5d0 con 0x7f377019b9e0 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f37769ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377019c790 con 0x7f37701061d0 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34830/0 (socket says 172.21.15.57:34830) 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 -- 172.21.15.57:0/2877103028 learned_addr learned my addr 172.21.15.57:0/2877103028 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:42.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f3774f9c700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377019b9e0 0x7f377019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376f7fe700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37701061d0 0x7f3770197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 -- 172.21.15.57:0/2877103028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37701061d0 msgr2=0x7f3770197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37701061d0 0x7f3770197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 -- 172.21.15.57:0/2877103028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377019b9e0 msgr2=0x7f377019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377019b9e0 0x7f377019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.547+0000 7f376ffff700 1 -- 172.21.15.57:0/2877103028 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37600095f0 con 0x7f37701010b0 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f376f7fe700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37701061d0 0x7f3770197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f3774f9c700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377019b9e0 0x7f377019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f376ffff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770197040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3760005910 tx=0x7f37600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f376d7fa700 1 -- 172.21.15.57:0/2877103028 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376001d070 con 0x7f37701010b0 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f376d7fa700 1 -- 172.21.15.57:0/2877103028 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3760022470 con 0x7f37701010b0 2023-10-06T22:26:42.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37701a3c90 con 0x7f37701010b0 2023-10-06T22:26:42.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f376d7fa700 1 -- 172.21.15.57:0/2877103028 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760017650 con 0x7f37701010b0 2023-10-06T22:26:42.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.548+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37701a40b0 con 0x7f37701010b0 2023-10-06T22:26:42.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.549+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3770067c60 con 0x7f37701010b0 2023-10-06T22:26:42.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.551+0000 7f376d7fa700 1 -- 172.21.15.57:0/2877103028 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f37600225e0 con 0x7f37701010b0 2023-10-06T22:26:42.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.552+0000 7f376d7fa700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f37580a15f0 0x7f37580a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:42.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.553+0000 7f376d7fa700 1 -- 172.21.15.57:0/2877103028 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f37600ce740 con 0x7f37701010b0 2023-10-06T22:26:42.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.553+0000 7f376f7fe700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f37580a15f0 0x7f37580a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:42.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.553+0000 7f376f7fe700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f37580a15f0 0x7f37580a3ab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3770198650 tx=0x7f376400b5f0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:42.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.556+0000 7f376d7fa700 1 -- 172.21.15.57:0/2877103028 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3760097a80 con 0x7f37701010b0 2023-10-06T22:26:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:42 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1610705150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:42 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2548289047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:26:42.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:42 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2073913115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:26:42.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:42 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1610705150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:42.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:42 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2548289047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:26:42.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:42 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2073913115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:26:42.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.768+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37701019d0 con 0x7f37701010b0 2023-10-06T22:26:42.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.768+0000 7f376d7fa700 1 -- 172.21.15.57:0/2877103028 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f376009b0a0 con 0x7f37701010b0 2023-10-06T22:26:42.770 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-06T22:26:42.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f37580a15f0 msgr2=0x7f37580a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f37580a15f0 0x7f37580a3ab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3770198650 tx=0x7f376400b5f0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 msgr2=0x7f3770197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770197040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3760005910 tx=0x7f37600057f0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 shutdown_connections 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f37580a15f0 0x7f37580a3ab0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377019b9e0 0x7f377019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f37701010b0 0x7f3770197040 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 --2- 172.21.15.57:0/2877103028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37701061d0 0x7f3770197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 >> 172.21.15.57:0/2877103028 conn(0x7f37700fc950 msgr2=0x7f3770104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 shutdown_connections 2023-10-06T22:26:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.771+0000 7f37769ff700 1 -- 172.21.15.57:0/2877103028 wait complete. 2023-10-06T22:26:42.791 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-06T22:26:42.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:42 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1610705150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:42.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:42 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2548289047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T22:26:42.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:42 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2073913115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 -- 172.21.15.57:0/1982005897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c073b80 msgr2=0x7f5e7c10d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 --2- 172.21.15.57:0/1982005897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c073b80 0x7f5e7c10d1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009990 tx=0x7f5e68009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 -- 172.21.15.57:0/1982005897 shutdown_connections 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 --2- 172.21.15.57:0/1982005897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c10d700 0x7f5e7c10faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 --2- 172.21.15.57:0/1982005897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 0x7f5e7c073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 --2- 172.21.15.57:0/1982005897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c073b80 0x7f5e7c10d1c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 -- 172.21.15.57:0/1982005897 >> 172.21.15.57:0/1982005897 conn(0x7f5e7c0fc930 msgr2=0x7f5e7c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 -- 172.21.15.57:0/1982005897 shutdown_connections 2023-10-06T22:26:42.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.996+0000 7f5e82dff700 1 -- 172.21.15.57:0/1982005897 wait complete. 2023-10-06T22:26:43.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.997+0000 7f5e82dff700 1 Processor -- start 2023-10-06T22:26:43.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.997+0000 7f5e82dff700 1 -- start start 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e82dff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 0x7f5e7c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e82dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c073b80 0x7f5e7c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e82dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c10d700 0x7f5e7c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e82dff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c19e170 con 0x7f5e7c073260 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e82dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c19e2e0 con 0x7f5e7c073b80 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e82dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c19e4a0 con 0x7f5e7c10d700 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e80b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 0x7f5e7c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e80b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 0x7f5e7c199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34848/0 (socket says 172.21.15.57:34848) 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e80b9b700 1 -- 172.21.15.57:0/2374909878 learned_addr learned my addr 172.21.15.57:0/2374909878 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e8139c700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c10d700 0x7f5e7c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:43.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e7bfff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c073b80 0x7f5e7c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e80b9b700 1 -- 172.21.15.57:0/2374909878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c10d700 msgr2=0x7f5e7c19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e80b9b700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c10d700 0x7f5e7c19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.998+0000 7f5e80b9b700 1 -- 172.21.15.57:0/2374909878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c073b80 msgr2=0x7f5e7c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e80b9b700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c073b80 0x7f5e7c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e80b9b700 1 -- 172.21.15.57:0/2374909878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70009520 con 0x7f5e7c073260 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e7bfff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c073b80 0x7f5e7c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e80b9b700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 0x7f5e7c199190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000eb30 tx=0x7f5e7000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e79ffb700 1 -- 172.21.15.57:0/2374909878 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70009880 con 0x7f5e7c073260 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e680095f0 con 0x7f5e7c073260 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e79ffb700 1 -- 172.21.15.57:0/2374909878 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e700044e0 con 0x7f5e7c073260 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e79ffb700 1 -- 172.21.15.57:0/2374909878 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70003e90 con 0x7f5e7c073260 2023-10-06T22:26:43.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:42.999+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c19e970 con 0x7f5e7c073260 2023-10-06T22:26:43.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.000+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c067c60 con 0x7f5e7c073260 2023-10-06T22:26:43.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.003+0000 7f5e79ffb700 1 -- 172.21.15.57:0/2374909878 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5e70004000 con 0x7f5e7c073260 2023-10-06T22:26:43.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.004+0000 7f5e79ffb700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5e6c0a15f0 0x7f5e6c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:43.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.004+0000 7f5e79ffb700 1 -- 172.21.15.57:0/2374909878 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f5e70014070 con 0x7f5e7c073260 2023-10-06T22:26:43.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.004+0000 7f5e7bfff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5e6c0a15f0 0x7f5e6c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:43.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.005+0000 7f5e7bfff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5e6c0a15f0 0x7f5e6c0a3ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800b2e0 tx=0x7f5e68005fd0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:43.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.006+0000 7f5e79ffb700 1 -- 172.21.15.57:0/2374909878 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5e700969d0 con 0x7f5e7c073260 2023-10-06T22:26:43.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.236+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e7c113260 con 0x7f5e7c073260 2023-10-06T22:26:43.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.236+0000 7f5e79ffb700 1 -- 172.21.15.57:0/2374909878 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) v1 ==== 81+0+942 (secure 0 0 0) 0x7f5e70099ff0 con 0x7f5e7c073260 2023-10-06T22:26:43.237 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/339414861 2023-10-07T22:14:45.661821+0000 2023-10-06T22:26:43.237 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/117215786 2023-10-07T22:14:45.661821+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3599227077 2023-10-07T22:06:05.527819+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/4144294466 2023-10-07T22:06:05.527819+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/4144294466 2023-10-07T22:06:05.527819+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/409950778 2023-10-07T22:14:45.661821+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1942116136 2023-10-07T22:05:38.356695+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3830563323 2023-10-07T22:06:05.527819+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/1962312800 2023-10-07T22:05:38.356695+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/1962312800 2023-10-07T22:05:38.356695+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/409950778 2023-10-07T22:14:45.661821+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3188291259 2023-10-07T22:05:38.356695+0000 2023-10-06T22:26:43.238 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2575175349 2023-10-07T22:14:45.661821+0000 2023-10-06T22:26:43.239 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3827189862 2023-10-07T22:05:38.356695+0000 2023-10-06T22:26:43.239 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3940203258 2023-10-07T22:14:45.661821+0000 2023-10-06T22:26:43.239 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2091122692 2023-10-07T22:06:05.527819+0000 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.238+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5e6c0a15f0 msgr2=0x7f5e6c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.238+0000 7f5e82dff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5e6c0a15f0 0x7f5e6c0a3ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800b2e0 tx=0x7f5e68005fd0 comp rx=0 tx=0).stop 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.238+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 msgr2=0x7f5e7c199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.238+0000 7f5e82dff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 0x7f5e7c199190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000eb30 tx=0x7f5e7000ee40 comp rx=0 tx=0).stop 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 shutdown_connections 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f5e6c0a15f0 0x7f5e6c0a3ab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c10d700 0x7f5e7c19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:43.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e7c073260 0x7f5e7c199190 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:43.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 --2- 172.21.15.57:0/2374909878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e7c073b80 0x7f5e7c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:43.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 >> 172.21.15.57:0/2374909878 conn(0x7f5e7c0fc930 msgr2=0x7f5e7c10f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:43.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 shutdown_connections 2023-10-06T22:26:43.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:43.239+0000 7f5e82dff700 1 -- 172.21.15.57:0/2374909878 wait complete. 2023-10-06T22:26:43.241 INFO:teuthology.orchestra.run.smithi057.stderr:listed 16 entries 2023-10-06T22:26:43.258 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T22:26:43.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:26:43.258 DEBUG:teuthology.orchestra.run.smithi057:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T22:26:43.424 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:26:43.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:26:43.424 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T22:26:43.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:26:43.623 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-06T22:26:43.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:43 smithi057 ceph-mon[98492]: pgmap v374: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:43.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:43 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2877103028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:26:43.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:43 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2374909878' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T22:26:43.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:43 smithi156 ceph-mon[134358]: pgmap v374: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:43.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:43 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2877103028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:26:43.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:43 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2374909878' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T22:26:43.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:43 smithi190 ceph-mon[134891]: pgmap v374: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-10-06T22:26:43.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:43 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2877103028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T22:26:43.829 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:43 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2374909878' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T22:26:43.835 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T22:26:43.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:26:43.835 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T22:26:44.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:26:44.182 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-06T22:26:44.313 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T22:26:44.313 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-06T22:26:44.313 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T22:26:44.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348692k,nr_inodes=4087173,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/6a2a1b408cafd8fe0190437daf5a46f1381fd27dfb876a3742b1de99aa5ca736/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/6a2a1b408cafd8fe0190437daf5a46f1381fd27dfb876a3742b1de99aa5ca736/diff,workdir=/var/lib/containers/storage/overlay/6a2a1b408cafd8fe0190437daf5a46f1381fd27dfb876a3742b1de99aa5ca736/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/adcc9c4f61d06a98819cd8603c83674715c1cb89fc37ecb0ccf966728da5f939/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/adcc9c4f61d06a98819cd8603c83674715c1cb89fc37ecb0ccf966728da5f939/diff,workdir=/var/lib/containers/storage/overlay/adcc9c4f61d06a98819cd8603c83674715c1cb89fc37ecb0ccf966728da5f939/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/c1f36ff1f17b0dc525bea05aab53602ddda034a44de8fbeb0a66d6b797f559f5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/c1f36ff1f17b0dc525bea05aab53602ddda034a44de8fbeb0a66d6b797f559f5/diff,workdir=/var/lib/containers/storage/overlay/c1f36ff1f17b0dc525bea05aab53602ddda034a44de8fbeb0a66d6b797f559f5/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6bf209f0e8b66394892591a9a55abb0c428eb5eae46de13d6ba1246958a19e4d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/6bf209f0e8b66394892591a9a55abb0c428eb5eae46de13d6ba1246958a19e4d/diff,workdir=/var/lib/containers/storage/overlay/6bf209f0e8b66394892591a9a55abb0c428eb5eae46de13d6ba1246958a19e4d/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/879ca2f2b5878993d0051efe12b81f695d057287cce1460544e28892e403ef6a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/879ca2f2b5878993d0051efe12b81f695d057287cce1460544e28892e403ef6a/diff,workdir=/var/lib/containers/storage/overlay/879ca2f2b5878993d0051efe12b81f695d057287cce1460544e28892e403ef6a/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c9a128c0c95c5be33ce3f121ef6093693e78ade654db197cd23a1ffc0b33f359/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/c9a128c0c95c5be33ce3f121ef6093693e78ade654db197cd23a1ffc0b33f359/diff,workdir=/var/lib/containers/storage/overlay/c9a128c0c95c5be33ce3f121ef6093693e78ade654db197cd23a1ffc0b33f359/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/883f102088662773e2b28a3edb6d26751b336a339d8a1215bf2d1ad276626b6e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/883f102088662773e2b28a3edb6d26751b336a339d8a1215bf2d1ad276626b6e/diff,workdir=/var/lib/containers/storage/overlay/883f102088662773e2b28a3edb6d26751b336a339d8a1215bf2d1ad276626b6e/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3fcf597f4c12a6443cedb46829477b92a0b7905a90c4cffe5660b7141714a452/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/3fcf597f4c12a6443cedb46829477b92a0b7905a90c4cffe5660b7141714a452/diff,workdir=/var/lib/containers/storage/overlay/3fcf597f4c12a6443cedb46829477b92a0b7905a90c4cffe5660b7141714a452/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a8778b48fe030e2201650664b5067c517e5180abbef7813cc036e1e78b09a8cb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/a8778b48fe030e2201650664b5067c517e5180abbef7813cc036e1e78b09a8cb/diff,workdir=/var/lib/containers/storage/overlay/a8778b48fe030e2201650664b5067c517e5180abbef7813cc036e1e78b09a8cb/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/56af595c4c0d9b10a816421f31db4af4b6bb1b6bd07eeaa5a3b6893c8740e7cf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MVY5HOCTCAXMMI5KSQT6P2X6N3:/var/lib/containers/storage/overlay/l/CDAQJOMC7QFTHO6AVLO5EKGFM5,upperdir=/var/lib/containers/storage/overlay/56af595c4c0d9b10a816421f31db4af4b6bb1b6bd07eeaa5a3b6893c8740e7cf/diff,workdir=/var/lib/containers/storage/overlay/56af595c4c0d9b10a816421f31db4af4b6bb1b6bd07eeaa5a3b6893c8740e7cf/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-06T22:26:44.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:26:44.363 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-06T22:26:44.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T22:26:44.448 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-06T22:26:44.563 INFO:teuthology.orchestra.run.smithi057.stderr:Cannot find device "ceph-brx" 2023-10-06T22:26:44.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:26:44.566 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-06T22:26:44.577 INFO:tasks.cephadm:Teardown begin 2023-10-06T22:26:44.578 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T22:26:44.578 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:26:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 -- 172.21.15.57:0/108119826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f30101a60 msgr2=0x7f7f30105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 --2- 172.21.15.57:0/108119826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f30101a60 0x7f7f30105aa0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009990 tx=0x7f7f18009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 -- 172.21.15.57:0/108119826 shutdown_connections 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 --2- 172.21.15.57:0/108119826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f301061d0 0x7f7f3010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 --2- 172.21.15.57:0/108119826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f301010b0 0x7f7f30101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 --2- 172.21.15.57:0/108119826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f30101a60 0x7f7f30105aa0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 -- 172.21.15.57:0/108119826 >> 172.21.15.57:0/108119826 conn(0x7f7f300fc8d0 msgr2=0x7f7f300fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 -- 172.21.15.57:0/108119826 shutdown_connections 2023-10-06T22:26:44.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.781+0000 7f7f365ff700 1 -- 172.21.15.57:0/108119826 wait complete. 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.782+0000 7f7f365ff700 1 Processor -- start 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.782+0000 7f7f365ff700 1 -- start start 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f365ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f301010b0 0x7f7f30199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f365ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f301061d0 0x7f7f30199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f365ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f3019daf0 0x7f7f3019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f365ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3019e570 con 0x7f7f301010b0 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f365ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3019e6e0 con 0x7f7f3019daf0 2023-10-06T22:26:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f365ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3019e8a0 con 0x7f7f301061d0 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f301010b0 0x7f7f30199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f301010b0 0x7f7f30199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34860/0 (socket says 172.21.15.57:34860) 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 -- 172.21.15.57:0/3166974012 learned_addr learned my addr 172.21.15.57:0/3166974012 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f27fff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f301061d0 0x7f7f30199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f34b9c700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f3019daf0 0x7f7f3019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 -- 172.21.15.57:0/3166974012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f301061d0 msgr2=0x7f7f30199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f301061d0 0x7f7f30199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 -- 172.21.15.57:0/3166974012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f3019daf0 msgr2=0x7f7f3019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f3019daf0 0x7f7f3019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f2ffff700 1 -- 172.21.15.57:0/3166974012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180095f0 con 0x7f7f301010b0 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.783+0000 7f7f27fff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f301061d0 0x7f7f30199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:26:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.784+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f301010b0 0x7f7f30199150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000e9a0 tx=0x7f7f2000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.784+0000 7f7f2dffb700 1 -- 172.21.15.57:0/3166974012 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20009790 con 0x7f7f301010b0 2023-10-06T22:26:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.784+0000 7f7f2dffb700 1 -- 172.21.15.57:0/3166974012 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f2000cbd0 con 0x7f7f301010b0 2023-10-06T22:26:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.784+0000 7f7f2dffb700 1 -- 172.21.15.57:0/3166974012 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20018680 con 0x7f7f301010b0 2023-10-06T22:26:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.784+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f301a5e00 con 0x7f7f301010b0 2023-10-06T22:26:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.784+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f301a6230 con 0x7f7f301010b0 2023-10-06T22:26:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.785+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f30067cd0 con 0x7f7f301010b0 2023-10-06T22:26:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.787+0000 7f7f2dffb700 1 -- 172.21.15.57:0/3166974012 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7f2000cd40 con 0x7f7f301010b0 2023-10-06T22:26:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.788+0000 7f7f2dffb700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7f1c0a1630 0x7f7f1c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.789+0000 7f7f2dffb700 1 -- 172.21.15.57:0/3166974012 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f7f20014070 con 0x7f7f301010b0 2023-10-06T22:26:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.789+0000 7f7f27fff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7f1c0a1630 0x7f7f1c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.789+0000 7f7f27fff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7f1c0a1630 0x7f7f1c0a3af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800b2e0 tx=0x7f7f18005fd0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:44.792+0000 7f7f2dffb700 1 -- 172.21.15.57:0/3166974012 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7f20052100 con 0x7f7f301010b0 2023-10-06T22:26:45.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.054+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f30101c80 con 0x7f7f301010b0 2023-10-06T22:26:45.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.054+0000 7f7f2dffb700 1 -- 172.21.15.57:0/3166974012 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+11404 (secure 0 0 0) 0x7f7f200b1ca0 con 0x7f7f301010b0 2023-10-06T22:26:45.056 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:26:45.056 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:26:32.833023+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":131,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":128,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27134":{"gid":27134,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/1181822965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1181822965},{"type":"v1","addr":"172.21.15.156:6835","nonce":1181822965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:26:45.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7f1c0a1630 msgr2=0x7f7f1c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7f1c0a1630 0x7f7f1c0a3af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800b2e0 tx=0x7f7f18005fd0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f301010b0 msgr2=0x7f7f30199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f301010b0 0x7f7f30199150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000e9a0 tx=0x7f7f2000ecb0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 shutdown_connections 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f7f1c0a1630 0x7f7f1c0a3af0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f3019daf0 0x7f7f3019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f301010b0 0x7f7f30199150 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 --2- 172.21.15.57:0/3166974012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f301061d0 0x7f7f30199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 >> 172.21.15.57:0/3166974012 conn(0x7f7f300fc8d0 msgr2=0x7f7f300fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 shutdown_connections 2023-10-06T22:26:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.057+0000 7f7f365ff700 1 -- 172.21.15.57:0/3166974012 wait complete. 2023-10-06T22:26:45.060 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 65 2023-10-06T22:26:45.077 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T22:26:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.279+0000 7f1ecebff700 1 -- 172.21.15.57:0/2238327302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec8101a60 msgr2=0x7f1ec8105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.279+0000 7f1ecebff700 1 --2- 172.21.15.57:0/2238327302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec8101a60 0x7f1ec8105aa0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009990 tx=0x7f1eb4009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.280+0000 7f1ecebff700 1 -- 172.21.15.57:0/2238327302 shutdown_connections 2023-10-06T22:26:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.280+0000 7f1ecebff700 1 --2- 172.21.15.57:0/2238327302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec81061d0 0x7f1ec810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.280+0000 7f1ecebff700 1 --2- 172.21.15.57:0/2238327302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 0x7f1ec8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.280+0000 7f1ecebff700 1 --2- 172.21.15.57:0/2238327302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec8101a60 0x7f1ec8105aa0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.280+0000 7f1ecebff700 1 -- 172.21.15.57:0/2238327302 >> 172.21.15.57:0/2238327302 conn(0x7f1ec80fc8d0 msgr2=0x7f1ec80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.280+0000 7f1ecebff700 1 -- 172.21.15.57:0/2238327302 shutdown_connections 2023-10-06T22:26:45.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.280+0000 7f1ecebff700 1 -- 172.21.15.57:0/2238327302 wait complete. 2023-10-06T22:26:45.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 Processor -- start 2023-10-06T22:26:45.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 -- start start 2023-10-06T22:26:45.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 0x7f1ec81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec81061d0 0x7f1ec8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec819db80 0x7f1ec819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec819e600 con 0x7f1ec81061d0 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec819e770 con 0x7f1ec819db80 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.281+0000 7f1ecebff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec819e930 con 0x7f1ec81010b0 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec81061d0 0x7f1ec8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ecc99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 0x7f1ec81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ecc99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 0x7f1ec81991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.57:44112/0 (socket says 172.21.15.57:44112) 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ecc99b700 1 -- 172.21.15.57:0/1871681487 learned_addr learned my addr 172.21.15.57:0/1871681487 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ecd19c700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec819db80 0x7f1ec819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec7fff700 1 -- 172.21.15.57:0/1871681487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 msgr2=0x7f1ec81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec7fff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 0x7f1ec81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec7fff700 1 -- 172.21.15.57:0/1871681487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec819db80 msgr2=0x7f1ec819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec7fff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec819db80 0x7f1ec819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec7fff700 1 -- 172.21.15.57:0/1871681487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40095f0 con 0x7f1ec81061d0 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ecc99b700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 0x7f1ec81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec7fff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec81061d0 0x7f1ec8199720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1ec806a460 tx=0x7f1eb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.282+0000 7f1ec5ffb700 1 -- 172.21.15.57:0/1871681487 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb401d070 con 0x7f1ec81061d0 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.283+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec81a5e30 con 0x7f1ec81061d0 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.283+0000 7f1ec5ffb700 1 -- 172.21.15.57:0/1871681487 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb4021410 con 0x7f1ec81061d0 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.283+0000 7f1ec5ffb700 1 -- 172.21.15.57:0/1871681487 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4017650 con 0x7f1ec81061d0 2023-10-06T22:26:45.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.283+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec81a6200 con 0x7f1ec81061d0 2023-10-06T22:26:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.284+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec8067cd0 con 0x7f1ec81061d0 2023-10-06T22:26:45.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.286+0000 7f1ec5ffb700 1 -- 172.21.15.57:0/1871681487 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1eb40217f0 con 0x7f1ec81061d0 2023-10-06T22:26:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.287+0000 7f1ec5ffb700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1eb80a1630 0x7f1eb80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.287+0000 7f1ec5ffb700 1 -- 172.21.15.57:0/1871681487 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f1eb40ce950 con 0x7f1ec81061d0 2023-10-06T22:26:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.287+0000 7f1ecc99b700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1eb80a1630 0x7f1eb80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.288+0000 7f1ecc99b700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1eb80a1630 0x7f1eb80a3af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc011550 tx=0x7f1ebc009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:45.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.289+0000 7f1ec5ffb700 1 -- 172.21.15.57:0/1871681487 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1eb4097c90 con 0x7f1ec81061d0 2023-10-06T22:26:45.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.550+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ec819a530 con 0x7f1ec81061d0 2023-10-06T22:26:45.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.551+0000 7f1ec5ffb700 1 -- 172.21.15.57:0/1871681487 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+11404 (secure 0 0 0) 0x7f1eb409b2b0 con 0x7f1ec81061d0 2023-10-06T22:26:45.552 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:26:45.553 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/1029690870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":1029690870},{"type":"v1","addr":"172.21.15.57:6835","nonce":1029690870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/2389529664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":2389529664},{"type":"v1","addr":"172.21.15.57:6837","nonce":2389529664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T22:12:43.212955+0000","modified":"2023-10-06T22:26:32.833023+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24553,"mds_1":24551,"mds_2":24529,"mds_3":24517,"mds_4":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14523":{"gid":14523,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6839/3033019918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":3033019918},{"type":"v1","addr":"172.21.15.57:6839","nonce":3033019918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14535":{"gid":14535,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/490100321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":490100321},{"type":"v1","addr":"172.21.15.57:6841","nonce":490100321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/1142570288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1142570288},{"type":"v1","addr":"172.21.15.156:6833","nonce":1142570288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1813464152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1813464152},{"type":"v1","addr":"172.21.15.190:6833","nonce":1813464152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":131,"addr":"172.21.15.156:6837/3773867071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3773867071},{"type":"v1","addr":"172.21.15.156:6837","nonce":3773867071}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":128,"addr":"172.21.15.190:6835/871306260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":871306260},{"type":"v1","addr":"172.21.15.190:6835","nonce":871306260}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/3737066182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3737066182},{"type":"v1","addr":"172.21.15.190:6837","nonce":3737066182}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24551":{"gid":24551,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6839/1657784543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1657784543},{"type":"v1","addr":"172.21.15.156:6839","nonce":1657784543}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/751138264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":751138264},{"type":"v1","addr":"172.21.15.190:6839","nonce":751138264}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27134":{"gid":27134,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/1181822965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1181822965},{"type":"v1","addr":"172.21.15.156:6835","nonce":1181822965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T22:26:45.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1eb80a1630 msgr2=0x7f1eb80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1eb80a1630 0x7f1eb80a3af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc011550 tx=0x7f1ebc009260 comp rx=0 tx=0).stop 2023-10-06T22:26:45.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec81061d0 msgr2=0x7f1ec8199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec81061d0 0x7f1ec8199720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1ec806a460 tx=0x7f1eb4004730 comp rx=0 tx=0).stop 2023-10-06T22:26:45.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 shutdown_connections 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f1eb80a1630 0x7f1eb80a3af0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ec819db80 0x7f1ec819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ec81010b0 0x7f1ec81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 --2- 172.21.15.57:0/1871681487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec81061d0 0x7f1ec8199720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 >> 172.21.15.57:0/1871681487 conn(0x7f1ec80fc8d0 msgr2=0x7f1ec80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 shutdown_connections 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.554+0000 7f1ecebff700 1 -- 172.21.15.57:0/1871681487 wait complete. 2023-10-06T22:26:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 65 2023-10-06T22:26:45.574 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T22:26:45.635 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:45 smithi057 ceph-mon[98492]: pgmap v375: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-06T22:26:45.635 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:45 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3166974012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:26:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:45 smithi156 ceph-mon[134358]: pgmap v375: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-06T22:26:45.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:45 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3166974012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:26:45.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 -- 172.21.15.57:0/301270981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 msgr2=0x7f211c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 --2- 172.21.15.57:0/301270981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c101490 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2118009990 tx=0x7f2118009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 -- 172.21.15.57:0/301270981 shutdown_connections 2023-10-06T22:26:45.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 --2- 172.21.15.57:0/301270981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f211c1061d0 0x7f211c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 --2- 172.21.15.57:0/301270981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c101490 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 --2- 172.21.15.57:0/301270981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f211c101a60 0x7f211c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 -- 172.21.15.57:0/301270981 >> 172.21.15.57:0/301270981 conn(0x7f211c0fc8d0 msgr2=0x7f211c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:45.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 -- 172.21.15.57:0/301270981 shutdown_connections 2023-10-06T22:26:45.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.777+0000 7f21237ff700 1 -- 172.21.15.57:0/301270981 wait complete. 2023-10-06T22:26:45.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.778+0000 7f21237ff700 1 Processor -- start 2023-10-06T22:26:45.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.779+0000 7f21237ff700 1 -- start start 2023-10-06T22:26:45.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.779+0000 7f21237ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.779+0000 7f21237ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f211c1061d0 0x7f211c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.779+0000 7f21237ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f211c19b920 0x7f211c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.779+0000 7f21237ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19c3a0 con 0x7f211c1010b0 2023-10-06T22:26:45.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.779+0000 7f21237ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19c510 con 0x7f211c19b920 2023-10-06T22:26:45.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.779+0000 7f21237ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19c6d0 con 0x7f211c1061d0 2023-10-06T22:26:45.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34906/0 (socket says 172.21.15.57:34906) 2023-10-06T22:26:45.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 -- 172.21.15.57:0/2432855889 learned_addr learned my addr 172.21.15.57:0/2432855889 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:45.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f2120d9a700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f211c1061d0 0x7f211c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f2121d9c700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f211c19b920 0x7f211c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 -- 172.21.15.57:0/2432855889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f211c1061d0 msgr2=0x7f211c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f211c1061d0 0x7f211c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 -- 172.21.15.57:0/2432855889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f211c19b920 msgr2=0x7f211c19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f211c19b920 0x7f211c19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 -- 172.21.15.57:0/2432855889 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21180095f0 con 0x7f211c1010b0 2023-10-06T22:26:45.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f2121d9c700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f211c19b920 0x7f211c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T22:26:45.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.780+0000 7f212159b700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c196f80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2118005910 tx=0x7f2118004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:45.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.781+0000 7f21127fc700 1 -- 172.21.15.57:0/2432855889 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211801d070 con 0x7f211c1010b0 2023-10-06T22:26:45.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.781+0000 7f21127fc700 1 -- 172.21.15.57:0/2432855889 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2118022470 con 0x7f211c1010b0 2023-10-06T22:26:45.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.781+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c1a3bd0 con 0x7f211c1010b0 2023-10-06T22:26:45.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.781+0000 7f21127fc700 1 -- 172.21.15.57:0/2432855889 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118017650 con 0x7f211c1010b0 2023-10-06T22:26:45.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.781+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1a3ff0 con 0x7f211c1010b0 2023-10-06T22:26:45.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.782+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c067cd0 con 0x7f211c1010b0 2023-10-06T22:26:45.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.784+0000 7f21127fc700 1 -- 172.21.15.57:0/2432855889 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f21180225e0 con 0x7f211c1010b0 2023-10-06T22:26:45.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.786+0000 7f21127fc700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f21080a1630 0x7f21080a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.786+0000 7f21127fc700 1 -- 172.21.15.57:0/2432855889 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8536+0+0 (secure 0 0 0) 0x7f21180ce5c0 con 0x7f211c1010b0 2023-10-06T22:26:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.787+0000 7f2120d9a700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f21080a1630 0x7f21080a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.787+0000 7f2120d9a700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f21080a1630 0x7f21080a3af0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f211c198590 tx=0x7f210c009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:45.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:45.788+0000 7f21127fc700 1 -- 172.21.15.57:0/2432855889 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2118097900 con 0x7f211c1010b0 2023-10-06T22:26:45.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:45 smithi190 ceph-mon[134891]: pgmap v375: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-06T22:26:45.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:45 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3166974012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:26:46.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.001+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f211c1982d0 con 0x7f211c1010b0 2023-10-06T22:26:46.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.003+0000 7f21127fc700 1 -- 172.21.15.57:0/2432855889 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+23045 (secure 0 0 0) 0x7f211809af20 con 0x7f211c1010b0 2023-10-06T22:26:46.005 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-06T22:26:46.005 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":99,"fsid":"495b2048-6494-11ee-8db5-212e2dc638e7","created":"2023-10-06T22:05:04.289350+0000","modified":"2023-10-06T22:26:10.301929+0000","last_up_change":"2023-10-06T22:11:55.351471+0000","last_in_change":"2023-10-06T22:11:36.401186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T22:08:24.134657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T22:12:41.630708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T22:12:42.661165+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"25492bd6-3367-475f-b073-59bd613eab8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6803","nonce":2194428703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6805","nonce":2194428703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6809","nonce":2194428703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":2194428703},{"type":"v1","addr":"172.21.15.57:6807","nonce":2194428703}]},"public_addr":"172.21.15.57:6803/2194428703","cluster_addr":"172.21.15.57:6805/2194428703","heartbeat_back_addr":"172.21.15.57:6809/2194428703","heartbeat_front_addr":"172.21.15.57:6807/2194428703","state":["exists","up"]},{"osd":1,"uuid":"b7f3e993-648c-4873-9ecc-fee568a3a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6801","nonce":2038383629}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6803","nonce":2038383629}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6807","nonce":2038383629}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2038383629},{"type":"v1","addr":"172.21.15.156:6805","nonce":2038383629}]},"public_addr":"172.21.15.156:6801/2038383629","cluster_addr":"172.21.15.156:6803/2038383629","heartbeat_back_addr":"172.21.15.156:6807/2038383629","heartbeat_front_addr":"172.21.15.156:6805/2038383629","state":["exists","up"]},{"osd":2,"uuid":"4ce3125d-3f65-4413-9fcb-486f5b565ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6801","nonce":445057088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6803","nonce":445057088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6807","nonce":445057088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":445057088},{"type":"v1","addr":"172.21.15.190:6805","nonce":445057088}]},"public_addr":"172.21.15.190:6801/445057088","cluster_addr":"172.21.15.190:6803/445057088","heartbeat_back_addr":"172.21.15.190:6807/445057088","heartbeat_front_addr":"172.21.15.190:6805/445057088","state":["exists","up"]},{"osd":3,"uuid":"8df05b17-2d03-43d9-a57d-5abed0d51e65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6811","nonce":140569851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6813","nonce":140569851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6817","nonce":140569851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":140569851},{"type":"v1","addr":"172.21.15.57:6815","nonce":140569851}]},"public_addr":"172.21.15.57:6811/140569851","cluster_addr":"172.21.15.57:6813/140569851","heartbeat_back_addr":"172.21.15.57:6817/140569851","heartbeat_front_addr":"172.21.15.57:6815/140569851","state":["exists","up"]},{"osd":4,"uuid":"48270831-4102-4536-b87e-6f3373e1125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6809","nonce":3180512737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6811","nonce":3180512737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6815","nonce":3180512737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3180512737},{"type":"v1","addr":"172.21.15.156:6813","nonce":3180512737}]},"public_addr":"172.21.15.156:6809/3180512737","cluster_addr":"172.21.15.156:6811/3180512737","heartbeat_back_addr":"172.21.15.156:6815/3180512737","heartbeat_front_addr":"172.21.15.156:6813/3180512737","state":["exists","up"]},{"osd":5,"uuid":"9d3bfbaa-0c61-4ca2-a225-d9d44f0d6fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6809","nonce":3750215386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6811","nonce":3750215386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6815","nonce":3750215386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":3750215386},{"type":"v1","addr":"172.21.15.190:6813","nonce":3750215386}]},"public_addr":"172.21.15.190:6809/3750215386","cluster_addr":"172.21.15.190:6811/3750215386","heartbeat_back_addr":"172.21.15.190:6815/3750215386","heartbeat_front_addr":"172.21.15.190:6813/3750215386","state":["exists","up"]},{"osd":6,"uuid":"a987a0ba-b9f4-43e5-be83-274279f3b00c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6819","nonce":2530618700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6821","nonce":2530618700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6825","nonce":2530618700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":2530618700},{"type":"v1","addr":"172.21.15.57:6823","nonce":2530618700}]},"public_addr":"172.21.15.57:6819/2530618700","cluster_addr":"172.21.15.57:6821/2530618700","heartbeat_back_addr":"172.21.15.57:6825/2530618700","heartbeat_front_addr":"172.21.15.57:6823/2530618700","state":["exists","up"]},{"osd":7,"uuid":"e4b99ae3-8f83-4e52-a9c0-9536579dc262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6817","nonce":2039885607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6819","nonce":2039885607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6823","nonce":2039885607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2039885607},{"type":"v1","addr":"172.21.15.156:6821","nonce":2039885607}]},"public_addr":"172.21.15.156:6817/2039885607","cluster_addr":"172.21.15.156:6819/2039885607","heartbeat_back_addr":"172.21.15.156:6823/2039885607","heartbeat_front_addr":"172.21.15.156:6821/2039885607","state":["exists","up"]},{"osd":8,"uuid":"ae70781d-2670-4606-a280-604141f62861","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6817","nonce":1850516582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6819","nonce":1850516582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6823","nonce":1850516582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1850516582},{"type":"v1","addr":"172.21.15.190:6821","nonce":1850516582}]},"public_addr":"172.21.15.190:6817/1850516582","cluster_addr":"172.21.15.190:6819/1850516582","heartbeat_back_addr":"172.21.15.190:6823/1850516582","heartbeat_front_addr":"172.21.15.190:6821/1850516582","state":["exists","up"]},{"osd":9,"uuid":"ea6cc1cb-92e6-4cf1-a891-ca82e3e20079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6827","nonce":377680664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6829","nonce":377680664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6833","nonce":377680664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":377680664},{"type":"v1","addr":"172.21.15.57:6831","nonce":377680664}]},"public_addr":"172.21.15.57:6827/377680664","cluster_addr":"172.21.15.57:6829/377680664","heartbeat_back_addr":"172.21.15.57:6833/377680664","heartbeat_front_addr":"172.21.15.57:6831/377680664","state":["exists","up"]},{"osd":10,"uuid":"d591b412-f98a-438b-bd2c-fd491fe9cf11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6825","nonce":1227399112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6827","nonce":1227399112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6831","nonce":1227399112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1227399112},{"type":"v1","addr":"172.21.15.156:6829","nonce":1227399112}]},"public_addr":"172.21.15.156:6825/1227399112","cluster_addr":"172.21.15.156:6827/1227399112","heartbeat_back_addr":"172.21.15.156:6831/1227399112","heartbeat_front_addr":"172.21.15.156:6829/1227399112","state":["exists","up"]},{"osd":11,"uuid":"88d41d0c-a9d4-45ea-9a40-6612ee98597e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6825","nonce":826697758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6827","nonce":826697758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6831","nonce":826697758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":826697758},{"type":"v1","addr":"172.21.15.190:6829","nonce":826697758}]},"public_addr":"172.21.15.190:6825/826697758","cluster_addr":"172.21.15.190:6827/826697758","heartbeat_back_addr":"172.21.15.190:6831/826697758","heartbeat_front_addr":"172.21.15.190:6829/826697758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:07:39.380107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:00.503350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:22.422541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:08:50.345011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:11.108872+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:09:32.731700+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:00.211619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:21.218472+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:10:42.633096+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:10.164117+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:31.165619+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T22:11:53.335032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/339414861":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/117215786":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3599227077":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/4144294466":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6801/409950778":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/1942116136":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3830563323":"2023-10-07T22:06:05.527819+0000","172.21.15.57:6800/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6801/1962312800":"2023-10-07T22:05:38.356695+0000","172.21.15.57:6800/409950778":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3188291259":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/2575175349":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/3827189862":"2023-10-07T22:05:38.356695+0000","172.21.15.57:0/3940203258":"2023-10-07T22:14:45.661821+0000","172.21.15.57:0/2091122692":"2023-10-07T22:06:05.527819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f21080a1630 msgr2=0x7f21080a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f21080a1630 0x7f21080a3af0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f211c198590 tx=0x7f210c009260 comp rx=0 tx=0).stop 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 msgr2=0x7f211c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c196f80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2118005910 tx=0x7f2118004730 comp rx=0 tx=0).stop 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 shutdown_connections 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f21080a1630 0x7f21080a3af0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f211c19b920 0x7f211c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f211c1010b0 0x7f211c196f80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 --2- 172.21.15.57:0/2432855889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f211c1061d0 0x7f211c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 >> 172.21.15.57:0/2432855889 conn(0x7f211c0fc8d0 msgr2=0x7f211c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:46.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.005+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 shutdown_connections 2023-10-06T22:26:46.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.006+0000 7f21237ff700 1 -- 172.21.15.57:0/2432855889 wait complete. 2023-10-06T22:26:46.026 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T22:26:46.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.234+0000 7f06495ff700 1 -- 172.21.15.57:0/121852040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441061d0 msgr2=0x7f064410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.234+0000 7f06495ff700 1 --2- 172.21.15.57:0/121852040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441061d0 0x7f064410dd50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f063c009f90 tx=0x7f063c00b330 comp rx=0 tx=0).stop 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.234+0000 7f06495ff700 1 -- 172.21.15.57:0/121852040 shutdown_connections 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.234+0000 7f06495ff700 1 --2- 172.21.15.57:0/121852040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441061d0 0x7f064410dd50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.234+0000 7f06495ff700 1 --2- 172.21.15.57:0/121852040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06441010b0 0x7f0644101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.234+0000 7f06495ff700 1 --2- 172.21.15.57:0/121852040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0644101a60 0x7f0644105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.235+0000 7f06495ff700 1 -- 172.21.15.57:0/121852040 >> 172.21.15.57:0/121852040 conn(0x7f06440fc950 msgr2=0x7f06440fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.235+0000 7f06495ff700 1 -- 172.21.15.57:0/121852040 shutdown_connections 2023-10-06T22:26:46.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.235+0000 7f06495ff700 1 -- 172.21.15.57:0/121852040 wait complete. 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 Processor -- start 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 -- start start 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441010b0 0x7f0644199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06441061d0 0x7f06441997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f064419dc20 0x7f064419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419e6a0 con 0x7f06441010b0 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419e810 con 0x7f06441061d0 2023-10-06T22:26:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f06495ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419e9d0 con 0x7f064419dc20 2023-10-06T22:26:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f0642ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441010b0 0x7f0644199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f0642ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441010b0 0x7f0644199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34930/0 (socket says 172.21.15.57:34930) 2023-10-06T22:26:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.236+0000 7f0642ffd700 1 -- 172.21.15.57:0/405266146 learned_addr learned my addr 172.21.15.57:0/405266146 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f06427fc700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06441061d0 0x7f06441997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f06437fe700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f064419dc20 0x7f064419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f0642ffd700 1 -- 172.21.15.57:0/405266146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f064419dc20 msgr2=0x7f064419e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f0642ffd700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f064419dc20 0x7f064419e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f0642ffd700 1 -- 172.21.15.57:0/405266146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06441061d0 msgr2=0x7f06441997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f0642ffd700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06441061d0 0x7f06441997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f0642ffd700 1 -- 172.21.15.57:0/405266146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340095f0 con 0x7f06441010b0 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f06437fe700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f064419dc20 0x7f064419e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f0642ffd700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441010b0 0x7f0644199280 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f063400ec10 tx=0x7f063400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f062bfff700 1 -- 172.21.15.57:0/405266146 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400cc90 con 0x7f06441010b0 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f062bfff700 1 -- 172.21.15.57:0/405266146 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06340044e0 con 0x7f06441010b0 2023-10-06T22:26:46.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.237+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c009c40 con 0x7f06441010b0 2023-10-06T22:26:46.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.238+0000 7f062bfff700 1 -- 172.21.15.57:0/405266146 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634010430 con 0x7f06441010b0 2023-10-06T22:26:46.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.238+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441a6270 con 0x7f06441010b0 2023-10-06T22:26:46.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.239+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0644067c60 con 0x7f06441010b0 2023-10-06T22:26:46.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.241+0000 7f062bfff700 1 -- 172.21.15.57:0/405266146 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0634003f90 con 0x7f06441010b0 2023-10-06T22:26:46.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.242+0000 7f062bfff700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f062c0a1670 0x7f062c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.242+0000 7f062bfff700 1 -- 172.21.15.57:0/405266146 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8492+0+0 (secure 0 0 0) 0x7f0634014070 con 0x7f06441010b0 2023-10-06T22:26:46.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.242+0000 7f06427fc700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f062c0a1670 0x7f062c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.243+0000 7f06427fc700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f062c0a1670 0x7f062c0a3b30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f064419a890 tx=0x7f0638009260 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:46.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.244+0000 7f062bfff700 1 -- 172.21.15.57:0/405266146 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0634096a20 con 0x7f06441010b0 2023-10-06T22:26:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.505+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f064419a5d0 con 0x7f06441010b0 2023-10-06T22:26:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.508+0000 7f062bfff700 1 -- 172.21.15.57:0/405266146 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0634017e90 con 0x7f06441010b0 2023-10-06T22:26:46.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f062c0a1670 msgr2=0x7f062c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f062c0a1670 0x7f062c0a3b30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f064419a890 tx=0x7f0638009260 comp rx=0 tx=0).stop 2023-10-06T22:26:46.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441010b0 msgr2=0x7f0644199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441010b0 0x7f0644199280 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f063400ec10 tx=0x7f063400ef20 comp rx=0 tx=0).stop 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 shutdown_connections 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f062c0a1670 0x7f062c0a3b30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f064419dc20 0x7f064419e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06441010b0 0x7f0644199280 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 --2- 172.21.15.57:0/405266146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06441061d0 0x7f06441997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 >> 172.21.15.57:0/405266146 conn(0x7f06440fc950 msgr2=0x7f06440fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 shutdown_connections 2023-10-06T22:26:46.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.511+0000 7f06495ff700 1 -- 172.21.15.57:0/405266146 wait complete. 2023-10-06T22:26:46.514 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T22:26:46.543 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T22:26:46.604 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-g[150047]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.605 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-g[150047]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.605 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-mds[150071]: -- 172.21.15.57:0/3179959829 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5647fe67f1e0 con 0x5647fe654800 2023-10-06T22:26:46.605 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-g[150047]: starting mds.g at 2023-10-06T22:26:46.605 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/1871681487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:26:46.605 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/2432855889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:46.605 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-mon[98492]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T22:26:46.605 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a[98468]: 2023-10-06T22:26:46.506+0000 7f57d7a98700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T22:26:46.753 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/1871681487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:26:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/2432855889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:46.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-mon[134358]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T22:26:46.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[177384]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.754 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[177384]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.755 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-mds[177408]: -- 172.21.15.156:0/1485713967 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e2e2c831e0 con 0x55e2e2c58c00 2023-10-06T22:26:46.755 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[177384]: starting mds.e at 2023-10-06T22:26:46.755 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-b[171928]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.755 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-b[171928]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.755 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-mds[171952]: -- 172.21.15.156:0/1076289016 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558c53aeb1e0 con 0x558c53ac0c00 2023-10-06T22:26:46.755 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-b[171928]: starting mds.b at 2023-10-06T22:26:46.756 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.756 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.756 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-mds[171613]: -- 172.21.15.156:0/2946294221 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555831e791e0 con 0x555831e4ec00 2023-10-06T22:26:46.756 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: starting mds.h at 2023-10-06T22:26:46.756 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-k[170906]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.756 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-k[170906]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.756 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-mds[170931]: -- 172.21.15.156:0/1767821996 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b85fdeb1e0 con 0x55b85fdc0800 2023-10-06T22:26:46.757 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:26:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-k[170906]: starting mds.k at 2023-10-06T22:26:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/1006912349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101a60 msgr2=0x7f4e54105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/1006912349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101a60 0x7f4e54105aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e400098f0 tx=0x7f4e40009c00 comp rx=0 tx=0).stop 2023-10-06T22:26:46.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/1006912349 shutdown_connections 2023-10-06T22:26:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/1006912349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e541061d0 0x7f4e5410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/1006912349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e541010b0 0x7f4e54101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/1006912349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101a60 0x7f4e54105aa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/1006912349 >> 172.21.15.57:0/1006912349 conn(0x7f4e540fc970 msgr2=0x7f4e540fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/1006912349 shutdown_connections 2023-10-06T22:26:46.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.763+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/1006912349 wait complete. 2023-10-06T22:26:46.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.764+0000 7f4e5b1ff700 1 Processor -- start 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.764+0000 7f4e5b1ff700 1 -- start start 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.764+0000 7f4e5b1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e541010b0 0x7f4e54197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.764+0000 7f4e5b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e541061d0 0x7f4e541975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5b1ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e5419ba00 0x7f4e5419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5b1ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5419c480 con 0x7f4e5419ba00 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5419c5f0 con 0x7f4e541061d0 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5b1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5419c7b0 con 0x7f4e541010b0 2023-10-06T22:26:46.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e5419ba00 0x7f4e5419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e5419ba00 0x7f4e5419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34982/0 (socket says 172.21.15.57:34982) 2023-10-06T22:26:46.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 -- 172.21.15.57:0/3732464054 learned_addr learned my addr 172.21.15.57:0/3732464054 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:46.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e53fff700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e541061d0 0x7f4e541975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e58f9b700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e541010b0 0x7f4e54197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 -- 172.21.15.57:0/3732464054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e541010b0 msgr2=0x7f4e54197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e541010b0 0x7f4e54197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 -- 172.21.15.57:0/3732464054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e541061d0 msgr2=0x7f4e541975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:46.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e541061d0 0x7f4e541975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:46.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 -- 172.21.15.57:0/3732464054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480095f0 con 0x7f4e5419ba00 2023-10-06T22:26:46.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.765+0000 7f4e5979c700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e5419ba00 0x7f4e5419be80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00ef30 tx=0x7f4e4c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:46.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.766+0000 7f4e51ffb700 1 -- 172.21.15.57:0/3732464054 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c018070 con 0x7f4e5419ba00 2023-10-06T22:26:46.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.766+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e40009520 con 0x7f4e5419ba00 2023-10-06T22:26:46.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.766+0000 7f4e51ffb700 1 -- 172.21.15.57:0/3732464054 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c00f040 con 0x7f4e5419ba00 2023-10-06T22:26:46.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.766+0000 7f4e51ffb700 1 -- 172.21.15.57:0/3732464054 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00cd00 con 0x7f4e5419ba00 2023-10-06T22:26:46.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.766+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e54073430 con 0x7f4e5419ba00 2023-10-06T22:26:46.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.767+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e540680b0 con 0x7f4e5419ba00 2023-10-06T22:26:46.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.769+0000 7f4e51ffb700 1 -- 172.21.15.57:0/3732464054 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4e4c003a00 con 0x7f4e5419ba00 2023-10-06T22:26:46.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.770+0000 7f4e51ffb700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4e440a15f0 0x7f4e440a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:46.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.770+0000 7f4e51ffb700 1 -- 172.21.15.57:0/3732464054 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8878+0+0 (secure 0 0 0) 0x7f4e4c0cd710 con 0x7f4e5419ba00 2023-10-06T22:26:46.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.771+0000 7f4e58f9b700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4e440a15f0 0x7f4e440a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:46.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.771+0000 7f4e58f9b700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4e440a15f0 0x7f4e440a3ab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e48005fd0 tx=0x7f4e48009310 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:46.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:46.773+0000 7f4e51ffb700 1 -- 172.21.15.57:0/3732464054 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e4c096960 con 0x7f4e5419ba00 2023-10-06T22:26:46.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/1871681487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T22:26:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/2432855889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T22:26:46.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-mon[134891]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T22:26:46.820 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.820 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-mds[170574]: -- 172.21.15.190:0/3792403156 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a14c671e0 con 0x555a14c3c800 2023-10-06T22:26:46.821 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: starting mds.f at 2023-10-06T22:26:46.821 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.821 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.822 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: starting mds.i at 2023-10-06T22:26:46.822 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-l[170205]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.822 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-l[170205]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.822 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-mds[170229]: -- 172.21.15.190:0/2499228725 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5627ede351e0 con 0x5627ede0a800 2023-10-06T22:26:46.822 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-l[170205]: starting mds.l at 2023-10-06T22:26:46.823 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c[170889]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.823 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c[170889]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.823 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-mds[170913]: -- 172.21.15.190:0/3416126539 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55853a91b1e0 con 0x55853a8f0800 2023-10-06T22:26:46.823 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c[170889]: starting mds.c at 2023-10-06T22:26:46.823 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:26:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c[170889]: 2023-10-06T22:26:46.562+0000 7f610531a700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-06T22:26:46.923 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-a[150435]: ignoring --setuser ceph since I am not root 2023-10-06T22:26:46.923 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-a[150435]: ignoring --setgroup ceph since I am not root 2023-10-06T22:26:46.923 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-mds[150459]: -- 172.21.15.57:0/265776062 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5645b11c51e0 con 0x5645b119b000 2023-10-06T22:26:46.924 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:26:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-a[150435]: starting mds.a at 2023-10-06T22:26:47.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.041+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4e54101a60 con 0x7f4e5419ba00 2023-10-06T22:26:47.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.509+0000 7f4e51ffb700 1 -- 172.21.15.57:0/3732464054 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4e4c099f80 con 0x7f4e5419ba00 2023-10-06T22:26:47.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.513+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4e440a15f0 msgr2=0x7f4e440a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4e440a15f0 0x7f4e440a3ab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e48005fd0 tx=0x7f4e48009310 comp rx=0 tx=0).stop 2023-10-06T22:26:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e5419ba00 msgr2=0x7f4e5419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e5419ba00 0x7f4e5419be80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00ef30 tx=0x7f4e4c00c3a0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 shutdown_connections 2023-10-06T22:26:47.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f4e440a15f0 0x7f4e440a3ab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e5419ba00 0x7f4e5419be80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e541010b0 0x7f4e54197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 --2- 172.21.15.57:0/3732464054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e541061d0 0x7f4e541975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 >> 172.21.15.57:0/3732464054 conn(0x7f4e540fc970 msgr2=0x7f4e540fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 shutdown_connections 2023-10-06T22:26:47.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.514+0000 7f4e5b1ff700 1 -- 172.21.15.57:0/3732464054 wait complete. 2023-10-06T22:26:47.540 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: pgmap v377: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 12 op/s 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/405266146' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/405266146' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T22:26:47.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-06T22:26:47.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:47 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3732464054' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:26:47.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 -- 172.21.15.57:0/2971120139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c101a60 msgr2=0x7f9d6c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:47.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 --2- 172.21.15.57:0/2971120139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c101a60 0x7f9d6c105aa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009990 tx=0x7f9d5c009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 -- 172.21.15.57:0/2971120139 shutdown_connections 2023-10-06T22:26:47.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 --2- 172.21.15.57:0/2971120139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d6c1061d0 0x7f9d6c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 --2- 172.21.15.57:0/2971120139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 0x7f9d6c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 --2- 172.21.15.57:0/2971120139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c101a60 0x7f9d6c105aa0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 -- 172.21.15.57:0/2971120139 >> 172.21.15.57:0/2971120139 conn(0x7f9d6c0fc950 msgr2=0x7f9d6c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:47.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 -- 172.21.15.57:0/2971120139 shutdown_connections 2023-10-06T22:26:47.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.751+0000 7f9d743ff700 1 -- 172.21.15.57:0/2971120139 wait complete. 2023-10-06T22:26:47.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.752+0000 7f9d743ff700 1 Processor -- start 2023-10-06T22:26:47.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.752+0000 7f9d743ff700 1 -- start start 2023-10-06T22:26:47.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: pgmap v377: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 12 op/s 2023-10-06T22:26:47.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/405266146' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T22:26:47.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-06T22:26:47.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T22:26:47.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-10-06T22:26:47.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/405266146' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-06T22:26:47.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T22:26:47.757 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-06T22:26:47.758 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:47 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3732464054' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:26:47.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d743ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 0x7f9d6c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d743ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c1061d0 0x7f9d6c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d743ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d6c19db50 0x7f9d6c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d743ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e5d0 con 0x7f9d6c1061d0 2023-10-06T22:26:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d743ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e740 con 0x7f9d6c1010b0 2023-10-06T22:26:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d743ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e900 con 0x7f9d6c19db50 2023-10-06T22:26:47.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7199a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c1061d0 0x7f9d6c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 0x7f9d6c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 0x7f9d6c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46290/0 (socket says 172.21.15.57:46290) 2023-10-06T22:26:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 -- 172.21.15.57:0/34249984 learned_addr learned my addr 172.21.15.57:0/34249984 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7299c700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d6c19db50 0x7f9d6c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 -- 172.21.15.57:0/34249984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d6c19db50 msgr2=0x7f9d6c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d6c19db50 0x7f9d6c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 -- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c1061d0 msgr2=0x7f9d6c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c1061d0 0x7f9d6c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7219b700 1 -- 172.21.15.57:0/34249984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0095f0 con 0x7f9d6c1010b0 2023-10-06T22:26:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.753+0000 7f9d7199a700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c1061d0 0x7f9d6c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T22:26:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.754+0000 7f9d7219b700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 0x7f9d6c1991b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800e9a0 tx=0x7f9d6800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.754+0000 7f9d637fe700 1 -- 172.21.15.57:0/34249984 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68009790 con 0x7f9d6c1010b0 2023-10-06T22:26:47.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.754+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c1a5e60 con 0x7f9d6c1010b0 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.754+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c1a6290 con 0x7f9d6c1010b0 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.754+0000 7f9d637fe700 1 -- 172.21.15.57:0/34249984 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d68010430 con 0x7f9d6c1010b0 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.754+0000 7f9d637fe700 1 -- 172.21.15.57:0/34249984 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68018680 con 0x7f9d6c1010b0 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.755+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c067c60 con 0x7f9d6c1010b0 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.757+0000 7f9d637fe700 1 -- 172.21.15.57:0/34249984 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9d6800ca10 con 0x7f9d6c1010b0 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.759+0000 7f9d637fe700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9d580a1670 0x7f9d580a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.759+0000 7f9d637fe700 1 -- 172.21.15.57:0/34249984 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8878+0+0 (secure 0 0 0) 0x7f9d68014070 con 0x7f9d6c1010b0 2023-10-06T22:26:47.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.759+0000 7f9d7199a700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9d580a1670 0x7f9d580a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.759+0000 7f9d7199a700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9d580a1670 0x7f9d580a3b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b2e0 tx=0x7f9d5c005960 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:47.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.761+0000 7f9d637fe700 1 -- 172.21.15.57:0/34249984 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9d68052100 con 0x7f9d6c1010b0 2023-10-06T22:26:47.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: pgmap v377: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 12 op/s 2023-10-06T22:26:47.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/405266146' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T22:26:47.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-06T22:26:47.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T22:26:47.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-10-06T22:26:47.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/405266146' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-06T22:26:47.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T22:26:47.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-06T22:26:47.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:47 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3732464054' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T22:26:47.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:47.975+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9d6c19a500 con 0x7f9d6c1010b0 2023-10-06T22:26:48.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.523+0000 7f9d637fe700 1 -- 172.21.15.57:0/34249984 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9d68096470 con 0x7f9d6c1010b0 2023-10-06T22:26:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9d580a1670 msgr2=0x7f9d580a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9d580a1670 0x7f9d580a3b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b2e0 tx=0x7f9d5c005960 comp rx=0 tx=0).stop 2023-10-06T22:26:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 msgr2=0x7f9d6c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 0x7f9d6c1991b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800e9a0 tx=0x7f9d6800ecb0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 shutdown_connections 2023-10-06T22:26:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7f9d580a1670 0x7f9d580a3b30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d6c19db50 0x7f9d6c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c1010b0 0x7f9d6c1991b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.527+0000 7f9d743ff700 1 --2- 172.21.15.57:0/34249984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d6c1061d0 0x7f9d6c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.528+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 >> 172.21.15.57:0/34249984 conn(0x7f9d6c0fc950 msgr2=0x7f9d6c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.528+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 shutdown_connections 2023-10-06T22:26:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.528+0000 7f9d743ff700 1 -- 172.21.15.57:0/34249984 wait complete. 2023-10-06T22:26:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-10-06T22:26:48.556 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T22:26:48.618 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-06T22:26:48.618 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-06T22:26:48.618 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: Cluster is now healthy 2023-10-06T22:26:48.618 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3732464054' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6833/1822011455,v1:172.21.15.190:6835/1822011455] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6834/2509641191,v1:172.21.15.156:6835/2509641191] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6836/2299093511,v1:172.21.15.156:6837/2299093511] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6832/1778998841,v1:172.21.15.156:6833/1778998841] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6832/4069337912,v1:172.21.15.190:6834/4069337912] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.156:6838/2477341541,v1:172.21.15.156:6839/2477341541] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6837/2777992593,v1:172.21.15.190:6839/2777992593] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.190:6836/2754758863,v1:172.21.15.190:6838/2754758863] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.57:6838/1771089764,v1:172.21.15.57:6839/1771089764] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: mds.? [v2:172.21.15.57:6840/2135356882,v1:172.21.15.57:6841/2135356882] up:boot 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: fsmap 12 up:standby 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:26:48.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/34249984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:48.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:48 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:48.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-06T22:26:48.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-06T22:26:48.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: Cluster is now healthy 2023-10-06T22:26:48.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3732464054' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-06T22:26:48.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6833/1822011455,v1:172.21.15.190:6835/1822011455] up:boot 2023-10-06T22:26:48.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6834/2509641191,v1:172.21.15.156:6835/2509641191] up:boot 2023-10-06T22:26:48.754 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6836/2299093511,v1:172.21.15.156:6837/2299093511] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6832/1778998841,v1:172.21.15.156:6833/1778998841] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6832/4069337912,v1:172.21.15.190:6834/4069337912] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.156:6838/2477341541,v1:172.21.15.156:6839/2477341541] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6837/2777992593,v1:172.21.15.190:6839/2777992593] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.190:6836/2754758863,v1:172.21.15.190:6838/2754758863] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.57:6838/1771089764,v1:172.21.15.57:6839/1771089764] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: mds.? [v2:172.21.15.57:6840/2135356882,v1:172.21.15.57:6841/2135356882] up:boot 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: fsmap 12 up:standby 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:26:48.755 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:26:48.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:26:48.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:26:48.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:26:48.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:26:48.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:26:48.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/34249984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:48.756 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:48 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.763+0000 7ff0b65ff700 1 -- 172.21.15.57:0/2055568747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073280 msgr2=0x7ff0b0073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.763+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/2055568747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073280 0x7ff0b0073660 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009990 tx=0x7ff0a0009ca0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.763+0000 7ff0b65ff700 1 -- 172.21.15.57:0/2055568747 shutdown_connections 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.763+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/2055568747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0b010d710 0x7ff0b010fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.763+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/2055568747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073280 0x7ff0b0073660 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.763+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/2055568747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b0073ba0 0x7ff0b010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.763+0000 7ff0b65ff700 1 -- 172.21.15.57:0/2055568747 >> 172.21.15.57:0/2055568747 conn(0x7ff0b00fc960 msgr2=0x7ff0b00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:48.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.764+0000 7ff0b65ff700 1 -- 172.21.15.57:0/2055568747 shutdown_connections 2023-10-06T22:26:48.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.764+0000 7ff0b65ff700 1 -- 172.21.15.57:0/2055568747 wait complete. 2023-10-06T22:26:48.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 Processor -- start 2023-10-06T22:26:48.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 -- start start 2023-10-06T22:26:48.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b0073280 0x7ff0b01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073ba0 0x7ff0b0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0b010d710 0x7ff0b019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b019e2b0 con 0x7ff0b010d710 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b019e420 con 0x7ff0b0073ba0 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b65ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b019e5e0 con 0x7ff0b0073280 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0b4b9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0b010d710 0x7ff0b019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073ba0 0x7ff0b0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0affff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b0073280 0x7ff0b01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073ba0 0x7ff0b0199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.57:46306/0 (socket says 172.21.15.57:46306) 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.765+0000 7ff0af7fe700 1 -- 172.21.15.57:0/3849367094 learned_addr learned my addr 172.21.15.57:0/3849367094 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0af7fe700 1 -- 172.21.15.57:0/3849367094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b0073280 msgr2=0x7ff0b01992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:48.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0af7fe700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b0073280 0x7ff0b01992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0af7fe700 1 -- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0b010d710 msgr2=0x7ff0b019dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0af7fe700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0b010d710 0x7ff0b019dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0af7fe700 1 -- 172.21.15.57:0/3849367094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8009c40 con 0x7ff0b0073ba0 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0af7fe700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073ba0 0x7ff0b0199810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9a0 tx=0x7ff0a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0ad7fa700 1 -- 172.21.15.57:0/3849367094 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4009790 con 0x7ff0b0073ba0 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0ad7fa700 1 -- 172.21.15.57:0/3849367094 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a400ce80 con 0x7ff0b0073ba0 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0ad7fa700 1 -- 172.21.15.57:0/3849367094 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4018680 con 0x7ff0b0073ba0 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.766+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a00095f0 con 0x7ff0b0073ba0 2023-10-06T22:26:48.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.767+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b01a5f20 con 0x7ff0b0073ba0 2023-10-06T22:26:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.767+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b0067c60 con 0x7ff0b0073ba0 2023-10-06T22:26:48.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.770+0000 7ff0ad7fa700 1 -- 172.21.15.57:0/3849367094 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 27) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff0a4010430 con 0x7ff0b0073ba0 2023-10-06T22:26:48.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.771+0000 7ff0ad7fa700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff0980a1670 0x7ff0980a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T22:26:48.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.771+0000 7ff0ad7fa700 1 -- 172.21.15.57:0/3849367094 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8345+0+0 (secure 0 0 0) 0x7ff0a4014070 con 0x7ff0b0073ba0 2023-10-06T22:26:48.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.771+0000 7ff0affff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff0980a1670 0x7ff0980a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T22:26:48.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.772+0000 7ff0affff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff0980a1670 0x7ff0980a3b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b2e0 tx=0x7ff0a0005fb0 comp rx=0 tx=0).ready entity=mgr.25414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T22:26:48.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.773+0000 7ff0ad7fa700 1 -- 172.21.15.57:0/3849367094 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff0a4096900 con 0x7ff0b0073ba0 2023-10-06T22:26:48.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-06T22:26:48.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-06T22:26:48.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: Cluster is now healthy 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3732464054' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6833/1822011455,v1:172.21.15.190:6835/1822011455] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6834/2509641191,v1:172.21.15.156:6835/2509641191] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6836/2299093511,v1:172.21.15.156:6837/2299093511] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6832/1778998841,v1:172.21.15.156:6833/1778998841] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6832/4069337912,v1:172.21.15.190:6834/4069337912] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.156:6838/2477341541,v1:172.21.15.156:6839/2477341541] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6837/2777992593,v1:172.21.15.190:6839/2777992593] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.190:6836/2754758863,v1:172.21.15.190:6838/2754758863] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.57:6838/1771089764,v1:172.21.15.57:6839/1771089764] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: mds.? [v2:172.21.15.57:6840/2135356882,v1:172.21.15.57:6841/2135356882] up:boot 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: fsmap 12 up:standby 2023-10-06T22:26:48.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='mgr.25414 172.21.15.57:0/1325871641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/34249984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:48.821 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:48 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:48.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:48.990+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0b019a620 con 0x7ff0b0073ba0 2023-10-06T22:26:49.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.535+0000 7ff0ad7fa700 1 -- 172.21.15.57:0/3849367094 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff0a4099f20 con 0x7ff0b0073ba0 2023-10-06T22:26:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff0980a1670 msgr2=0x7ff0980a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff0980a1670 0x7ff0980a3b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b2e0 tx=0x7ff0a0005fb0 comp rx=0 tx=0).stop 2023-10-06T22:26:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073ba0 msgr2=0x7ff0b0199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T22:26:49.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073ba0 0x7ff0b0199810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9a0 tx=0x7ff0a400ecb0 comp rx=0 tx=0).stop 2023-10-06T22:26:49.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 shutdown_connections 2023-10-06T22:26:49.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:6800/3210017833,v1:172.21.15.57:6801/3210017833] conn(0x7ff0980a1670 0x7ff0980a3b30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:49.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0b010d710 0x7ff0b019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:49.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b0073280 0x7ff0b01992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:49.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 --2- 172.21.15.57:0/3849367094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b0073ba0 0x7ff0b0199810 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T22:26:49.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.540+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 >> 172.21.15.57:0/3849367094 conn(0x7ff0b00fc960 msgr2=0x7ff0b010f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T22:26:49.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.541+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 shutdown_connections 2023-10-06T22:26:49.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-06T22:26:49.541+0000 7ff0b65ff700 1 -- 172.21.15.57:0/3849367094 wait complete. 2023-10-06T22:26:49.554 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-10-06T22:26:49.577 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:26:49.690 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:26:49.730 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:26:49.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:49 smithi190 ceph-mon[134891]: pgmap v379: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-10-06T22:26:49.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:49 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-06T22:26:49.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:49 smithi190 ceph-mon[134891]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T22:26:49.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:49 smithi190 ceph-mon[134891]: from='client.? 172.21.15.57:0/3849367094' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:49.816 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:49 smithi190 ceph-mon[134891]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:49.817 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-06T22:26:49.817 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T22:26:49.862 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T22:26:49.879 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T22:26:49.896 INFO:tasks.cephadm:Stopping all daemons... 2023-10-06T22:26:49.896 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-06T22:26:49.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a 2023-10-06T22:26:49.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:49 smithi057 ceph-mon[98492]: pgmap v379: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-10-06T22:26:49.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:49 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-06T22:26:49.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:49 smithi057 ceph-mon[98492]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T22:26:49.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:49 smithi057 ceph-mon[98492]: from='client.? 172.21.15.57:0/3849367094' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:49.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:49 smithi057 ceph-mon[98492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:50.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:49 smithi156 ceph-mon[134358]: pgmap v379: 97 pgs: 97 active+clean; 1.8 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-10-06T22:26:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:49 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-06T22:26:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:49 smithi156 ceph-mon[134358]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T22:26:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:49 smithi156 ceph-mon[134358]: from='client.? 172.21.15.57:0/3849367094' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:50.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:49 smithi156 ceph-mon[134358]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T22:26:50.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:49 smithi057 systemd[1]: Stopping Ceph mon.a for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:26:50.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a[98468]: 2023-10-06T22:26:50.423+0000 7f57dd2a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:26:50.923 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a[98468]: 2023-10-06T22:26:50.423+0000 7f57dd2a3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-06T22:26:51.419 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 06 22:26:50 smithi057 bash[180948]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-a 2023-10-06T22:26:51.453 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-06T22:26:51.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:26:51.606 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-06T22:26:51.606 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-06T22:26:51.607 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.b 2023-10-06T22:26:51.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:51 smithi156 systemd[1]: Stopping Ceph mon.b for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:26:51.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-b[134334]: 2023-10-06T22:26:51.984+0000 7fa9548a8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:26:51.988 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-b[134334]: 2023-10-06T22:26:51.984+0000 7fa9548a8700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-06T22:26:52.522 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-06T22:26:52.548 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:52 smithi156 bash[178534]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-b 2023-10-06T22:26:52.548 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:52 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.b.service: Succeeded. 2023-10-06T22:26:52.548 INFO:journalctl@ceph.mon.b.smithi156.stdout:Oct 06 22:26:52 smithi156 systemd[1]: Stopped Ceph mon.b for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:26:52.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:26:52.575 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-06T22:26:52.576 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-06T22:26:52.576 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.c 2023-10-06T22:26:52.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:52 smithi190 systemd[1]: Stopping Ceph mon.c for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:26:53.286 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-c[134864]: 2023-10-06T22:26:52.992+0000 7f60c0951700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:26:53.286 INFO:journalctl@ceph.mon.c.smithi190.stdout:Oct 06 22:26:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mon-c[134864]: 2023-10-06T22:26:52.992+0000 7f60c0951700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-06T22:26:53.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-06T22:26:53.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:26:53.813 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-06T22:26:53.813 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-06T22:26:53.813 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.x 2023-10-06T22:26:54.173 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:26:53 smithi057 systemd[1]: Stopping Ceph mgr.x for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:26:54.572 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:26:54 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:26:54.313+0000 7fddfde9e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:26:54.573 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:26:54 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x[98738]: 2023-10-06T22:26:54.313+0000 7fddfde9e700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-06T22:26:54.922 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 06 22:26:54 smithi057 bash[181097]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-x 2023-10-06T22:26:55.048 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-06T22:26:55.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:26:55.143 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-06T22:26:55.143 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-06T22:26:55.143 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.y 2023-10-06T22:26:55.503 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:26:55 smithi156 systemd[1]: Stopping Ceph mgr.y for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:26:55.846 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Oct 06 22:26:55 smithi156 bash[178674]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-y 2023-10-06T22:26:56.029 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-06T22:26:56.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:26:56.066 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-06T22:26:56.066 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-06T22:26:56.066 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.z 2023-10-06T22:26:56.522 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:26:56 smithi190 systemd[1]: Stopping Ceph mgr.z for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:26:57.050 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:26:56 smithi190 bash[177398]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mgr-z 2023-10-06T22:26:57.050 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:26:56 smithi190 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-06T22:26:57.054 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-10-06T22:26:57.069 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:26:57 smithi190 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mgr.z.service: Failed with result 'exit-code'. 2023-10-06T22:26:57.069 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Oct 06 22:26:57 smithi190 systemd[1]: Stopped Ceph mgr.z for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:26:57.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:26:57.091 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-06T22:26:57.091 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-06T22:26:57.091 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.0 2023-10-06T22:26:57.423 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:26:57 smithi057 systemd[1]: Stopping Ceph osd.0 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:26:58.173 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:26:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-0[116815]: 2023-10-06T22:26:57.680+0000 7f1845bfd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:26:58.173 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:26:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-0[116815]: 2023-10-06T22:26:57.680+0000 7f1845bfd700 -1 osd.0 103 *** Got signal Terminated *** 2023-10-06T22:26:58.173 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:26:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-0[116815]: 2023-10-06T22:26:57.680+0000 7f1845bfd700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:03.923 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 06 22:27:03 smithi057 bash[181243]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-0 2023-10-06T22:27:05.423 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-06T22:27:05.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:27:05.515 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-06T22:27:05.515 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-06T22:27:05.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.1 2023-10-06T22:27:05.868 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:27:05 smithi156 systemd[1]: Stopping Ceph osd.1 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:27:06.253 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:27:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-1[139452]: 2023-10-06T22:27:05.865+0000 7fb300205700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:27:06.253 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:27:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-1[139452]: 2023-10-06T22:27:05.865+0000 7fb300205700 -1 osd.1 103 *** Got signal Terminated *** 2023-10-06T22:27:06.253 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:27:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-1[139452]: 2023-10-06T22:27:05.865+0000 7fb300205700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:11.654 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:27:11 smithi156 bash[178819]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-1 2023-10-06T22:27:12.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-06T22:27:12.860 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:27:12 smithi156 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.1.service: Succeeded. 2023-10-06T22:27:12.860 INFO:journalctl@ceph.osd.1.smithi156.stdout:Oct 06 22:27:12 smithi156 systemd[1]: Stopped Ceph osd.1 for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:27:12.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:27:12.877 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-06T22:27:12.877 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-06T22:27:12.877 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.2 2023-10-06T22:27:13.275 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:27:12 smithi190 systemd[1]: Stopping Ceph osd.2 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:27:13.569 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:27:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-2[140652]: 2023-10-06T22:27:13.272+0000 7f58690bd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:27:13.569 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:27:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-2[140652]: 2023-10-06T22:27:13.272+0000 7f58690bd700 -1 osd.2 103 *** Got signal Terminated *** 2023-10-06T22:27:13.570 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:27:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-2[140652]: 2023-10-06T22:27:13.272+0000 7f58690bd700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:19.319 INFO:journalctl@ceph.osd.2.smithi190.stdout:Oct 06 22:27:19 smithi190 bash[177546]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-2 2023-10-06T22:27:20.563 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-06T22:27:20.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:27:20.603 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-06T22:27:20.603 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-06T22:27:20.603 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.3 2023-10-06T22:27:20.923 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:27:20 smithi057 systemd[1]: Stopping Ceph osd.3 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:27:21.673 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:27:21 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-3[125351]: 2023-10-06T22:27:21.226+0000 7f8351392700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:27:21.673 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:27:21 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-3[125351]: 2023-10-06T22:27:21.226+0000 7f8351392700 -1 osd.3 103 *** Got signal Terminated *** 2023-10-06T22:27:21.673 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:27:21 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-3[125351]: 2023-10-06T22:27:21.226+0000 7f8351392700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:24.423 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:23 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:23.952+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:24.819 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:24.348+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:25.423 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:24 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:24.932+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:25.819 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:25.380+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:25.819 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:25.550+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:26.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:25 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:25.892+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:26.819 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:26.514+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:26.819 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:26.344+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:27.159 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:26 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:26.915+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:27.159 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:26 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:26.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:27.423 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 06 22:27:27 smithi057 bash[181525]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-3 2023-10-06T22:27:27.819 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:27.547+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:27.819 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:27.349+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:28.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:27 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:27.944+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:28.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:27 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:27.756+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:28.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:27 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:27.783+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:28.569 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:28.516+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:28.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:28.301+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:29.034 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:28 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:28.920+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:29.040 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:28 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:28.775+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:29.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:28 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:28.767+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:29.431 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-06T22:27:29.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:27:29.522 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-06T22:27:29.522 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-06T22:27:29.522 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.4 2023-10-06T22:27:29.569 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:29.480+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:29.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:29.963 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:29 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:29.777+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:29.964 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:27:29 smithi156 systemd[1]: Stopping Ceph osd.4 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:27:30.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:29 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:29.925+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:30.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:29 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:29.739+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:30.253 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:27:29 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:27:29.961+0000 7ff397cf0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:27:30.253 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:27:29 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:27:29.961+0000 7ff397cf0700 -1 osd.4 103 *** Got signal Terminated *** 2023-10-06T22:27:30.254 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:27:29 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:27:29.961+0000 7ff397cf0700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:30.569 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:30 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:30.524+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:30.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:30 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:30.291+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:31.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:30 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:30.932+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:31.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:30 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:30.769+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:31.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:30 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:30.760+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:31.568 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:31 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:31.284+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:31.819 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:31 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:31.566+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:32.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:31 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:31.929+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:32.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:31 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:31.789+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:32.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:31 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:31.770+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:32.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:32 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:32.249+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:33.003 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:32 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:32.730+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:33.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:32 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:32.592+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:33.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:32 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:32.936+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:33.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:32 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:32.813+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:33.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:33 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:33.228+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:33.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:33 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:33.228+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:33.767 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:27:33 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:27:33.504+0000 7ff3942e6700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.811360+0000 front 2023-10-06T22:27:10.811386+0000 (oldest deadline 2023-10-06T22:27:32.511512+0000) 2023-10-06T22:27:34.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:33 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:33.600+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:34.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:33 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:33.953+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:34.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:33 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:33.953+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:34.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:33 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:33.791+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:34.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:33 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:33.765+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:34.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:33 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:33.765+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:34.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:34 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:34.250+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:34.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:34 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:34.250+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:34.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:34 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:34.375+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:34.793 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:27:34 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4[148017]: 2023-10-06T22:27:34.516+0000 7ff3942e6700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.811360+0000 front 2023-10-06T22:27:10.811386+0000 (oldest deadline 2023-10-06T22:27:32.511512+0000) 2023-10-06T22:27:35.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:34 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:34.644+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:35.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:34 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:34.644+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:35.099 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:34 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:34.791+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:35.099 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:34 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:34.791+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:35.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:34 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:34.952+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:35.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:34 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:34.952+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:35.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:34 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:34.797+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:35.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:35 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:35.097+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:35.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:35 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:35.402+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:35.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:35 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:35.267+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:35.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:35 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:35.267+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:35.818 INFO:journalctl@ceph.osd.4.smithi156.stdout:Oct 06 22:27:35 smithi156 bash[179088]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-4 2023-10-06T22:27:36.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:35 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:35.600+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:36.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:35 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:35.600+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:36.079 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:35 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:35.816+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:36.079 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:35 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:35.816+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:36.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:35 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:35.783+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:36.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:35 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:35.964+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:36.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:35 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:35.964+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:36.504 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:36 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:36.077+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:36.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:36.227+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:36.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:36.227+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:36.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:36.371+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:36.816 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:36 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:36.798+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:36.816 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:36 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:36.798+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:36.819 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-06T22:27:36.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:27:36.860 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-06T22:27:36.860 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-06T22:27:36.860 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.5 2023-10-06T22:27:36.888 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:36.621+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:36.888 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:36 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:36.621+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:37.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:36.972+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:37.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:36.972+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:37.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:36.821+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:37.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:36 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:36.821+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:37.188 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:36 smithi190 systemd[1]: Stopping Ceph osd.5 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:27:37.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:37.102+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:37.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:37.375+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:37.570 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:37.227+0000 7fd736b8d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:27:37.570 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:37.227+0000 7fd736b8d700 -1 osd.5 103 *** Got signal Terminated *** 2023-10-06T22:27:37.570 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:37.227+0000 7fd736b8d700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:37.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:37.186+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:37.571 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:37.186+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:38.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:37.623+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:38.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:37 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:37.623+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:38.127 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:37.812+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:38.127 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:37 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:37.812+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:38.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:37.953+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:38.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:37.953+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:38.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:37.805+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:38.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:37 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:37.805+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:38.422 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:38 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:38.137+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:38.422 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:38 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:38.137+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:38.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:38 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:38.124+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:38.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:38 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:38.420+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:38.819 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:38 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:38.659+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:38.819 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:38 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:38.659+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:39.151 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:38 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:38.837+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:39.151 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:38 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:38.837+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:39.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:38.961+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:39.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:38.961+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:39.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:38.788+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:39.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:38 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:38.788+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:39.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:39 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:39.149+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:39.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:39.370+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:39.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:39.141+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:39.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:39.141+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:40.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:39.649+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:40.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:39.649+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:40.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:39 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:39.649+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:13.728616+0000 front 2023-10-06T22:27:13.728635+0000 (oldest deadline 2023-10-06T22:27:39.628785+0000) 2023-10-06T22:27:40.142 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:39 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:39.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:40.142 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:39 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:39.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:40.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:39 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:39.819+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:40.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:39 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:39.819+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:40.174 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:40 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:40.001+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:40.174 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:40 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:40.001+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:40.395 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:40.100+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:40.395 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:40.100+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:40.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:40.139+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:40.691 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:40.393+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:41.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:40.689+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:41.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:40.689+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:41.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:40 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:40.689+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:13.728616+0000 front 2023-10-06T22:27:13.728635+0000 (oldest deadline 2023-10-06T22:27:39.628785+0000) 2023-10-06T22:27:41.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:41.000+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:41.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:41.000+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:41.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:41.000+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:41.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:40 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:40.777+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:41.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:40 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:40.777+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:41.189 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:40.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:41.189 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:40 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:40.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:41.414 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:41.074+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:41.414 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:41.074+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:41.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:41 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:41.188+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:41.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:41 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:41.188+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:41.723 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:41.412+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:41.723 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:41.412+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:42.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:41.721+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:00.327016+0000 front 2023-10-06T22:27:00.327115+0000 (oldest deadline 2023-10-06T22:27:25.026829+0000) 2023-10-06T22:27:42.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:41.721+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:10.828576+0000 front 2023-10-06T22:27:10.828364+0000 (oldest deadline 2023-10-06T22:27:33.728290+0000) 2023-10-06T22:27:42.069 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:41 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5[148583]: 2023-10-06T22:27:41.721+0000 7fd73117f700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:13.728616+0000 front 2023-10-06T22:27:13.728635+0000 (oldest deadline 2023-10-06T22:27:39.628785+0000) 2023-10-06T22:27:42.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:41.954+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:42.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:41.954+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:42.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:41.954+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:42.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:41.758+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:42.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:41 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:41.758+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:42.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:41 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:41.871+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:42.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:41 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:41.871+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:42.408 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:42.119+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:42.409 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:42.119+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:42.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:42.231+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:42.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:42.231+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:42.823 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:42.406+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:42.824 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:42 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:42.406+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:43.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:42.932+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:43.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:42.932+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:43.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:42.932+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:43.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:42.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:43.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:42.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:43.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:42 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:42.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:43.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:42.834+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:43.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:42 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:42.834+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:43.319 INFO:journalctl@ceph.osd.5.smithi190.stdout:Oct 06 22:27:43 smithi190 bash[177808]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-5 2023-10-06T22:27:43.319 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:43 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:43.120+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:43.320 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:43 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:43.120+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:43.320 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:43 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:43.120+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:43.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:43 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:43.261+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:43.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:43 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:43.261+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:43.805 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:43 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:43.396+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:43.805 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:43 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:43.396+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:44.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:43.941+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:44.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:43.941+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:44.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:43.941+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:44.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:43.788+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:44.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:43.788+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:44.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:43.788+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:44.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:43 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:43.820+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:44.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:43 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:43.820+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:44.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:43 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:43.820+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:44.395 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:44 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:44.107+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:44.395 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:44 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:44.107+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:44.396 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:44 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:44.107+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:44.558 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-06T22:27:44.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:27:44.639 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-06T22:27:44.639 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-06T22:27:44.639 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.6 2023-10-06T22:27:44.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:44 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:44.256+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:44.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:44 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:44.256+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:44.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:44 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:44.393+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:44.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:44 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:44.393+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:45.142 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:44.778+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:45.142 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:44.778+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:45.143 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:44.778+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:45.143 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:44 smithi057 systemd[1]: Stopping Ceph osd.6 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:27:45.143 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:44.933+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:45.143 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:44.933+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:45.144 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:44 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:44.933+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:45.214 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:44 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:44.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:45.214 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:44 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:44.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:45.214 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:44 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:44.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:45.423 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:45.140+0000 7fac1f7a7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:27:45.423 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:45.140+0000 7fac1f7a7700 -1 osd.6 103 *** Got signal Terminated *** 2023-10-06T22:27:45.423 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:45.140+0000 7fac1f7a7700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:45.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:45 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:45.211+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:45.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:45 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:45.211+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:45.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:45.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:45.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:45.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:45.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:45.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:45.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:45.350+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:45.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:45 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:45.350+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:46.168 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:45 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:45.874+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:46.168 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:45 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:45.874+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:46.169 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:45 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:45.874+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:46.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:45.911+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:46.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:45.911+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:46.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:45.911+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:46.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:45.753+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:46.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:45.753+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:46.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:45 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:45.753+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:46.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:46.166+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:46.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:46.166+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:46.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:46.316+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:46.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:46.316+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:46.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:46.124+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:46.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:46.124+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:46.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:46 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:46.124+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:47.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:46.870+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:47.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:46.870+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:47.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:46.870+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:47.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:46.727+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:47.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:46.727+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:47.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:46 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:46.727+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:47.215 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:46.862+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:47.215 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:46.862+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:47.215 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:46 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:46.862+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:47.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:47 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:47.214+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:47.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:47 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:47.214+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:47.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:47 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:47.321+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:47.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:47 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:47.322+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:47.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:47 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:47.162+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:47.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:47 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:47.162+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:47.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:47 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:47.162+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:48.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:47 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:47.896+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:48.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:47 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:47.896+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:48.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:47 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:47.896+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:48.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:47 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:47.697+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:48.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:47 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:47.697+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:48.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:47 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:47.697+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:48.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:47 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:47.697+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:48.231 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:47 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:47.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:48.231 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:47 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:47.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:48.231 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:47 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:47.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:48.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:48.228+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:48.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:48.228+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:48.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:48 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:48.288+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:48.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:48 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:48.288+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:48.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:48 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:48.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:48.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:48 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:48.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:48.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:48 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:48.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:49.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:48 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:48.867+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:49.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:48 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:48.867+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:49.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:48 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:48.867+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:49.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:48 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:48.729+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:49.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:48 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:48.729+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:49.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:48 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:48.729+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:49.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:48 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:48.729+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:49.212 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:48.868+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:49.212 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:48.868+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:49.213 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:48.868+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:49.213 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:48.868+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:49.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:49 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:49.211+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:49.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:49 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:49.211+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:49.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:49 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:49.152+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:49.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:49 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:49.152+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:49.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:49 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:49.152+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:49.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:49 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:49.297+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:49.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:49 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:49.297+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:50.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:49.859+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:58.795483+0000 front 2023-10-06T22:26:58.795460+0000 (oldest deadline 2023-10-06T22:27:23.495658+0000) 2023-10-06T22:27:50.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:49.859+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:09.198433+0000 front 2023-10-06T22:27:09.198411+0000 (oldest deadline 2023-10-06T22:27:33.799105+0000) 2023-10-06T22:27:50.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:49.859+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.300712+0000 front 2023-10-06T22:27:17.300501+0000 (oldest deadline 2023-10-06T22:27:40.200609+0000) 2023-10-06T22:27:50.173 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6[133205]: 2023-10-06T22:27:49.859+0000 7fac19d99700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.402269+0000 front 2023-10-06T22:27:25.402300+0000 (oldest deadline 2023-10-06T22:27:48.902325+0000) 2023-10-06T22:27:50.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:49.728+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:50.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:49.728+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:50.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:49.728+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:50.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:49 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:49.728+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:50.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:49 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:49.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:50.234 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:49 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:49.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:50.235 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:49 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:49.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:50.235 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:49 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:49.861+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:50.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:50 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:50.233+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:50.504 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:50 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:50.233+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:50.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:50 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:50.304+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:50.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:50 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:50.304+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:50.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:50 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:50.121+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:50.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:50 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:50.121+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:50.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:50 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:50.121+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:50.923 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 06 22:27:50 smithi057 bash[181807]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-6 2023-10-06T22:27:50.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:50.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:50.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:50.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:50.924 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:50.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:50.924 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:50 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:50.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:51.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:50 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:50.828+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:51.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:50 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:50.828+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:51.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:50 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:50.828+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:51.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:50 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:50.828+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:51.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:51 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:51.135+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:51.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:51 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:51.135+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:51.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:51 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:51.135+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:51.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:51 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:51.305+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:51.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:51 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:51.306+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:51.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:51 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:51.306+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:51.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:51.271+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:51.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:51.271+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:51.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:51.271+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:51.922 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:51 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:51.664+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:51.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:51 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:51.664+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:51.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:51 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:51.664+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:51.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:51 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:51.664+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:52.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:51.860+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:52.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:51.860+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:52.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:51.860+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:52.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:51.860+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:52.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:52.135+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:52.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:52.135+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:52.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:52.135+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:52.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:52.302+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:52.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:52.302+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:52.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:52 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:52.302+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:52.754 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:52 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:52.309+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:52.754 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:52 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:52.309+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:52.754 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:52 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:52.309+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:52.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:52 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:52.649+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:52.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:52 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:52.649+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:52.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:52 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:52.649+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:52.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:52 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:52.649+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:52.995 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-06T22:27:53.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:27:53.086 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-06T22:27:53.086 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-06T22:27:53.087 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.7 2023-10-06T22:27:53.117 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:52 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:52.850+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:53.117 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:52 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:52.850+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:53.117 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:52 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:52.850+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:53.117 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:52 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:52.850+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:53.443 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:53 smithi156 systemd[1]: Stopping Ceph osd.7 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:27:53.443 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:53.311+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:53.443 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:53.311+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:53.443 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:53.311+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:53.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:53 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:53.086+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:53.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:53 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:53.086+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:53.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:53 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:53.086+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:53.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:53 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:53.309+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:53.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:53 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:53.309+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:53.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:53 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:53.309+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:53.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:53.441+0000 7fdae5488700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:27:53.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:53.441+0000 7fdae5488700 -1 osd.7 103 *** Got signal Terminated *** 2023-10-06T22:27:53.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:53.441+0000 7fdae5488700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:27:53.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:53 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:53.629+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:53.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:53 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:53.629+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:53.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:53 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:53.629+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:53.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:53 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:53.629+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:54.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:53.853+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:54.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:53.853+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:54.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:53.853+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:54.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:53 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:53.853+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:54.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:54.127+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:54.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:54.127+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:54.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:54.127+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:54.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:54.305+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:54.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:54.305+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:54.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:54 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:54.305+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:54.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:54 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:54.265+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:54.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:54 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:54.265+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:54.753 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:54 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:54.265+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:55.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:54 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:54.671+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:55.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:54 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:54.671+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:55.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:54 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:54.671+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:55.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:54 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:54.671+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:55.221 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:54 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:54.822+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:55.221 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:54 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:54.822+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:55.221 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:54 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:54.822+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:55.221 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:54 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:54.822+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:55.505 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:55.219+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:55.505 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:55.219+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:55.506 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:55.219+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:55.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:55 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:55.262+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:55.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:55 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:55.262+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:55.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:55 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:55.262+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:55.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:55 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:55.138+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:55.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:55 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:55.138+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:55.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:55 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:55.138+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:55.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:55 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:55.655+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:55.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:55 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:55.655+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:55.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:55 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:55.655+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:55.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:55 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:55.655+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:56.208 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:55.779+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:56.208 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:55.779+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:56.208 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:55.779+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:56.208 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:55.779+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:56.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:56 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:56.206+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:56.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:56 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:56.206+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:56.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:56 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:56.206+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:56.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:56 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:56.145+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:56.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:56 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:56.145+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:56.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:56 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:56.145+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:56.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:56 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:56.284+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:56.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:56 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:56.284+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:56.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:56 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:56.284+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:56.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:56.641+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:56.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:56.641+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:56.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:56.641+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:56.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:56 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:56.641+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:57.212 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:56 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:56.824+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:57.213 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:56 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:56.824+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:57.213 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:56 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:56.824+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:57.213 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:56 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:56.824+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:57.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:57 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:57.211+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:57.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:57 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:57.211+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:57.504 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:57 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:57.211+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:57.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:57 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:57.260+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:57.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:57 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:57.260+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:57.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:57 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:57.260+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:57.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:57 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:57.136+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:57.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:57 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:57.136+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:57.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:57 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:57.136+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:57.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:57 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:57.136+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:27:57.923 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:57.668+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:57.924 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:57.668+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:57.924 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:57.668+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:57.924 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:57 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:57.668+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:58.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:57 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:57.862+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:58.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:57 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:57.862+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:58.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:57 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:57.862+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:58.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:57 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:57.862+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:58.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:58 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:58.270+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:58.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:58 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:58.270+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:58.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:58 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:58.270+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:58.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:58 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:58.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:58.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:58 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:58.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:58.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:58 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:58.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:58.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:58 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:58.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:27:58.754 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:58.260+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.572637+0000 front 2023-10-06T22:27:08.572665+0000 (oldest deadline 2023-10-06T22:27:34.472824+0000) 2023-10-06T22:27:58.754 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:58.260+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:14.974008+0000 front 2023-10-06T22:27:14.973988+0000 (oldest deadline 2023-10-06T22:27:40.274141+0000) 2023-10-06T22:27:58.754 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:58.260+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.376725+0000 front 2023-10-06T22:27:25.376609+0000 (oldest deadline 2023-10-06T22:27:50.676498+0000) 2023-10-06T22:27:58.754 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7[156743]: 2023-10-06T22:27:58.260+0000 7fdadfa7a700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:34.678119+0000 front 2023-10-06T22:27:34.678146+0000 (oldest deadline 2023-10-06T22:27:57.578430+0000) 2023-10-06T22:27:59.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:58.688+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:27:59.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:58.688+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:27:59.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:58.688+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:27:59.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:58 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:58.688+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:27:59.192 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:58.892+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:27:59.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:58.892+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:27:59.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:58.892+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:27:59.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:58.892+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:27:59.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:58 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:58.892+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:27:59.319 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:59 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:59.055+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:27:59.319 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:59 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:59.055+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:27:59.319 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:59 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:59.055+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:27:59.319 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:27:59 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:27:59.055+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:27:59.320 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:59 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:59.279+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:27:59.320 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:59 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:59.279+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:27:59.320 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:27:59 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:27:59.279+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:27:59.503 INFO:journalctl@ceph.osd.7.smithi156.stdout:Oct 06 22:27:59 smithi156 bash[179362]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-7 2023-10-06T22:28:00.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:59.730+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:00.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:59.730+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:00.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:59.730+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:00.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:59.730+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:00.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:27:59 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:27:59.730+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:00.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:59.899+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:00.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:59.899+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:00.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:59.900+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:00.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:59.900+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:00.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:27:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:27:59.900+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:00.314 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:00 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:00.023+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:00.315 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:00 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:00.023+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:00.315 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:00 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:00.023+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:00.315 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:00 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:00.023+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:00.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:00 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:00.313+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:28:00.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:00 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:00.313+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:28:00.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:00 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:00.313+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:28:00.691 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-06T22:28:00.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:00.770 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-06T22:28:00.770 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-06T22:28:00.770 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.8 2023-10-06T22:28:01.003 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:00.896+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:01.003 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:00.896+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:01.003 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:00.896+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:01.004 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:00.896+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:01.004 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:00 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:00.896+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:01.069 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:00 smithi190 systemd[1]: Stopping Ceph osd.8 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:01.070 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:01.047+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:01.070 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:01.047+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:01.070 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:01.047+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:01.070 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:01.047+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:01.172 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:00.710+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:01.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:00.710+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:01.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:00.710+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:01.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:00.710+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:01.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:00 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:00.710+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:01.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:01.166+0000 7f9ec8b14700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:01.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:01.166+0000 7f9ec8b14700 -1 osd.8 103 *** Got signal Terminated *** 2023-10-06T22:28:01.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:01.166+0000 7f9ec8b14700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:28:01.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:01.324+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:28:01.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:01.324+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:28:01.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:01.324+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:28:01.570 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:01 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:01.324+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:34.716244+0000 front 2023-10-06T22:27:34.716303+0000 (oldest deadline 2023-10-06T22:28:00.616126+0000) 2023-10-06T22:28:02.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:01.699+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:02.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:01.699+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:02.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:01.699+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:02.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:01.699+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:02.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:01 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:01.699+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:02.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:01 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:01.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:02.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:01 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:01.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:02.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:01 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:01.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:02.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:01 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:01.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:02.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:01 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:01.867+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:02.354 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:02.094+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:02.354 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:02.094+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:02.354 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:02.094+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:02.354 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:02.094+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:02.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:02.352+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:28:02.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:02.352+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:28:02.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:02.352+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:28:02.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:02 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:02.352+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:34.716244+0000 front 2023-10-06T22:27:34.716303+0000 (oldest deadline 2023-10-06T22:28:00.616126+0000) 2023-10-06T22:28:03.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:02.736+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:03.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:02.736+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:03.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:02.736+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:03.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:02.736+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:03.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:02 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:02.736+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:03.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:02 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:02.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:03.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:02 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:02.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:03.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:02 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:02.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:03.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:02 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:02.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:03.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:02 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:02.898+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:03.401 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:03.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:03.401 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:03.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:03.402 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:03.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:03.402 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:03.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:03.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:03.399+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:28:03.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:03.399+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:28:03.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:03.399+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:28:03.820 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:03.399+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:34.716244+0000 front 2023-10-06T22:27:34.716303+0000 (oldest deadline 2023-10-06T22:28:00.616126+0000) 2023-10-06T22:28:03.820 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:03.399+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.117289+0000 front 2023-10-06T22:27:41.117237+0000 (oldest deadline 2023-10-06T22:28:02.817317+0000) 2023-10-06T22:28:04.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:03.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:04.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:03.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:04.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:03.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:04.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:03.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:04.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:03.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:04.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:03 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:03.759+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:04.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:03 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:03.879+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:04.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:03 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:03.879+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:04.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:03 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:03.879+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:04.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:03 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:03.879+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:04.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:03 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:03.879+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:04.430 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:04.077+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:04.430 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:04.077+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:04.430 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:04.077+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:04.430 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:04.077+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:04.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:04.427+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:28:04.881 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:04.427+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:28:04.881 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:04.427+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:28:04.881 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:04.427+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:34.716244+0000 front 2023-10-06T22:27:34.716303+0000 (oldest deadline 2023-10-06T22:28:00.616126+0000) 2023-10-06T22:28:04.881 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:04 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:04.427+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.117289+0000 front 2023-10-06T22:27:41.117237+0000 (oldest deadline 2023-10-06T22:28:02.817317+0000) 2023-10-06T22:28:05.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:04.742+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:05.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:04.742+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:05.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:04.742+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:05.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:04.742+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:05.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:04.742+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:05.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:04 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:04.742+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:05.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:04.833+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:05.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:04.833+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:05.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:04.833+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:05.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:04.833+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:05.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:04.833+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:05.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:04 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:04.833+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:05.416 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:05.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:05.416 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:05.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:05.416 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:05.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:05.416 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:05.095+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:05.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:05.414+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.812668+0000 front 2023-10-06T22:27:07.812437+0000 (oldest deadline 2023-10-06T22:27:33.712308+0000) 2023-10-06T22:28:05.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:05.414+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.113857+0000 front 2023-10-06T22:27:17.113888+0000 (oldest deadline 2023-10-06T22:27:41.214002+0000) 2023-10-06T22:28:05.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:05.414+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:25.315065+0000 front 2023-10-06T22:27:25.315184+0000 (oldest deadline 2023-10-06T22:27:51.215134+0000) 2023-10-06T22:28:05.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:05.414+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:34.716244+0000 front 2023-10-06T22:27:34.716303+0000 (oldest deadline 2023-10-06T22:28:00.616126+0000) 2023-10-06T22:28:05.819 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:05 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8[156527]: 2023-10-06T22:28:05.414+0000 7f9ec2905700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.117289+0000 front 2023-10-06T22:27:41.117237+0000 (oldest deadline 2023-10-06T22:28:02.817317+0000) 2023-10-06T22:28:06.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:05 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:05.775+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:06.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:05 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:05.775+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:06.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:05 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:05.775+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:06.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:05 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:05.775+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:06.175 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:05 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:05.775+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:06.175 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:05 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:05.775+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:06.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:05.872+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:06.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:05.872+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:06.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:05.872+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:06.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:05.872+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:06.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:05.872+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:06.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:05 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:05.872+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:06.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:06 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:06.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:06.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:06 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:06.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:06.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:06 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:06.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:06.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:06 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:06.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:06.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:06 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:06.123+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:07.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:06 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:06.767+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:07.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:06 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:06.767+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:07.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:06 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:06.767+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:07.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:06 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:06.767+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:07.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:06 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:06.767+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:07.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:06 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:06.767+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:07.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:06 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:06.921+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:07.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:06 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:06.921+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:07.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:06 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:06.921+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:07.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:06 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:06.921+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:07.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:06 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:06.921+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:07.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:06 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:06.921+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:07.569 INFO:journalctl@ceph.osd.8.smithi190.stdout:Oct 06 22:28:07 smithi190 bash[178084]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-8 2023-10-06T22:28:07.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:07 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:07.078+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:07.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:07 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:07.078+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:07.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:07 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:07.078+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:07.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:07 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:07.078+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:07.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:07 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:07.078+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:08.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:07 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:07.731+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:08.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:07 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:07.731+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:08.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:07 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:07.731+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:08.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:07 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:07.731+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:08.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:07 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:07.731+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:08.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:07 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:07.731+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:08.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:07 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:07.922+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:08.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:07 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:07.922+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:08.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:07 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:07.922+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:08.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:07 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:07.922+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:08.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:07 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:07.922+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:08.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:07 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:07.922+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:08.358 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:08 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:08.100+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:08.358 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:08 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:08.100+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:08.358 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:08 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:08.100+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:08.358 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:08 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:08.100+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:08.358 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:08 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:08.100+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:08.576 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.8.service' 2023-10-06T22:28:08.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:08.616 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-06T22:28:08.616 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-06T22:28:08.616 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.9 2023-10-06T22:28:09.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:08 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:08.700+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:09.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:08 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:08.700+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:09.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:08 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:08.700+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:09.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:08 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:08.700+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:09.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:08 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:08.700+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:09.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:08 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:08.700+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:09.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:08 smithi057 systemd[1]: Stopping Ceph osd.9 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:09.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:08 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:08.944+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:09.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:08 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:08.944+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:09.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:08 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:08.944+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:09.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:08 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:08.944+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:09.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:08 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:08.944+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:09.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:08 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:08.944+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:09.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:09 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:09.139+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:09.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:09 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:09.139+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:09.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:09 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:09.139+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:09.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:09 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:09.139+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:09.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:09 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:09.139+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:09.673 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.252+0000 7fdb19eae700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:09.673 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.252+0000 7fdb19eae700 -1 osd.9 103 *** Got signal Terminated *** 2023-10-06T22:28:09.673 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.252+0000 7fdb19eae700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:28:10.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.694+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:10.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.694+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:10.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.694+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:10.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.694+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:10.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.694+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:10.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:09 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:09.694+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:10.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:09 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:09.937+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:10.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:09 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:09.937+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:10.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:09 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:09.937+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:10.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:09 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:09.937+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:10.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:09 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:09.937+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:10.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:09 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:09.937+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:10.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:10.141+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:10.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:10.141+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:10.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:10.141+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:10.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:10.141+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:10.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:10 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:10.141+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:11.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:10.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:11.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:10.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:11.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:10.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:11.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:10.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:11.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:10.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:11.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:10 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:10.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:11.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:10 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:10.946+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:11.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:10 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:10.946+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:11.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:10 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:10.946+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:11.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:10 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:10.946+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:11.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:10 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:10.946+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:11.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:10 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:10.946+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:11.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:11.168+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:11.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:11.168+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:11.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:11.168+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:11.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:11.168+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:11.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:11.168+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:12.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:11 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:11.734+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:12.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:11 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:11.734+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:12.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:11 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:11.734+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:12.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:11 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:11.734+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:12.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:11 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:11.734+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:12.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:11 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:11.734+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:12.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:11.924+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:12.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:11.924+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:12.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:11.924+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:12.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:11.924+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:12.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:11.924+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:12.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:11.924+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:12.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:11 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:11.924+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:12.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:12.158+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:12.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:12.158+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:12.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:12.158+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:12.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:12.158+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:12.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:12.158+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:12.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:12 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:12.158+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:13.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:12 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:12.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:13.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:12 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:12.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:13.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:12 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:12.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:13.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:12 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:12.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:13.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:12 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:12.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:13.173 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:12 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:12.690+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:13.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:12.903+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:13.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:12.903+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:13.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:12.903+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:13.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:12.903+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:13.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:12.903+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:13.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:12.903+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:13.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:12 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:12.903+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:13.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:13.133+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:13.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:13.133+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:13.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:13.133+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:13.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:13.133+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:13.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:13.133+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:13.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:13 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:13.133+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:14.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:13.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:27:01.194972+0000 front 2023-10-06T22:27:01.195066+0000 (oldest deadline 2023-10-06T22:27:25.895246+0000) 2023-10-06T22:28:14.175 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:13.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:05.895947+0000 front 2023-10-06T22:27:05.895968+0000 (oldest deadline 2023-10-06T22:27:36.496812+0000) 2023-10-06T22:28:14.175 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:13.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:16.497588+0000 front 2023-10-06T22:27:16.497654+0000 (oldest deadline 2023-10-06T22:27:41.797468+0000) 2023-10-06T22:28:14.175 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:13.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:24.098662+0000 front 2023-10-06T22:27:24.098689+0000 (oldest deadline 2023-10-06T22:27:47.598912+0000) 2023-10-06T22:28:14.175 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:13.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:33.500471+0000 front 2023-10-06T22:27:33.500579+0000 (oldest deadline 2023-10-06T22:27:59.400594+0000) 2023-10-06T22:28:14.175 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:13 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9[141905]: 2023-10-06T22:28:13.709+0000 7fdb16ca5700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:39.401402+0000 front 2023-10-06T22:27:39.401338+0000 (oldest deadline 2023-10-06T22:28:02.901157+0000) 2023-10-06T22:28:14.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:13.950+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:14.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:13.950+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:14.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:13.950+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:14.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:13.950+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:14.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:13.950+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:14.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:13.950+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:14.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:13 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:13.950+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:14.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:14.115+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:14.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:14.115+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:14.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:14.115+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:14.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:14.115+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:14.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:14.115+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:14.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:14 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:14.115+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:15.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:14.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:15.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:14.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:15.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:14.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:15.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:14.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:15.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:14.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:15.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:14.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:15.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:14 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:14.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:15.423 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 06 22:28:15 smithi057 bash[182073]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-9 2023-10-06T22:28:15.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:15.103+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:15.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:15.103+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:15.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:15.103+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:15.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:15.103+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:15.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:15.103+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:15.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:15.103+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:16.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:15.930+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:16.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:15.930+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:16.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:15.930+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:16.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:15.930+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:16.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:15.930+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:16.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:15.930+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:16.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:15 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:15.930+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:16.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:16.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:16.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:16.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:16.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:16.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:16.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:16.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:16.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:16.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:16.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:16 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:16.147+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:16.998 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.9.service' 2023-10-06T22:28:17.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:17.090 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-06T22:28:17.090 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-06T22:28:17.090 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.10 2023-10-06T22:28:17.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:16.911+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:17.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:16.911+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:17.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:16.911+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:17.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:16.911+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:17.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:16.911+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:17.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:16.911+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:17.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:16 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:16.911+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:17.255 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 systemd[1]: Stopping Ceph osd.10 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:17.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:17 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:17.122+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:17.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:17 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:17.122+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:17.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:17 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:17.122+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:17.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:17 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:17.122+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:17.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:17 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:17.122+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:17.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:17 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:17.122+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:17.753 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.445+0000 7f4bb155b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:17.753 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.445+0000 7f4bb155b700 -1 osd.10 103 *** Got signal Terminated *** 2023-10-06T22:28:17.753 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.445+0000 7f4bb155b700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:28:18.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.920+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:18.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.920+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:18.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.920+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:18.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.920+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:18.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.920+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:18.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.920+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:18.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:17 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:17.920+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:18.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:18 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:18.155+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:18.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:18 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:18.155+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:18.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:18 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:18.155+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:18.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:18 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:18.155+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:18.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:18 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:18.155+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:18.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:18 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:18.155+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:19.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:18.923+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:19.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:18.923+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:19.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:18.923+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:19.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:18.923+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:19.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:18.923+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:19.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:18.923+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:19.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:18 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:18.923+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:19.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:19 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:19.185+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:19.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:19 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:19.185+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:19.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:19 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:19.185+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:19.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:19 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:19.185+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:19.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:19 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:19.185+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:19.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:19 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:19.185+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:20.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:19.935+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:20.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:19.935+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:20.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:19.935+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:20.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:19.935+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:20.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:19.935+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:20.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:19.935+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:20.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:19 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:19.935+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:20.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:20 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:20.233+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:20.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:20 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:20.233+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:20.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:20 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:20.233+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:20.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:20 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:20.233+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:20.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:20 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:20.233+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:20.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:20 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:20.233+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:21.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:20.974+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:21.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:20.974+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:21.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:20.974+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:21.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:20.974+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:21.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:20.974+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:21.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:20.974+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:21.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:20 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:20.974+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:21.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:21 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:21.268+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:21.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:21 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:21.268+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:21.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:21 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:21.268+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:21.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:21 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:21.268+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:21.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:21 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:21.268+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:21.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:21 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:21.268+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:22.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:21.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:57.912660+0000 front 2023-10-06T22:26:57.912741+0000 (oldest deadline 2023-10-06T22:27:27.313038+0000) 2023-10-06T22:28:22.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:21.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:07.814142+0000 front 2023-10-06T22:27:07.814211+0000 (oldest deadline 2023-10-06T22:27:33.714228+0000) 2023-10-06T22:28:22.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:21.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:17.716076+0000 front 2023-10-06T22:27:17.715876+0000 (oldest deadline 2023-10-06T22:27:43.615966+0000) 2023-10-06T22:28:22.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:21.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-06T22:27:23.616843+0000 front 2023-10-06T22:27:23.616823+0000 (oldest deadline 2023-10-06T22:27:48.316515+0000) 2023-10-06T22:28:22.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:21.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.918193+0000 front 2023-10-06T22:27:32.918235+0000 (oldest deadline 2023-10-06T22:27:58.218391+0000) 2023-10-06T22:28:22.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:21.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.119534+0000 front 2023-10-06T22:27:41.119626+0000 (oldest deadline 2023-10-06T22:28:04.619575+0000) 2023-10-06T22:28:22.254 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:21 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10[164786]: 2023-10-06T22:28:21.960+0000 7f4badb51700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:44.620213+0000 front 2023-10-06T22:27:44.620443+0000 (oldest deadline 2023-10-06T22:28:11.620591+0000) 2023-10-06T22:28:22.819 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:22.318+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:22.820 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:22.318+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:22.820 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:22.318+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:22.820 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:22.318+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:22.820 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:22.318+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:22.820 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:22 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:22.318+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:23.253 INFO:journalctl@ceph.osd.10.smithi156.stdout:Oct 06 22:28:23 smithi156 bash[179640]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-10 2023-10-06T22:28:23.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:23 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:23.275+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:23.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:23 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:23.275+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:23.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:23 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:23.275+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:23.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:23 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:23.275+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:23.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:23 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:23.275+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:23.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:23 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:23.275+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:24.329 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.10.service' 2023-10-06T22:28:24.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:24.370 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-06T22:28:24.370 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-06T22:28:24.370 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.11 2023-10-06T22:28:24.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:24.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:24.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:24.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:24.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:24.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:24.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2023-10-06T22:27:57.249394+0000 front 2023-10-06T22:27:57.249321+0000 (oldest deadline 2023-10-06T22:28:23.649736+0000) 2023-10-06T22:28:24.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 systemd[1]: Stopping Ceph osd.11 for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:25.069 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.737+0000 7fd0a9659700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:25.069 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.737+0000 7fd0a9659700 -1 osd.11 103 *** Got signal Terminated *** 2023-10-06T22:28:25.069 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:24 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:24.737+0000 7fd0a9659700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T22:28:25.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:25.225+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:25.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:25.225+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:25.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:25.225+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:25.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:25.225+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:25.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:25.225+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:25.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:25.225+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:25.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:25 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:25.225+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2023-10-06T22:27:57.249394+0000 front 2023-10-06T22:27:57.249321+0000 (oldest deadline 2023-10-06T22:28:23.649736+0000) 2023-10-06T22:28:26.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:26.255+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:26.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:26.255+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:26.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:26.255+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:26.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:26.255+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:26.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:26.255+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:26.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:26.255+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:26.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:26 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:26.255+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2023-10-06T22:27:57.249394+0000 front 2023-10-06T22:27:57.249321+0000 (oldest deadline 2023-10-06T22:28:23.649736+0000) 2023-10-06T22:28:27.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:27.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:27.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:27.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:27.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:27.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:27.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:27.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:27.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:27.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:27.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:27.258+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:27.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:27 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:27.259+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2023-10-06T22:27:57.249394+0000 front 2023-10-06T22:27:57.249321+0000 (oldest deadline 2023-10-06T22:28:23.649736+0000) 2023-10-06T22:28:28.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:28.295+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:28.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:28.295+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:28.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:28.295+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:28.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:28.295+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:28.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:28.295+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:28.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:28.295+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:28.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:28 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:28.295+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2023-10-06T22:27:57.249394+0000 front 2023-10-06T22:27:57.249321+0000 (oldest deadline 2023-10-06T22:28:23.649736+0000) 2023-10-06T22:28:29.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-06T22:26:59.540757+0000 front 2023-10-06T22:26:59.540745+0000 (oldest deadline 2023-10-06T22:27:23.640460+0000) 2023-10-06T22:28:29.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2023-10-06T22:27:08.842006+0000 front 2023-10-06T22:27:08.841948+0000 (oldest deadline 2023-10-06T22:27:32.341872+0000) 2023-10-06T22:28:29.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-10-06T22:27:18.143241+0000 front 2023-10-06T22:27:18.143224+0000 (oldest deadline 2023-10-06T22:27:42.243435+0000) 2023-10-06T22:28:29.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2023-10-06T22:27:32.844865+0000 front 2023-10-06T22:27:32.845005+0000 (oldest deadline 2023-10-06T22:27:56.944944+0000) 2023-10-06T22:28:29.569 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-10-06T22:27:41.546228+0000 front 2023-10-06T22:27:41.546255+0000 (oldest deadline 2023-10-06T22:28:05.646444+0000) 2023-10-06T22:28:29.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-06T22:27:45.647080+0000 front 2023-10-06T22:27:45.647020+0000 (oldest deadline 2023-10-06T22:28:11.447526+0000) 2023-10-06T22:28:29.570 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:29 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11[164677]: 2023-10-06T22:28:29.296+0000 7fd0a344a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2023-10-06T22:27:57.249394+0000 front 2023-10-06T22:27:57.249321+0000 (oldest deadline 2023-10-06T22:28:23.649736+0000) 2023-10-06T22:28:30.413 INFO:journalctl@ceph.osd.11.smithi190.stdout:Oct 06 22:28:30 smithi190 bash[178352]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-osd-11 2023-10-06T22:28:31.775 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@osd.11.service' 2023-10-06T22:28:31.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:31.814 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-06T22:28:31.814 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-06T22:28:31.814 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.a 2023-10-06T22:28:32.173 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:28:31 smithi057 systemd[1]: Stopping Ceph mds.a for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:32.673 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:28:32 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-a[150435]: 2023-10-06T22:28:32.315+0000 7ff3105ac700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:32.673 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:28:32 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-a[150435]: 2023-10-06T22:28:32.315+0000 7ff3105ac700 -1 mds.a *** got signal Terminated *** 2023-10-06T22:28:35.172 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 06 22:28:34 smithi057 bash[182346]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-a 2023-10-06T22:28:35.269 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.a.service' 2023-10-06T22:28:35.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:35.356 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-06T22:28:35.357 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-06T22:28:35.357 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.d 2023-10-06T22:28:35.673 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:28:35 smithi057 systemd[1]: Stopping Ceph mds.d for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:36.173 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:28:35 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-d[149425]: 2023-10-06T22:28:35.840+0000 7f449be84700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:36.173 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:28:35 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-d[149425]: 2023-10-06T22:28:35.840+0000 7f449be84700 -1 mds.d *** got signal Terminated *** 2023-10-06T22:28:39.153 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 06 22:28:38 smithi057 bash[182492]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-d 2023-10-06T22:28:39.179 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.d.service' 2023-10-06T22:28:39.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:39.263 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-06T22:28:39.264 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-06T22:28:39.264 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.g 2023-10-06T22:28:39.422 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:28:39 smithi057 systemd[1]: Stopping Ceph mds.g for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:40.173 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:28:39 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-g[150047]: 2023-10-06T22:28:39.755+0000 7f156bc64700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:40.173 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:28:39 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-g[150047]: 2023-10-06T22:28:39.755+0000 7f156bc64700 -1 mds.g *** got signal Terminated *** 2023-10-06T22:28:43.173 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 06 22:28:42 smithi057 bash[182638]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-g 2023-10-06T22:28:43.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.g.service' 2023-10-06T22:28:43.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:43.379 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-06T22:28:43.379 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-06T22:28:43.379 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.j 2023-10-06T22:28:43.673 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:28:43 smithi057 systemd[1]: Stopping Ceph mds.j for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:44.173 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:28:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-j[149040]: 2023-10-06T22:28:43.863+0000 7fb566a2c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:44.173 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:28:43 smithi057 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-j[149040]: 2023-10-06T22:28:43.863+0000 7fb566a2c700 -1 mds.j *** got signal Terminated *** 2023-10-06T22:28:47.818 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:28:47 smithi057 bash[182782]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-j 2023-10-06T22:28:48.026 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.j.service' 2023-10-06T22:28:48.087 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:28:48 smithi057 systemd[1]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.j.service: Succeeded. 2023-10-06T22:28:48.087 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 06 22:28:48 smithi057 systemd[1]: Stopped Ceph mds.j for 495b2048-6494-11ee-8db5-212e2dc638e7. 2023-10-06T22:28:48.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:48.111 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-06T22:28:48.111 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-06T22:28:48.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.b 2023-10-06T22:28:48.468 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:28:48 smithi156 systemd[1]: Stopping Ceph mds.b for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:48.764 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:28:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-b[171928]: 2023-10-06T22:28:48.465+0000 7f36c08df700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:48.764 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:28:48 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-b[171928]: 2023-10-06T22:28:48.465+0000 7f36c08df700 -1 mds.b *** got signal Terminated *** 2023-10-06T22:28:50.931 INFO:journalctl@ceph.mds.b.smithi156.stdout:Oct 06 22:28:50 smithi156 bash[179907]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-b 2023-10-06T22:28:51.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.b.service' 2023-10-06T22:28:51.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:51.157 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-06T22:28:51.158 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-06T22:28:51.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e 2023-10-06T22:28:51.503 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:28:51 smithi156 systemd[1]: Stopping Ceph mds.e for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:52.003 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:28:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[177384]: 2023-10-06T22:28:51.548+0000 7f7d70c4c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:52.003 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:28:51 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e[177384]: 2023-10-06T22:28:51.548+0000 7f7d70c4c700 -1 mds.e *** got signal Terminated *** 2023-10-06T22:28:54.906 INFO:journalctl@ceph.mds.e.smithi156.stdout:Oct 06 22:28:54 smithi156 bash[180051]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-e 2023-10-06T22:28:55.093 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.e.service' 2023-10-06T22:28:55.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:55.128 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-06T22:28:55.128 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-06T22:28:55.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.h 2023-10-06T22:28:55.503 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:28:55 smithi156 systemd[1]: Stopping Ceph mds.h for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:28:56.003 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:28:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:28:55.504+0000 7f4bdea9c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:28:56.003 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:28:55 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h[171589]: 2023-10-06T22:28:55.504+0000 7f4bdea9c700 -1 mds.h *** got signal Terminated *** 2023-10-06T22:28:59.101 INFO:journalctl@ceph.mds.h.smithi156.stdout:Oct 06 22:28:58 smithi156 bash[180191]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-h 2023-10-06T22:28:59.106 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.h.service' 2023-10-06T22:28:59.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:28:59.140 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-06T22:28:59.140 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-06T22:28:59.140 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.k 2023-10-06T22:28:59.503 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:28:59 smithi156 systemd[1]: Stopping Ceph mds.k for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:29:00.003 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:28:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-k[170906]: 2023-10-06T22:28:59.512+0000 7f4b5bfeb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:29:00.003 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:28:59 smithi156 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-k[170906]: 2023-10-06T22:28:59.512+0000 7f4b5bfeb700 -1 mds.k *** got signal Terminated *** 2023-10-06T22:29:03.105 INFO:journalctl@ceph.mds.k.smithi156.stdout:Oct 06 22:29:02 smithi156 bash[180333]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-k 2023-10-06T22:29:03.194 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.k.service' 2023-10-06T22:29:03.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:29:03.227 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-06T22:29:03.228 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-06T22:29:03.228 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.c 2023-10-06T22:29:03.569 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:29:03 smithi190 systemd[1]: Stopping Ceph mds.c for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:29:04.069 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:29:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c[170889]: 2023-10-06T22:29:03.672+0000 7f6103316700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:29:04.069 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:29:03 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c[170889]: 2023-10-06T22:29:03.672+0000 7f6103316700 -1 mds.c *** got signal Terminated *** 2023-10-06T22:29:07.036 INFO:journalctl@ceph.mds.c.smithi190.stdout:Oct 06 22:29:06 smithi190 bash[178627]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-c 2023-10-06T22:29:07.042 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.c.service' 2023-10-06T22:29:07.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:29:07.078 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-06T22:29:07.079 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-06T22:29:07.079 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.f 2023-10-06T22:29:07.319 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:29:07 smithi190 systemd[1]: Stopping Ceph mds.f for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:29:07.819 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:29:07 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:29:07.463+0000 7fb7bf68a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:29:07.819 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:29:07 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f[170549]: 2023-10-06T22:29:07.463+0000 7fb7bf68a700 -1 mds.f *** got signal Terminated *** 2023-10-06T22:29:10.930 INFO:journalctl@ceph.mds.f.smithi190.stdout:Oct 06 22:29:10 smithi190 bash[178765]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-f 2023-10-06T22:29:11.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.f.service' 2023-10-06T22:29:11.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:29:11.148 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-06T22:29:11.149 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-06T22:29:11.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.i 2023-10-06T22:29:11.531 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:29:11 smithi190 systemd[1]: Stopping Ceph mds.i for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:29:11.819 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:29:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: 2023-10-06T22:29:11.528+0000 7fe744450700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:29:11.819 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:29:11 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i[171229]: 2023-10-06T22:29:11.529+0000 7fe744450700 -1 mds.i *** got signal Terminated *** 2023-10-06T22:29:14.895 INFO:journalctl@ceph.mds.i.smithi190.stdout:Oct 06 22:29:14 smithi190 bash[178904]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-i 2023-10-06T22:29:15.080 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.i.service' 2023-10-06T22:29:15.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:29:15.116 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-06T22:29:15.116 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-06T22:29:15.116 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.l 2023-10-06T22:29:15.496 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:29:15 smithi190 systemd[1]: Stopping Ceph mds.l for 495b2048-6494-11ee-8db5-212e2dc638e7... 2023-10-06T22:29:15.819 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:29:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-l[170205]: 2023-10-06T22:29:15.494+0000 7f3952cff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T22:29:15.819 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:29:15 smithi190 ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-l[170205]: 2023-10-06T22:29:15.494+0000 7f3952cff700 -1 mds.l *** got signal Terminated *** 2023-10-06T22:29:19.151 INFO:journalctl@ceph.mds.l.smithi190.stdout:Oct 06 22:29:18 smithi190 bash[179041]: ceph-495b2048-6494-11ee-8db5-212e2dc638e7-mds-l 2023-10-06T22:29:19.273 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-495b2048-6494-11ee-8db5-212e2dc638e7@mds.l.service' 2023-10-06T22:29:19.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T22:29:19.307 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-06T22:29:19.307 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T22:29:19.711 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:29:23.105 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T22:29:23.413 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:29:25.024 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T22:29:25.330 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:29:26.952 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:29:27.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:29:27.069 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T22:29:27.099 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-06T22:29:27.101 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi057/crash 2023-10-06T22:29:27.102 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T22:29:27.173 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T22:29:27.173 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now 2023-10-06T22:29:27.177 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi156/crash 2023-10-06T22:29:27.177 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T22:29:27.225 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T22:29:27.226 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2023-10-06T22:29:27.227 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi190/crash 2023-10-06T22:29:27.228 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T22:29:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T22:29:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2023-10-06T22:29:27.293 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-06T22:29:27.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-06T22:29:27.378 INFO:teuthology.orchestra.run.smithi057.stderr:grep: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-06T22:29:27.382 INFO:tasks.cephadm:Compressing logs... 2023-10-06T22:29:27.382 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T22:29:27.387 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T22:29:27.389 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T22:29:27.431 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T22:29:27.431 INFO:teuthology.orchestra.run.smithi190.stderr:: No such file or directory 2023-10-06T22:29:27.434 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2023-10-06T22:29:27.435 INFO:teuthology.orchestra.run.smithi156.stderr:-5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T22:29:27.440 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-06T22:29:27.443 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-06T22:29:27.443 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-10-06T22:29:27.443 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-10-06T22:29:27.443 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.2.log: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-10-06T22:29:27.443 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-10-06T22:29:27.443 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-10-06T22:29:27.443 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-10-06T22:29:27.444 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-06T22:29:27.444 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.c.log: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T22:29:27.444 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-10-06T22:29:27.445 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-10-06T22:29:27.445 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-10-06T22:29:27.445 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-10-06T22:29:27.445 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-06T22:29:27.445 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-06T22:29:27.447 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-10-06T22:29:27.447 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T22:29:27.447 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-06T22:29:27.447 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-06T22:29:27.447 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-10-06T22:29:27.448 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-06T22:29:27.448 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.l.log: 92.0% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-06T22:29:27.451 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T22:29:27.451 INFO:teuthology.orchestra.run.smithi057.stderr:: No such file or directory 2023-10-06T22:29:27.452 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T22:29:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-06T22:29:27.452 INFO:teuthology.orchestra.run.smithi156.stderr: 91.9% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-06T22:29:27.453 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-10-06T22:29:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-10-06T22:29:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-06T22:29:27.457 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-06T22:29:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-10-06T22:29:27.460 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.3.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T22:29:27.460 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-06T22:29:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.d.log: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-10-06T22:29:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-10-06T22:29:27.468 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T22:29:27.477 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.9.log: /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-10-06T22:29:27.482 INFO:teuthology.orchestra.run.smithi057.stderr: 91.0% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T22:29:27.491 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-06T22:29:27.491 INFO:teuthology.orchestra.run.smithi057.stderr: 94.3% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-10-06T22:29:27.555 INFO:teuthology.orchestra.run.smithi190.stderr: 91.0% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T22:29:27.820 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.a.log: 89.8% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-06T22:29:28.071 INFO:teuthology.orchestra.run.smithi156.stderr: 91.8% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-06T22:29:28.338 INFO:teuthology.orchestra.run.smithi156.stderr: 91.0% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T22:29:28.345 INFO:teuthology.orchestra.run.smithi190.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T22:29:28.861 INFO:teuthology.orchestra.run.smithi057.stderr: 90.4% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-06T22:29:28.902 INFO:teuthology.orchestra.run.smithi190.stderr: 91.8% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-06T22:29:31.857 INFO:teuthology.orchestra.run.smithi057.stderr: 95.2% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-10-06T22:29:31.865 INFO:teuthology.orchestra.run.smithi057.stderr: 94.9% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-10-06T22:29:37.583 INFO:teuthology.orchestra.run.smithi057.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-10-06T22:29:39.840 INFO:teuthology.orchestra.run.smithi057.stderr: 93.7% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-06T22:29:40.889 INFO:teuthology.orchestra.run.smithi156.stderr: 95.4% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-10-06T22:29:41.098 INFO:teuthology.orchestra.run.smithi057.stderr: 93.5% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-06T22:29:41.633 INFO:teuthology.orchestra.run.smithi057.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-10-06T22:29:41.666 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-06T22:29:41.666 INFO:teuthology.orchestra.run.smithi057.stderr:real 0m14.213s 2023-10-06T22:29:41.666 INFO:teuthology.orchestra.run.smithi057.stderr:user 0m56.065s 2023-10-06T22:29:41.667 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m4.709s 2023-10-06T22:29:46.022 INFO:teuthology.orchestra.run.smithi156.stderr: 95.1% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-10-06T22:29:47.463 INFO:teuthology.orchestra.run.smithi156.stderr: 93.9% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-06T22:29:47.853 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-10-06T22:29:49.319 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-06T22:29:49.911 INFO:teuthology.orchestra.run.smithi190.stderr: 95.4% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-10-06T22:29:50.261 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-10-06T22:29:51.206 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-06T22:29:51.521 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-10-06T22:29:52.189 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-06T22:29:55.070 INFO:teuthology.orchestra.run.smithi190.stderr: 93.5% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-06T22:31:01.148 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-10-06T22:31:21.112 INFO:teuthology.orchestra.run.smithi190.stderr: 95.4% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-10-06T22:31:31.242 INFO:teuthology.orchestra.run.smithi156.stderr: 95.6% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-10-06T22:31:48.246 INFO:teuthology.orchestra.run.smithi190.stderr: 94.9% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-10-06T22:31:48.327 INFO:teuthology.orchestra.run.smithi190.stderr: 2023-10-06T22:31:48.327 INFO:teuthology.orchestra.run.smithi190.stderr:real 2m20.922s 2023-10-06T22:31:48.327 INFO:teuthology.orchestra.run.smithi190.stderr:user 4m29.923s 2023-10-06T22:31:48.327 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m11.123s 2023-10-06T22:33:08.479 INFO:teuthology.orchestra.run.smithi156.stderr: 94.8% -- replaced with /var/log/ceph/495b2048-6494-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-10-06T22:33:08.604 INFO:teuthology.orchestra.run.smithi156.stderr: 2023-10-06T22:33:08.604 INFO:teuthology.orchestra.run.smithi156.stderr:real 3m41.198s 2023-10-06T22:33:08.604 INFO:teuthology.orchestra.run.smithi156.stderr:user 4m59.474s 2023-10-06T22:33:08.604 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m12.930s 2023-10-06T22:33:08.604 INFO:tasks.cephadm:Archiving logs... 2023-10-06T22:33:08.606 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi057/log 2023-10-06T22:33:08.607 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T22:33:15.795 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi156/log 2023-10-06T22:33:15.796 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T22:33:56.106 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi190/log 2023-10-06T22:33:56.107 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T22:34:32.281 INFO:tasks.cephadm:Removing cluster... 2023-10-06T22:34:32.282 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 --force 2023-10-06T22:34:32.685 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:34:33.576 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 --force 2023-10-06T22:34:34.609 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:34:35.384 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 495b2048-6494-11ee-8db5-212e2dc638e7 --force 2023-10-06T22:34:36.229 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: 495b2048-6494-11ee-8db5-212e2dc638e7 2023-10-06T22:34:36.993 INFO:tasks.cephadm:Removing cephadm ... 2023-10-06T22:34:36.993 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T22:34:37.043 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T22:34:37.075 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T22:34:37.093 INFO:tasks.cephadm:Teardown complete 2023-10-06T22:34:37.093 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T22:34:37.104 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T22:34:37.104 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T22:34:37.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T22:34:37.118 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T22:34:37.266 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T22:34:37.278 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T22:34:37.278 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T22:34:37.283 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T22:34:37.285 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T22:34:37.301 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-10-06T22:34:37.302 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-10-06T22:34:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-10-06T22:34:37.328 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T22:34:37.328 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-10-06T22:34:37.328 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 -766us[ -760us] +/- 91ms 2023-10-06T22:34:37.328 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 +402us[ +408us] +/- 69ms 2023-10-06T22:34:37.328 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +170us[ +176us] +/- 72ms 2023-10-06T22:34:37.328 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:34:37.329 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T22:34:37.329 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-10-06T22:34:37.329 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 60 -717us[ -717us] +/- 91ms 2023-10-06T22:34:37.329 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 +437us[ +457us] +/- 69ms 2023-10-06T22:34:37.329 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +214us[ +214us] +/- 72ms 2023-10-06T22:34:37.329 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:34:37.331 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T22:34:37.332 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-10-06T22:34:37.332 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 -750us[ -728us] +/- 91ms 2023-10-06T22:34:37.332 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 +405us[ +428us] +/- 69ms 2023-10-06T22:34:37.332 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +176us[ +198us] +/- 72ms 2023-10-06T22:34:37.332 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T22:34:37.333 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T22:34:37.344 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T22:34:37.344 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T22:34:37.381 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T22:34:37.562 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T22:34:37.706 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T22:34:37.835 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T22:34:37.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:34:37.913 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T22:34:37.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:34:37.949 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T22:34:37.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:34:37.981 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T22:34:37.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T22:34:38.039 INFO:teuthology.task.internal:Duration was 2791.320657 seconds 2023-10-06T22:34:38.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T22:34:38.089 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T22:34:38.089 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T22:34:38.093 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T22:34:38.096 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T22:34:38.163 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T22:34:38.177 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T22:34:38.209 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T22:34:38.621 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T22:34:38.621 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2023-10-06T22:34:38.621 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T22:34:38.681 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-10-06T22:34:38.681 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T22:34:38.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2023-10-06T22:34:38.719 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T22:34:38.749 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T22:34:38.750 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T22:34:38.754 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T22:34:38.761 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T22:34:38.852 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T22:34:38.853 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T22:34:38.857 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T22:34:38.859 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T22:34:39.055 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T22:34:39.067 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T22:34:39.067 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T22:34:39.137 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T22:34:39.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T22:34:39.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T22:34:39.232 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T22:34:39.236 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T22:34:39.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T22:34:39.274 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-10-06T22:34:39.290 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2023-10-06T22:34:39.300 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2023-10-06T22:34:39.344 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T22:34:39.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:34:39.386 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T22:34:39.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:34:39.402 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T22:34:39.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T22:34:39.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T22:34:39.431 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T22:34:39.432 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi057 2023-10-06T22:34:39.432 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T22:34:39.516 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi156 2023-10-06T22:34:39.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T22:34:39.555 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695/remote/smithi190 2023-10-06T22:34:39.556 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T22:34:39.595 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T22:34:39.596 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T22:34:39.601 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T22:34:39.603 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T22:34:39.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T22:34:39.660 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T22:34:39.660 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T22:34:39.682 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T22:34:39.682 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T22:34:39.687 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T22:34:39.689 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T22:34:39.703 INFO:teuthology.orchestra.run.smithi156.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 22:34 /home/ubuntu/cephtest 2023-10-06T22:34:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 22:34 /home/ubuntu/cephtest 2023-10-06T22:34:39.725 INFO:teuthology.orchestra.run.smithi057.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 22:34 /home/ubuntu/cephtest 2023-10-06T22:34:39.729 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T22:34:39.741 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T22:34:39.832 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T22:34:39.858 DEBUG:teuthology.nuke:shortname: smithi057 2023-10-06T22:34:39.859 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T22:34:39.885 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 21:37:23.215983', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T22:34:39.891 DEBUG:teuthology.nuke:shortname: smithi156 2023-10-06T22:34:39.891 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T22:34:39.918 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 21:37:23.212972', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T22:34:39.922 DEBUG:teuthology.nuke:shortname: smithi190 2023-10-06T22:34:39.922 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T22:34:39.934 INFO:teuthology.orchestra.console.smithi057:Power off 2023-10-06T22:34:39.935 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T22:34:39.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 21:37:23.214517', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T22:34:39.962 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2023-10-06T22:34:39.963 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:39.982 INFO:teuthology.orchestra.console.smithi156:Power off 2023-10-06T22:34:39.982 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T22:34:39.994 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-06T22:34:40.003 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-10-06T22:34:40.003 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:40.025 INFO:teuthology.orchestra.console.smithi190:Power off 2023-10-06T22:34:40.025 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T22:34:40.035 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-10-06T22:34:40.046 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-10-06T22:34:40.047 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:40.069 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-10-06T22:34:43.995 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:44.036 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:44.070 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:44.105 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-06T22:34:44.145 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-10-06T22:34:44.180 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-10-06T22:34:48.110 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:48.146 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:48.181 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:48.219 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-10-06T22:34:48.256 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-10-06T22:34:48.290 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-10-06T22:34:48.320 INFO:teuthology.orchestra.console.smithi057:Power off completed 2023-10-06T22:34:48.356 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-10-06T22:34:52.295 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T22:34:52.404 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-10-06T22:34:52.505 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-10-06T22:34:52.610 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} duration: 2791.320656776428 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-10-06T22:34:52.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T22:34:52.689 INFO:teuthology.run:FAIL